$ account mage-ai
Mage is an open-source data pipeline orchestration platform that combines a visual notebook-style editor with a production-grade scheduling and execution engine. The system enables users to build, run, and monitor data pipelines composed of Python, SQL, R, and dbt blocks. Architecturally, it is a full-stack web application with a Python backend (Tornado-based server, SQLAlchemy ORM, REST API) and a Next.js/React frontend with a custom component library. Key entry points include the CLI (`mage` command), the web UI served by `mage_ai/server/server.py`, and the API layer at `mage_ai/api/` which exposes CRUD resources for pipelines, blocks, runs, schedules, users, and integrations. Data flows center around pipeline execution: blocks are connected in a DAG, each block can be a data loader, transformer, or exporter. Execution is handled by a configurable executor (local, K8s, EMR, PySpark) and scheduled via a built-in scheduler or external triggers. The platform integrates with dozens of data sources and destinations through a Singer-based integration framework, supports streaming pipelines, and includes advanced features like RBAC, version control, AI-assisted pipeline generation, and a command center for global search. It depends heavily on external services such as cloud storage (S3, GCS, Azure), databases (PostgreSQL, BigQuery, Snowflake, etc.), and authentication providers (OAuth, LDAP, OIDC).