Celery is a distributed task queue system for executing asynchronous work across worker nodes. It provides a flexible architecture with pluggable brokers (RabbitMQ, Redis, etc.), result backends, and concurrency models (prefork, eventlet, gevent). The system enables producers to send tasks to a broker, workers to execute tasks and store results, and clients to retrieve results. Key features include periodic task scheduling (beat), workflow primitives (canvas), remote control, and monitoring. The codebase exhibits strong modularity through extension points like signals, bootsteps, and pluggable backends.
pgflow is a PostgreSQL/Supabase-native workflow orchestration engine for defining and executing typed DAGs as background jobs. The system is split into six coordinated packages: a PL/pgSQL core (pkgs/core) that implements the complete workflow state machine in the database — flows, steps, runs, step states, tasks, retries with exponential backoff, map/reduce fan-out, conditional step execution, skip-cascade propagation, and realtime event broadcasting — plus a type-safe TypeScript DSL (pkgs/dsl) for declaring flows with compile-time-enforced inputs/outputs, a Deno/edge worker runtime (pkgs/edge-worker) that consumes PGMQ messages and executes step handlers via platform adapters (Supabase vs. plain process), a realtime-subscription client library (pkgs/client), and a CLI (pkgs/cli) for scaffolding new Supabase projects. Execution relies on pgmq for task queues, pg_cron for stalled-task recovery and worker health, pg_net for HTTP worker invocation, supabase_vault for credentials, and Supabase Realtime for atomic state-transition broadcasts.