Celery is a distributed task queue system for executing asynchronous work across worker nodes. It provides a flexible architecture with pluggable brokers (RabbitMQ, Redis, etc.), result backends, and concurrency models (prefork, eventlet, gevent). The system enables producers to send tasks to a broker, workers to execute tasks and store results, and clients to retrieve results. Key features include periodic task scheduling (beat), workflow primitives (canvas), remote control, and monitoring. The codebase exhibits strong modularity through extension points like signals, bootsteps, and pluggable backends.
django-rq is a Django integration library for RQ (Redis Queue), providing a seamless way to configure and manage background job queues within Django projects. It allows developers to define queues in Django's settings.py, enqueue jobs using a simple API or decorators, and monitor queues via an admin dashboard. The library supports multiple Redis connection configurations (including Sentinel, SSL, and URL-based), custom job/worker/queue classes, and various commit modes (auto, on_db_commit, request_finished) to control when jobs are enqueued relative to database transactions. It also includes management commands for running workers, schedulers, and cron jobs, as well as Prometheus metrics and JSON statistics endpoints for monitoring. Architecturally, django-rq extends RQ's core classes (Queue, Worker, Job) with Django-aware subclasses that integrate with Django's settings, signals, and admin interface. The key entry points are the `django_rq` module's public API (`enqueue`, `get_queue`, `get_worker`, `job` decorator), Django management commands (`rqworker`, `rqscheduler`, `rqcron`, `rqstats`, etc.), and the admin views automatically registered via Django's admin site. Data flows involve enqueuing jobs (which may be deferred via thread-local queue or database transaction hooks), processing by workers, and storing results/status in Redis. External dependencies are Django (4.2+), RQ (2.6.1+), and Redis (3.5+), with optional prometheus_client for metrics.