$ account bitmakerla
Estela is an open-source web scraping orchestration platform (analogous to Scrapy Cloud) for deploying, scheduling, and monitoring Scrapy-based spider projects on Kubernetes. It consists of a Django REST API backend with Celery for async task management, a React/TypeScript frontend with Ant Design, and a Kafka-based queue consumer for streaming scraped data into MongoDB. Core workflows include project management, spider deployment (Docker image build via Kaniko), job execution with configurable resource tiers, cron scheduling, data persistence settings, and a metering subsystem for usage tracking. The system relies on external dependencies such as Kubernetes, Kafka, MySQL, MongoDB, Redis, and object storage (MinIO/S3). The frontend provides interactive charts, a tour overlay for onboarding, and notification management. While functional and extensible, the codebase shows signs of age (outdated Django/Celery versions, class-heavy React components, duplicate logic in cron expression handling) and has incomplete test coverage.