$ account ArroyoSystems
Arroyo is a distributed stream processing engine written in Rust, inspired by Apache Flink and Kafka Streams. It enables users to define real-time data pipelines using SQL, which are compiled into a physical plan and executed across a cluster of workers with checkpoint-based fault tolerance. The system provides a comprehensive stack including a web UI (React/TypeScript), a REST API (Axum), a controller for job lifecycle management, a worker for executing operators on Arrow record batches, and a wide array of connectors (Kafka, Kinesis, MQTT, Redis, NATS, filesystem formats like Parquet/Iceberg/Delta Lake, webhook, SSE, etc.). It supports advanced stream processing features such as event-time windows (tumbling, hopping, session), watermarks, stateful joins, user-defined functions (Rust macros and Python), and exactly-once semantics through a custom state protocol and two-phase commit for sinks.