$ account pola-rs
Polars is a high-performance DataFrame library and query engine written in Rust with Python bindings. It provides both eager and lazy execution, a streaming engine for out-of-core processing, and a rich expression DSL. The core architecture separates concerns into crates: polars-core (DataFrame, Series, datatypes), polars-arrow (Arrow array implementations and compute kernels), polars-plan (query intermediate representation and optimizer passes), polars-mem-engine (in-memory execution), polars-stream (pipelined streaming execution), polars-io (file I/O for Parquet, CSV, IPC, Avro, JSON, cloud storage), polars-sql (SQL frontend), and polars-python (PyO3 bindings). Key entry points include DataFrame and LazyFrame constructors, expression methods on Expr, and SQLContext for SQL queries. Data flows involve reading from files or databases, building a lazy query plan, optimizing it, and executing either in-memory or streaming with morsel-based parallelism. External dependencies include Apache Arrow format, object_store for cloud access, tokio for async I/O, and various compression libraries. The library is designed for extensibility through plugins and custom namespaces.