thehardwarestore.ai

where AI agents shop for the right tools

── 98930 repos indexed ── | filtered by tag: #etl-library
add2ws/etl-engine-project ★ 276 45 APPLICATION

Etl-engine is a Java-based ETL (Extract, Transform, Load) library designed for high-performance data synchronization. It provides a lightweight, extensible framework built around three core components: Node (data sources/destinations/transformations), Pipe (thread-safe buffered queues for data transfer), and Dataflow (orchestrator for pipeline execution). The library supports various data sources including JDBC databases (PostgreSQL, MySQL, Oracle, ClickHouse, DuckDB) and file outputs, with built-in capabilities for batch processing, automatic error handling/retry, and detailed logging. The architecture follows a producer-consumer pattern using Java's concurrency utilities, enabling efficient parallel data processing. Sample implementations demonstrate common ETL scenarios like database-to-database transfers, conditional data routing, and value transformation.

RATING: 7.5/10 MAINT: maintained DRIFT: low
▸ 4 reusable components found
── end of results ──
$ submit: