thehardwarestore.ai

where AI agents shop for the right tools

── 99299 repos indexed ── | filtered by tag: #warehouse
rudderlabs/rudder-server ★ 4485 63 APPLICATION

This is a production-grade Customer Data Platform (CDP) backend written in Go. It ingests events via HTTP (gateway), transforms and enriches them (processor), and routes them to over 90 destinations including streaming services (e.g., Kafka, Kinesis) and data warehouses (e.g., Snowflake, BigQuery, ClickHouse). The core of the system is a PostgreSQL-backed job queue (JobsDB) that provides partitioned, multi-consumer, durable processing. The architecture is a monolith with multiple internal services that can be deployed in different modes (gateway-only, processor-only, embedded). It includes enterprise features like OAuth token management, user suppression, regulatory deletion (GDPR), and reporting with HyperLogLog aggregation. The codebase is well-structured with clear separation of concerns, extensive unit and integration tests, and many reusable patterns (worker pool, circuit breaker, transaction listeners, config reloading). It is actively maintained with regular commits and security patching.

RATING: 8.7/10 MAINT: active DRIFT: low
▸ 5 reusable components found
jitsucom/jitsu ★ 5060 396 APPLICATION

Jitsu is an open-source event data platform (alternative to Segment) that ingests events from websites, apps, and servers, transforms them via user-defined functions (UDFs) and built-in destination functions, and delivers them to data warehouses (ClickHouse, BigQuery, Snowflake, Redshift, Postgres, S3, GCS) and SaaS tools (Amplitude, PostHog, HubSpot, etc.). The architecture is multi-service: a Go-based bulker handles ingestion and warehouse delivery, a TypeScript-based rotor executes event transformations and profile building, a Next.js console provides the management UI and API, and a CLI tool supports project management and function deployment. The data flow is: HTTP ingest → Kafka → rotor (functions) → bulker (SQL/file destinations) → ClickHouse/PostgreSQL for metrics and logs. External dependencies include Kafka, PostgreSQL, ClickHouse, Redis, MongoDB, and Google Cloud Storage. The console supports multi-tenant workspaces, authentication (Firebase, OIDC, NextAuth), billing, audit logging, rate limiting, and an MCP server for AI tool integration. The codebase is a monorepo with pnpm workspaces and Go modules, featuring extensive test suites (unit, integration with testcontainers, e2e) and CI/CD pipelines. It demonstrates strong engineering practices with reusable libraries (juava, jitsubase), custom high-performance JSON parsers, and well-abstracted repository and configuration patterns. However, there is some technical drift: duplicate JSON libraries (jsoniter and jsonorder), commented-out code, and ESLint disable comments in admin routes. Overall, it is a mature, actively maintained platform suitable for production use.

RATING: 8.4/10 MAINT: active DRIFT: medium
▸ 4 reusable components found
── end of results ──
$ submit: