$ account ozontech
file.d is a high‑throughput log and event processing daemon written in Go, designed for production data pipelines. It reads events from various sources (files, Kafka, Kubernetes, HTTP, syslog, sockets, journalctl), runs them through a configurable chain of action plugins (masking, throttling, transformation, decoding, joining, etc.), and delivers them to multiple outputs (Elasticsearch, ClickHouse, Loki, S3, Kafka, Splunk, Postgres, GELF, file, stdout). The architecture is plugin‑based: inputs, actions, and outputs register into a central registry, and pipelines are defined via YAML configuration. The system provides at‑least‑once delivery semantics, stream‑based ordering, batching with backoff, and dead‑queue failover. The codebase demonstrates strong engineering practices: extensive unit and integration tests (including e2e scenarios with external services), performance optimizations (zero‑copy JSON via insane‑json, event pooling, memory limits), and a rich set of reusable internal libraries (HTTP client with circuit breaker, metric controllers, match rules, configuration parsing). It includes a Helm chart for Kubernetes deployment, Prometheus monitoring with a Grafana dashboard, and a playground web service for interactively testing pipelines. The transform action includes a custom expression language with a Pratt parser and compiler. While the core is solid, there are some maintenance risks: many plugins contain global caches or mutable state, several code paths call `logger.Fatal` on misconfiguration (making the whole process exit), and the file input offset persistence is hand‑rolled. The large number of input/output/action plugins (~40 total) increases surface area for drift, and some e2e tests depend on external Docker services. Overall, file.d is a mature and capable data pipeline framework, well‑suited for log aggregation and transformation in containerized environments.