AIStore (AIS) is a high-performance, distributed object storage system engineered for AI/ML workloads. It operates as a cluster of proxy (gateway) and target (storage) nodes, providing S3-compatible API, multi-cloud backend support (AWS, GCP, Azure, OCI), and advanced data management features such as erasure coding, mirroring, LRU caching, and distributed resharding (dsort). The system is designed for linear scale-out and consistent performance, with a rich set of extended actions (xactions) for batch operations like copy-bucket, archive, blob download, and ETL transformations. It includes a full-featured CLI (`ais`), a Python SDK with PyTorch dataset integration, and a standalone authentication server (AuthN). The codebase is primarily a production-grade application with strong software engineering practices: modular architecture, extensive unit and integration tests, code generation for high-performance serialization (msgp), and build-tag-based feature gating. It demonstrates non-obvious craftsmanship in areas like memory management (slab allocators, SGL), intra-cluster transport (stream bundles), and probabilistic data structures (dynamic cuckoo filters). While the core is a storage system, it has significant data/research features (ETL pipelines, distributed sorting, shard indexing) that make it a hybrid tool for AI data pipelines. Key entry points include `cmd/aisnode/main.go` (server), `cmd/cli/main.go` (CLI), `cmd/authn/main.go` (authentication), and `python/aistore/sdk/` (Python SDK). The system relies on cloud SDKs, Kubernetes client, and various Go libraries. It is actively maintained with frequent commits, comprehensive CI/CD, and a large test suite.
This is a large, multi-language monorepo from X (formerly Twitter) containing the core code for the For You feed recommendation system. It includes real-time Rust services (home-mixer, thunder, vm-ranker), a Scala rule engine (botmaker) for spam/abuse detection, Python ML training infrastructure (phoenix/xrex), content safety pipelines (grox), and batch feature extraction jobs (simclusters, agatha). The system is composed of loosely coupled services communicating via thrift/gRPC and Kafka, with a strong emphasis on modular pipeline architectures (sources, hydrators, filters, scorers, selectors). While the code demonstrates high craftsmanship with clean abstractions and extensive use of traits/interfaces, it is deeply integrated with internal infrastructure (Manhattan, Strato, Finagle) and requires significant adaptation for external reuse. The repo is hybrid, containing both production application services and substantial ML research/training code.
This is a Python application and library for singing voice conversion and real-time voice changing, forked from so-vits-svc and rebuilt around PyTorch, PyTorch Lightning, and a Click-based CLI. The actual code provides two user-facing entry points: the `svc` CLI with commands for inference, realtime microphone conversion, dataset preprocessing, training, clustering, and cleanup, and the `svcg` PySimpleGUI desktop application for interactive and realtime inference. Inference loads a HuBERT content encoder from Hugging Face, extracts pitch using CREPE, DIO, Harvest, or Parselmouth, then runs a VITS-style SynthesizerTrn with either HiFi-GAN or MS/MB-iSTFT decoders conditioned on speaker embeddings. Despite fastapi being listed as a dependency, no HTTP API server exists in the source; the project is a desktop/CLI tool, not a web service.