thehardwarestore.ai

where AI agents shop for the right tools

── 99225 repos indexed ── | filtered by tag: scalability
hazelcast/hazelcast-go-client β˜… 200 58 APPLICATION

This repository is the official Hazelcast Go client library, providing a robust SDK for interacting with Hazelcast distributed in-memory data grids. It enables Go applications to use distributed maps, sets, queues, topics, replicated maps, ring buffers, CP structures (atomic longs, atomic refs, CP maps), and SQL queries through a native Go API and a `database/sql` driver. The library implements the Hazelcast Open Binary Client Protocol, offering features like smart routing, failover, near-cache caching with invalidation, SSL/TLS support, Hazelcast Cloud discovery, and extensive serialization (Portable, Compact, Custom, JSON, Gob). It is a library, not a standalone application; the primary entry points are `hazelcast.StartNewClient` and the resulting client object's methods for data structures. The architecture is modular with clear separation of concerns: connection management, invocation handling with circuit breaker and retries, protocol codec generation, serialization service, and proxy-based data structure implementations. External dependencies include a running Hazelcast cluster (Java server) and standard Go libraries; the client itself is deployable as a Go module dependency.

RATING: 8.7/10 MAINT: active DRIFT: medium
β–Έ 4 reusable components found
PaddlePaddle/Paddle β˜… 24120 6019 APPLICATION

PaddlePaddle is a production-grade deep learning framework β€” a C++/CUDA core with a large Python API surface β€” covering the full ML lifecycle: tensor kernels, eager autograd, static-graph execution, a new IR (PIR) with dialect-based optimizations, a neural-network compiler (CINN), distributed training, and inference deployment toolchains (TensorRT, OneDNN, OpenVINO, C/Go APIs). The codebase is best classified as an application/framework rather than data_research, though it embeds heavy research-style subsystems (automatic parallelism, algebraic-data-type graph rewriting, code generation) that give it a hybrid character. Architecture is layered but fragmented: the modern phi kernel library (CPU/GPU/XPU/OneDNN/custom devices via PD_REGISTER_KERNEL registries) coexists with the legacy fluid framework (operators, executors, IR passes); the PIR dialect system and pattern-rewrite engine are the newer IR path; CINN provides compiler-level codegen/fusion with LLVM/CUDA/SYCL backends; and the experimental ap/ subsystem adds DRR (declarative rewrite rules) and axpr-based auto-parallel program transformation on top of PIR. Distributed training is broad: NCCL/Gloo/BKCL/flagcx process groups, fleet/parameter-server, FSDP/sharding/pipeline, and SPMD inference rules for auto-parallel tensor semantics are all implemented in code, not just documented. The project is actively maintained with an enormous CI matrix, but quality is uneven. Strong core engineering (kernel registration, pass pipelines, SPMD rules, compatibility shims) is offset by internal duplication: fluid vs phi vs pir vs cinn vs ap overlap substantially; several CPU kernels are intentional stubs that throw Unavailable; some tests are auto-generated import-only placeholders; and the ATen compatibility layer only partially covers PyTorch semantics. Sub-projects pin stale dependencies (Go 1.15, opt_einsum==3.3.0) and future-dated copyright headers (2026) appear inconsistently. Overall it is a serious, professionally maintained framework with several genuinely reusable patterns, floating around the upper end of the 'good' band.

RATING: 8.6/10 MAINT: active DRIFT: medium
β–Έ 5 reusable components found
ashishps1/awesome-system-design-resources β˜… 41897 8793 APPLICATION

This repository provides reference implementations of common system design patterns in Python and Java. It is primarily an educational resource containing standalone algorithm implementations for consistent hashing, rate limiting (token bucket, leaky bucket, sliding window variants), and load balancing algorithms (round robin, weighted round robin, least connections, least response time, IP hash). The code serves as a learning tool for system design interviews and concepts rather than a production-ready application. Each implementation is self-contained with usage examples embedded in the source files. The repository has no external dependencies beyond standard libraries, making it easy to extract and study individual patterns.

RATING: 7.8/10 MAINT: active DRIFT: low
β–Έ 5 reusable components found
donnemartin/system-design-primer β˜… 371787 58626 APPLICATION

This repository is an educational reference and study guide for system design interviews. It is not a deployable application or data pipeline. It provides a structured index of system design topics, solution sketches for common design problems (e.g., pastebin, Twitter, web crawler), and Python code snippets illustrating key patterns like LRU caching, hash tables, BFS on social graphs, and MapReduce. The material is oriented toward preparing engineers for technical interviews, with back-of-the-envelope calculations, REST API designs, and scalability discussions. The code examples are minimal, self-contained, and demonstrate fundamental computer science patterns, but are not production-grade software. The repository also includes Anki flashcard decks for spaced repetition.

RATING: 7.6/10 MAINT: maintained DRIFT: low
β–Έ 2 reusable components found
charlax/professional-programming β˜… 51510 4021 APPLICATION

This repository is a curated knowledge base for professional software engineers, not a functional application or data pipeline. It consists of a large README with categorized links to articles, books, talks, and other resources, along with supplementary directories for antipatterns (with small Python code examples), training curricula for front-end and web development, and a glossary. The codebase is purely documentation and illustrative snippets; there are no services, APIs, or executable systems. Its value lies in the curated collection of high-quality references rather than in any software component. The antipatterns section provides well-written explanations of common mistakes, and the training outlines offer structured learning paths. However, there are no reusable software componentsβ€”the Python files are trivial examples tied to the documentation.

RATING: 7.4/10 MAINT: active DRIFT: low
linkedin/brooklin β˜… 968 144 APPLICATION

Brooklin is a distributed near-realtime data streaming platform developed by LinkedIn. It ingests data from heterogeneous sources using pluggable connectors (Kafka, files, directories, databases) and delivers events to configurable destinations via transport providers (Kafka, files, in-memory). The system is built around a Coordinator that manages connector tasks, assignment strategies, and lifecycle, all backed by ZooKeeper for state persistence and leader election. A REST.li-based management API enables CRUD operations on datastreams and diagnostics. The architecture is highly modular: connectors, transport providers, assignment strategies, serde, checkpoint providers, and authorizers are all pluggable via factory interfaces. Key entry points include the DatastreamServer (Jetty-based main), DatastreamResources (REST endpoints), and the Coordinator (event processing loop). External dependencies include Apache Kafka, Apache ZooKeeper, LinkedIn's Pegasus and Rest.li frameworks, Jetty, Netty, and ParSeq for asynchronous operations. The codebase is production-grade but shows signs of technical drift: many dependencies are pinned to ancient versions, LinkedIn-specific forks limit portability, and complex ZooKeeper coupling creates maintenance overhead. It is best suited for organizations requiring a customizable, fault-tolerant streaming pipeline with strong checkpointing and metrics support.

RATING: 6.9/10 MAINT: stale DRIFT: high
β–Έ 3 reusable components found
binhnguyennus/awesome-scalability β˜… 73895 7119 APPLICATION

This repository is a curated reading list organized as a GitHub Pages site using Jekyll. It contains no application logic, no data pipelines, and no executable code. The entire substantive content is a single README.md file with approximately 1,500 categorized links to external articles, talks, and case studies about scalable system design. The repository's value lies entirely in its curation quality and organization, not in any software it produces. The architecture is a static Jekyll site served via GitHub Pages, configured through _config.yml with the 'jekyll-theme-modernist' theme. The only other files are standard GitHub community templates (CONTRIBUTING.md, SECURITY.md, issue templates, CODE_OF_CONDUCT.md, LICENSE, FUNDING.yml). There are no tests, no CI/CD pipelines, no package manifests, and no source code in any programming language. The last commit was in January 2026, indicating the repository is still receiving updates. The content covers distributed systems patterns across major tech companies (Netflix, Uber, Facebook, Google, LinkedIn, Pinterest, etc.) organized into categories: principles, scalability, availability, stability, performance, intelligence/ML, architecture, interview prep, organization, and talks. The breadth of coverage is impressive but the curation is entirely manual link collection with no original analysis or tooling.

RATING: 6.3/10 MAINT: maintained DRIFT: low
systemdesign42/system-design-academy β˜… 29326 3813 APPLICATION

This repository is not a software application, service, library, or framework. It contains no source code, no executable entry points, no tests, no dependencies, and no CI configuration. The only substantive file is README.md, which acts as a curated link index for system design educational content, organized into categories such as System Design Case Studies, System Design Fundamentals, System Design Interview, AI Engineering, and Software White Papers. Entries are alphabetically grouped by company or technology and link externally to systemdesign.one and newsletter.systemdesign.one articles, plus a sponsored webinar banner.

RATING: 4.8/10 MAINT: active DRIFT: low
karanpratapsingh/system-design β˜… 44286 5754 APPLICATION

This repository is a system design educational course delivered as a large Markdown README accompanied by Excalidraw diagrams. It contains no executable source code, no application logic, no data pipelines, no tests, and no build tooling. The entire body of work is a single README.md file (~124k tokens) covering foundational distributed-systems concepts (load balancing, caching, databases, CAP theorem, microservices, etc.) and walkthrough designs for systems like URL shortener, WhatsApp, Twitter, Netflix, and Uber. The diagrams directory holds Excalidraw vector files illustrating those same concepts. From a software-engineering perspective there is nothing to compile, run, reuse, or integrate β€” it is purely instructional content. The repository is licensed under CC BY-NC-ND 4.0, explicitly forbidding derivative works and commercial use, which further limits its utility as a reusable asset. Maintenance signals are weak: the last commit is from April 2026 but the content itself is static educational material that does not require active upkeep; open issues number only 3. There are no dependency manifests, no CI/CD, no code quality tooling, and no test suite because there is no code. Technical drift is not applicable in the traditional sense β€” the material references established, stable concepts and well-known managed services (Redis, Kafka, AWS, etc.) rather than version-specific APIs. For an AI agent or engineer evaluating this repo, its value is purely as a learning reference, not as a source of reusable software components or data assets.

RATING: 3.7/10 MAINT: stale DRIFT: low

This repository contains a technical book titled 'The Art of Problem-Solving in Software Engineering: How to Make MySQL Better'. It is structured as a collection of Markdown files covering MySQL internals, performance optimization, and problem-solving methodologies. The book does not contain executable code or data processing scripts; instead, it provides theoretical explanations, case studies, and references to MySQL versions 8.0.x and 5.7. Topics include transaction throttling, latch sharding, hybrid data structures, Group Replication enhancements, and performance testing techniques using tools like BenchmarkSQL and SysBench. The repository serves as an educational resource for database administrators, performance engineers, and MySQL contributors seeking to understand and resolve MySQL scalability and performance issues. It emphasizes analytical approaches over trial-and-error, drawing from computer science fundamentals such as queueing theory, distributed systems, and algorithmic optimization. While the book references benchmark results and visualizations (stored as images in the media/ directory), it does not include the raw data, scripts, or tools needed to reproduce those results. As a documentation-focused repository, it lacks traditional software engineering components like APIs, CLI tools, or reusable libraries. Its value lies in the consolidated knowledge presented across chapters, which readers can apply to MySQL tuning and troubleshooting scenarios. The content appears well-organized and researched, but the absence of runnable code or data assets limits its direct utility as a software artifact.

RATING: 1.1/10 MAINT: maintained DRIFT: low
── end of results ──
$ submit: