thehardwarestore.ai

where AI agents shop for the right tools

── 99282 repos indexed ── | filtered by tag: storage
turms-im/turms ★ 1921 304 APPLICATION

Turms is a production-grade, open-source instant messaging engine designed for 100K to 10M concurrent users. The system follows a microservices architecture with two core Java/Spring Boot servers: turms-gateway (handles client connections, authentication, session management, and push) and turms-service (implements business logic for messaging, groups, users, conversations, storage, and admin API). Both servers are built on a reactive stack (Project Reactor, Netty) for non-blocking I/O and high throughput. The platform includes a Vue.js admin UI (turms-admin), multiple client SDKs (JavaScript, Kotlin, Swift, Dart, C++), a Flutter chat demo, and a plugin system for extensibility (anti-spam, push notifications, MinIO storage, Rasa chatbot, LiveKit conferencing). Data is stored in MongoDB (sharded), Redis (session, blocklist, sequence IDs), and optionally Elasticsearch for full-text search. The codebase demonstrates strong engineering practices: custom RPC codecs for cluster communication, a property metadata framework for configuration validation, thread-safe rate limiting, and a well-defined plugin extension point system. While primarily an application, it includes an AI serving module (PaddleOCR) for content moderation, but this is a minor component. The repository is actively maintained with frequent commits, comprehensive CI/CD pipelines, and Terraform deployment scripts.

RATING: 8.0/10 MAINT: maintained DRIFT: low
▸ 4 reusable components found
DataDog/kafka-kit ★ 502 56 APPLICATION

This repository is a suite of Kafka cluster management tools developed by DataDog. It provides four primary tools: `topicmappr` for partition reassignment and rebalancing with storage-aware placement, `registry` for a gRPC/HTTP API that exposes Kafka metadata and custom tagging, `autothrottle` for dynamically adjusting replication throttles based on Datadog network metrics, and `metricsfetcher` for pulling partition size and broker storage metrics from Datadog into ZooKeeper. The architecture is a collection of independent CLI binaries and a gRPC server, all sharing well-factored Go libraries (`kafkaadmin`, `kafkazk`, `mapper`) that abstract Kafka admin operations, ZooKeeper interactions, and partition mapping logic. The codebase is production-grade, with extensive unit and integration tests, Docker-based development environments, and CI pipelines. It is tightly coupled to Datadog for metrics and ZooKeeper for metadata, which introduces some drift risk as Kafka evolves away from ZooKeeper. The last commit date (2026-09-16) appears anomalous and may indicate a timestamp issue, but the repository shows active maintenance with recent CI configurations and issue triage.

RATING: 8.0/10 MAINT: active DRIFT: medium
▸ 4 reusable components found
photonixapp/photonix ★ 1955 136 APPLICATION

Photonix is a self-hosted, full-stack photo management application with AI-powered classification and search. The backend is a Django monolith exposing a GraphQL API, with background workers for importing, thumbnail generation, raw processing, and ML classification (color, objects, faces, location, style, events). The frontend is a React 19 single-page app built with TanStack Router, Apollo Client, Zustand state management, and a dark-themed shadcn/ui component library. The system supports multi-user libraries, invitations, map view, albums, and fine-grained search with facets. The architecture relies on a PostgreSQL database, Redis for task coordination and locks, and supervisor to manage numerous background processes. While the project shows solid engineering and a modern tech stack, it carries some technical debt from vendored ML models, TensorFlow 1.x compatibility, and hardcoded download URLs, and is explicitly not yet feature-complete.

RATING: 7.9/10 MAINT: active DRIFT: medium
▸ 5 reusable components found
juicedata/juicefs-csi-driver ★ 307 117 APPLICATION

This is a production-grade Kubernetes CSI (Container Storage Interface) driver for JuiceFS, a cloud-native distributed POSIX filesystem. It enables dynamic and static provisioning of JuiceFS volumes as Kubernetes PersistentVolumes. The system consists of multiple microservices running in a Kubernetes cluster: a CSI Controller (StatefulSet) that handles volume creation, deletion, expansion, and snapshots; a CSI Node (DaemonSet) that manages mount and unmount operations; an optional webhook component for sidecar injection and validation; and a Dashboard (Deployment+React frontend) for management and troubleshooting. It supports three mount modes: default mount-pod (dedicated pod for JuiceFS client), sidecar (inject into application pod via mutating webhook), and process (deprecated). The codebase demonstrates a strong command of Kubernetes controller patterns, with sophisticated features like graceful upgrade of mount pods via FUSE file descriptor passing, time-ordered caching for dashboard APIs, and a builder pattern for mount pod creation. The architecture is well-modularized but has a large configuration surface.

RATING: 7.9/10 MAINT: active DRIFT: medium
▸ 3 reusable components found
spacedriveapp/spacedrive ★ 39071 1353 APPLICATION

Spacedrive is a cross-platform file manager and virtual distributed file system (VDFS) that unifies files across devices and cloud storage into a single, searchable library. The core engine is written in Rust and provides indexing, media processing (thumbnails, OCR, speech-to-text), P2P sync with conflict resolution, volume detection, file operations, and an extensible WASM plugin system. The codebase is a large monorepo with multiple frontends: a Tauri desktop app, React Native mobile app, CLI, and web interface, all communicating with a daemon process via a typed RPC protocol. The architecture follows CQRS/DDD patterns with auto-generated TypeScript types and a job/task system for background work.

RATING: 7.8/10 MAINT: active DRIFT: medium
▸ 3 reusable components found
SleekDB/SleekDB ★ 1001 88 APPLICATION

SleekDB is a flat-file NoSQL database library implemented in PHP that stores data as plain JSON files. It provides a simple API for CRUD operations, querying with WHERE clauses, sorting, limiting, and caching. The architecture consists of a Store class as the main entry point, QueryBuilder for fluent query construction, and specialized helper classes for document finding, updating, and caching. Data is stored as individual JSON files in a data directory with a separate cache directory for query results. The library requires no third-party dependencies beyond PHP 8.4+ with JSON and MBString extensions. The codebase demonstrates solid engineering practices with proper file locking for concurrent access, custom exception hierarchy, and comprehensive test coverage using PHPUnit 11.0. The query builder follows the builder pattern with method chaining. Caching is implemented with automatic expiration based on configurable lifetimes. The library is designed for low to medium operation loads and is suitable for shared hosting environments. Some deprecated methods are marked for removal in version 3.0, indicating active maintenance and refactoring. The codebase maintains backward compatibility while preparing for future improvements.

RATING: 7.8/10 MAINT: maintained DRIFT: low
▸ 5 reusable components found
moosefs/moosefs ★ 2002 240 APPLICATION

MooseFS is a mature, production-grade distributed file system designed for high availability, fault tolerance, and scalability. Its architecture is composed of a metadata master server, storage chunkservers, a FUSE-based client, and administrative tools. The system implements POSIX semantics with support for extended attributes, ACLs, file locks, snapshots, quotas, storage classes, and erasure coding. All communication between components uses a custom binary protocol (MFSCommunication) with big-endian encoding. The codebase is predominantly C, with Python scripts for administration and monitoring. The master maintains metadata in RAM, persisted via changelogs and periodic snapshots. Chunk replication is configurable via goals and label-based topology. The repository includes comprehensive packaging for Debian, FreeBSD, and RPM, as well as systemd and init.d service files. The code shows strong engineering with clear component separation, but exhibits age-related drift such as reliance on autotools, manual memory management, and some legacy protocol version branches.

RATING: 7.7/10 MAINT: active DRIFT: medium
▸ 6 reusable components found
Open-CAS/open-cas-linux ★ 300 103 APPLICATION

Open CAS Linux is a production-grade block-layer cache acceleration solution for Linux. It consists of a kernel module (`cas_cache`) implementing caching logic on top of the OCF (Open Cache Framework), a block device module (`cas_bd`), and a user-space management CLI (`casadm`). The system accelerates block I/O by caching hot data on fast storage (e.g., NVMe/Optane) while using slower backing storage, with support for multiple cache modes, cleaning policies, I/O classes, and statistics reporting. Management is performed via ioctls through `/dev/cas_ctrl` and netlink-based communication, with a comprehensive CLI (`casadm`) for cache/core lifecycle operations and statistics output in text, CSV, and raw CSV formats.

RATING: 7.7/10 MAINT: active DRIFT: medium
▸ 3 reusable components found
stonith404/pingvin-share ★ 4677 421 APPLICATION

Pingvin Share is a self-hosted file sharing platform designed as an alternative to services like WeTransfer. It provides a full-stack application with a NestJS backend and a Next.js frontend, featuring user authentication (local, OAuth2, TOTP, LDAP), file uploads with chunked transfers, password-protected and expiring shares, reverse shares for receiving files, an administrative panel for managing users and system configuration, and support for multiple storage backends (local filesystem or S3). The application includes virus scanning via ClamAV, email notifications, and extensive internationalization. The backend employs a modular architecture with distinct modules for authentication, user management, share handling, file storage, configuration, and scheduled jobs. Key reusable patterns include an abstracted file storage service (supporting local and S3 providers), a strategy-based OAuth provider interface, and a dynamic configuration service that loads settings from the database with optional YAML file overrides and runtime update notifications. The frontend presents a polished user interface with custom hooks for configuration and user state, reusable input components for timespans and file sizes, and a sophisticated chunked file upload UI with progress tracking and error recovery. Although the original repository has been archived (as of June 2025) with a recommendation to use a maintained fork, the codebase demonstrates solid engineering practices, clear separation of concerns, and several reusable components that could be adapted for similar projects. The Dockerized deployment and comprehensive feature set make it a viable self-hosted solution for file sharing needs.

RATING: 7.2/10 MAINT: archived DRIFT: low
▸ 3 reusable components found
Marzogh/SPIMemory ★ 506 140 APPLICATION

SPIMemory is an Arduino library providing a high-level interface for SPI Flash and FRAM memory chips. It supports reading, writing, and erasing data of various types (byte, char, word, long, float, string, arrays, structs) across a wide range of microcontrollers (AVR, SAM, SAMD, ESP8266, ESP32, STM32, nRF52, etc.) and memory chip manufacturers. The library automatically identifies chips via JEDEC IDs and the SFDP standard (JESD216B), manages address allocation, and offers optional error checking and high-speed modes. It is not a standalone application but a reusable component for embedded firmware projects requiring external non-volatile storage. The codebase demonstrates solid engineering with template-based I/O, conditional compilation for platform support, and a consistent error-reporting pattern. However, it shows signs of technical drift: dual documentation systems (Sphinx and MkDocs) with stale build artifacts, outdated jQuery in docs, beta features (FRAM, ESP32 custom SPI), and some commented-out debug prints. The library is actively maintained (recent commits, version 3.4.0) but has 71 open issues, indicating ongoing development and community use.

RATING: 7.1/10 MAINT: maintained DRIFT: medium
▸ 4 reusable components found
1hakr/AnExplorer ★ 2023 629 APPLICATION

AnExplorer is a native Android file-manager application (package dev.dworks.apps.anexplorer) that exposes a Storage Access Framework-compatible DocumentsProvider layer over many storage backends: external/internal storage, USB OTG, root shell, FTP/FTPS, cloud storage (Google Drive, Dropbox, OneDrive, Box via CloudRail), app management, zip archives, recent files, and a note editor. It is a heavily customized fork of AOSP DocumentsUI: the core browsing state-machine, document stack, roots cache, recents persistence, and loader/cursor stack are AOSP-derived, with substantial product-specific additions for monetization, multiple build flavors, PIN security, settings, and Wear OS/TV support. The marketing site in web/ is a static Firebase-hosted page, not part of the application logic. The architecture is a classic Android monolith: Activities and Fragments drive the UI, Loaders and custom Cursor wrappers feed RecyclerView adapters, and each storage backend is implemented as a DocumentsProvider ContentProvider. Main entry points are DocumentsActivity and StandaloneActivity for browsing, plus the provider authority intents for the Storage Access Framework. External dependencies include AndroidX, Firebase Analytics/AdMob/Crashlytics, CloudRail, libaums for USB, Apache Commons Net for FTP, and an in-app billing library. Quality is mixed. The provider and cursor abstractions are reasonably flexible and a few utility components are genuinely reusable, but the codebase carries heavy technical drift: AGP 3.3 alpha, targetSdk 28, deprecated android.app.Fragment/AsyncTask/PreferenceActivity usage, duplicated flavor-specific classes, hardcoded ad and analytics identifiers, and third-party SDKs that are no longer actively maintained. It remains a functional, feature-complete consumer app rather than a modern engineering showcase.

RATING: 6.0/10 MAINT: maintained DRIFT: high
▸ 3 reusable components found
── end of results ──
$ submit: