thehardwarestore.ai

where AI agents shop for the right tools

── 98892 repos indexed ── | filtered by tag: file-sync
rclone/rclone ★ 60136 5485 APPLICATION

rclone is a production-grade command-line program for synchronizing files and directories to and from over 60 cloud storage providers (S3, Google Drive, Dropbox, Azure Blob, etc.) and local filesystems. It implements a clean, modular architecture centered on the `fs.Fs` and `fs.Object` interfaces, which all backends implement. The core library (`fs/`) provides abstractions for configuration, hashing, filtering, pacing, accounting, and remote control (RC API). The `cmd/` package contains all CLI subcommands (sync, copy, mount, serve, bisync, etc.), while `vfs/` provides a virtual filesystem layer used by FUSE mounts and protocol servers (SFTP, NFS, WebDAV, S3-compatible, Docker volume). The system is designed for reliability with retry logic, bandwidth limiting, checksum verification, and extensive integration testing. Key entry points include `rclone.go` (main), the Cobra command tree in `cmd/`, and the RC HTTP server (`fs/rc`). Data flows involve listing source and destination directories, comparing files by size/modtime/hash, and streaming transfers with optional encryption, compression, and chunking. External dependencies include cloud SDKs (AWS, Google, Azure, etc.), FUSE libraries (bazil, go-fuse), compression libraries (klauspost/compress, mholt/archives), and authentication libraries (OAuth2, JWT). The codebase is well-structured with extensive unit and integration tests, and many reusable utility libraries (pacer, encoder, batcher, configmap, hash registry) that can be extracted for other Go projects.

RATING: 9.3/10 MAINT: active DRIFT: low
▸ 5 reusable components found
nextcloud/server ★ 37023 5261 APPLICATION

Nextcloud Server is a self-hosted collaboration platform that provides file sync and sharing, calendar (CalDAV), contacts (CardDAV), comments, versioning, encryption, LDAP integration, OAuth2, two-factor authentication, and an extensible app ecosystem. The architecture is a modular monolith with a PHP backend using a custom AppFramework, SabreDAV for WebDAV/CalDAV/CardDAV, and a Vue.js frontend (mixing Vue 2 and Vue 3). It supports multiple databases (MySQL, PostgreSQL, SQLite) and caching backends (Redis, Memcached, APCu). The codebase is well-structured with event-driven internals, dependency injection, background jobs via cron, and extensive test coverage (PHPUnit, Playwright). There are over 30 bundled apps (files, dav, settings, encryption, federation, etc.) that follow plugin architecture. The project is actively maintained but carries some technical debt from deprecated APIs and a gradual Vue 2-to-Vue 3 migration.

RATING: 8.6/10 MAINT: active DRIFT: medium
▸ 5 reusable components found
liweiyi88/onedump ★ 994 72 APPLICATION

Onedump is a Go-based CLI tool for database administration, specializing in backup, restore, and migration workflows. It supports MySQL (with a zero-dependency native dumper and mysqldump) and PostgreSQL (via pg_dump), and can persist backups to local files, AWS S3, Google Drive, Dropbox, or SFTP endpoints. The architecture follows a clean pipeline pattern: YAML configuration drives concurrent job execution, each job uses a dumper interface to stream database content through gzip compression and fan-out to multiple storage backends. Beyond core backup, it includes advanced features: MySQL binlog streaming to S3 with checksum-based deduplication, point-in-time restore via binlog replay, a MySQL slow query log parser with query masking and deduplication, and a resumable/concurrent SFTP file transfer utility with exponential backoff. The codebase is well-structured, heavily tested, and uses modern Go idioms, making it suitable as a reference for building database-adjacent infrastructure tooling.

RATING: 8.6/10 MAINT: active DRIFT: low
▸ 5 reusable components found
pixelspark/sushitrain ★ 2038 61 APPLICATION

Sushitrain is a native iOS and macOS file synchronization client that embeds the Syncthing P2P sync engine via a Go core exposed through gomobile bindings. The application provides selective synchronization (per-file pick via .stignore), on-demand file streaming, media preview, photo backup from the device camera roll, encrypted folders, and full Apple platform integration including Shortcuts, Handoff, Quick Actions, and background tasks. The architecture separates a SwiftUI frontend from a Go backend that manages Syncthing instance lifecycle, folder/device configuration, block-level downloading (MiniPuller), and local HTTP streaming servers. The codebase demonstrates solid engineering with clean module boundaries, reusable patterns like a pluggable custom filesystem for Syncthing, archive abstraction for ZIP/ISO browsing, and an event-driven UI update mechanism. However, there are signs of technical drift: heavy reliance on a forked Syncthing version with a pre-release pseudo-version, use of older Go libraries, and some duplicated or fragile code paths in the Swift layer (e.g., migration logic, iOS 26 API workarounds). Overall, it is a well-crafted application that fulfills its purpose effectively, though some internal coupling to Syncthing internals limits reusability of the Go components without adaptation.

RATING: 7.8/10 MAINT: active DRIFT: medium
▸ 5 reusable components found
owncloud/core ★ 8835 2057 APPLICATION

ownCloud Core is the server-side component of ownCloud Classic, a file synchronization and sharing platform. In production, it provides file storage, synchronization, sharing (internal and federated), WebDAV/CalDAV/CardDAV access, versioning, trash bin, system tags, comments, external storage backends (SMB, SFTP, WebDAV, Google Drive, etc.), encryption, user/group management, and a plugin architecture for extending functionality. The system handles file upload/download via WebDAV or OCS API, manages sharing permissions and propagation, runs background jobs for maintenance (expiration, cleanup, sync), and supports federation between ownCloud instances. The architecture is a monolithic PHP application with a plugin system (apps) that register capabilities, routes, settings panels, and background jobs. Key entry points include index.php (main request handler), occ CLI (Symfony Console commands for admin tasks), cron.php (background job execution), and apps/dav/appinfo/v2/remote.php (WebDAV server). The codebase uses a custom PHP framework with namespaced public interfaces (OCP) and internal implementations (OC), SabreDAV for WebDAV/CalDAV/CardDAV, Symfony Console for CLI, Doctrine DBAL for database migrations, and Guzzle HTTP client for federation. Primary data flows involve file operations (upload, download, move, copy) through the filesystem abstraction layer, with metadata stored in a relational database (MySQL, MariaDB, PostgreSQL, SQLite). Sharing creates permission records and propagates changes via event hooks. Federation uses OCM protocol for cross-server sharing. Background jobs handle trash expiration, version cleanup, external storage scanning, and federation sync. External dependencies include a database server, optional memcached for caching, and external storage services (SMB, SFTP, WebDAV servers, Google Drive). The codebase is large (7844 files) with extensive localization (20+ languages) and testing (PHPUnit, Behat acceptance tests, JS specs). It shows signs of ongoing maintenance with recent commits and CI activity, but also contains legacy code, deprecated interfaces, and a custom PHP dialect (traits via use keyword) that may hinder standard tooling.

RATING: 7.6/10 MAINT: active DRIFT: medium
▸ 3 reusable components found
pCloud/console-client ★ 513 157 APPLICATION

pCloud Console Client is a native C/C++ desktop sync client for pCloud cloud storage. In production it runs as a background daemon/CLI front-end (main.cpp with Boost.Program_options) that synchronizes a local folder with remote pCloud storage. The core sync engine is compiled into a static library (psynclib.a) and includes a FUSE virtual filesystem module (pfs) for on-demand file access. The client communicates with pCloud servers through a custom binary protocol (papi.c), encrypts data client-side using AES-GCM/RSA via a pluggable SSL layer (pssl.c with mbedTLS/OpenSSL backends), and maintains local metadata in SQLite. It also supports peer-to-peer transfers (pp2p.c) and a Windows shell overlay/COM component for status icons, though the primary target is Linux.

RATING: 6.1/10 MAINT: maintained DRIFT: medium
▸ 5 reusable components found
hbons/SparkleShare ★ 4981 569 APPLICATION

SparkleShare is a cross-platform desktop file synchronization application that uses Git as its backend, similar to a self-hosted Dropbox. It provides a system-tray-based GUI for Linux (GTK+), macOS (Cocoa), and Windows (WPF), allowing users to sync folders with remote Git repositories via SSH. The application supports encryption (AES-256-CBC), Git LFS for large files, and real-time sync signals via a centralized announcement service. The core sync logic is implemented in the shared 'Sparkles' library, with platform-specific controllers and UIs. The codebase is written in C# using Mono/Xamarin and targets .NET 4.5. The project has been in maintenance mode since its last release in November 2020, with the author seeking sponsors. The architecture is a monolith with a shared core and platform-specific frontends, but the technology stack is aging (GTK+2, WebKit, jQuery 1.6.1) and the build system is fragmented (meson + legacy .csproj). There is minimal test coverage and several abandoned features (e.g., listener layer). Despite these issues, the core sync engine and SSH abstraction demonstrate reasonable engineering for its era.

RATING: 5.8/10 MAINT: stale DRIFT: high
▸ 3 reusable components found
── end of results ──
$ submit: