gog is a comprehensive, production-grade CLI tool for Google Workspace automation that unifies 20+ Google services (Gmail, Calendar, Drive, Docs, Sheets, Slides, Forms, Meet, Chat, Contacts, Tasks, Classroom, YouTube, Analytics, Search Console, Admin, Groups, Keep, Photos, Maps, Apps Script) into a single binary. It excels at task-first workflows: searching and sanitizing mail, resolving calendar names, auditing Drive sharing, editing Docs, appending Sheet tables, building Slides from Markdown, managing Workspace users, and performing encrypted backups. The architecture is a modular monolith with clean internal package boundaries for each service domain. Key entry points are cmd/gog/main.go and the Kong-based command tree in internal/cmd. Primary data flows: command invocation -> account/client resolution -> authenticated Google API client creation -> API execution -> structured output (JSON, plain TSV, or human-readable tables). External dependencies include Google APIs, OAuth2, keyring backends (macOS Keychain, Windows Credential Manager, Linux Secret Service, encrypted file), MCP for agent integration, and age encryption for backups. The codebase demonstrates exceptional craftsmanship in cross-platform secret storage, Gmail push notification processing, sed-style Docs editing with UTF-16 index tracking, atomic Docs request batching, encrypted backup with Git shard checkpointing, email open tracking with versioned key rotation and bot detection, and build-time safety profile compilation to hash-based allow/deny switches. Active development with recent commits, comprehensive test coverage (unit + integration + live tests), and modern dependency versions.
restic is a fast, secure, and efficient backup program written in Go. It provides a command-line interface with subcommands for creating, restoring, managing, and verifying backups. The architecture is modular: a core repository layer handles encryption (AES-256-GCM), content-defined chunking for deduplication, compression (zstd), and pack-file storage. Multiple storage backends are supported via a common interface, including local filesystem, SFTP, S3, Azure Blob, Google Cloud Storage, Backblaze B2, OpenStack Swift, and rclone. The codebase is production-grade with extensive unit and integration tests, a FUSE mount for browsing snapshots, and a self-update mechanism. It is actively maintained with frequent releases and a large community.
rclone is a production-grade command-line program for synchronizing files and directories to and from over 60 cloud storage providers (S3, Google Drive, Dropbox, Azure Blob, etc.) and local filesystems. It implements a clean, modular architecture centered on the `fs.Fs` and `fs.Object` interfaces, which all backends implement. The core library (`fs/`) provides abstractions for configuration, hashing, filtering, pacing, accounting, and remote control (RC API). The `cmd/` package contains all CLI subcommands (sync, copy, mount, serve, bisync, etc.), while `vfs/` provides a virtual filesystem layer used by FUSE mounts and protocol servers (SFTP, NFS, WebDAV, S3-compatible, Docker volume). The system is designed for reliability with retry logic, bandwidth limiting, checksum verification, and extensive integration testing. Key entry points include `rclone.go` (main), the Cobra command tree in `cmd/`, and the RC HTTP server (`fs/rc`). Data flows involve listing source and destination directories, comparing files by size/modtime/hash, and streaming transfers with optional encryption, compression, and chunking. External dependencies include cloud SDKs (AWS, Google, Azure, etc.), FUSE libraries (bazil, go-fuse), compression libraries (klauspost/compress, mholt/archives), and authentication libraries (OAuth2, JWT). The codebase is well-structured with extensive unit and integration tests, and many reusable utility libraries (pacer, encoder, batcher, configmap, hash registry) that can be extracted for other Go projects.
This is a production-grade Kubernetes operator written in Go that manages the full lifecycle of MariaDB databases (including Galera clusters, asynchronous replication, and MaxScale proxies) on Kubernetes. It implements a custom resource definition (CRD) pattern with controllers, validating webhooks, and sidecar agents to handle deployment, scaling, TLS, backups (logical and physical), point-in-time recovery, and SQL resource management (users, grants, databases). The operator is built with controller-runtime and kubebuilder, and includes extensive integration tests using Ginkgo/Gomega and envtest. Its architecture is modular, with separate controllers for each CRD, reusable builder packages for Kubernetes objects, and well-factored utility packages for data structures, PKI, condition management, and statefulset helpers. The codebase is actively maintained, with a rich set of examples, Helm charts, and multi-cluster support. It does not include any data research or analytics components.
Onedump is a Go-based CLI tool for database administration, specializing in backup, restore, and migration workflows. It supports MySQL (with a zero-dependency native dumper and mysqldump) and PostgreSQL (via pg_dump), and can persist backups to local files, AWS S3, Google Drive, Dropbox, or SFTP endpoints. The architecture follows a clean pipeline pattern: YAML configuration drives concurrent job execution, each job uses a dumper interface to stream database content through gzip compression and fan-out to multiple storage backends. Beyond core backup, it includes advanced features: MySQL binlog streaming to S3 with checksum-based deduplication, point-in-time restore via binlog replay, a MySQL slow query log parser with query masking and deduplication, and a resumable/concurrent SFTP file transfer utility with exponential backoff. The codebase is well-structured, heavily tested, and uses modern Go idioms, making it suitable as a reference for building database-adjacent infrastructure tooling.
LiteFS is a FUSE-based distributed filesystem for replicating SQLite databases across a cluster of machines. It operates as a Linux daemon that intercepts SQLite's file-level operations (journal, WAL, SHM) to capture every transaction. These transactions are recorded in a custom LTX (Lite Transaction) binary format and streamed to replica nodes via an HTTP/2 h2c protocol. The system uses leader election (via Consul or a static configuration) to designate a primary node that accepts writes, while replicas synchronously apply changes and enforce read-after-write consistency via TXID cookies and a proxy layer. The architecture is fundamentally a thin FUSE layer wrapping a distributed state machine: each node runs a store that manages multiple databases, tracks guards for SQLite lock compatibility (PENDING, SHARED, etc.), and exposes an HTTP API for streaming replication, manual imports/exports, handoff, and backup integration. The proxy server sits in front of the application to route read requests to the primary or a sufficiently caught-up replica, using Fly-Replay headers for transparent redirect. Backup is supported through either a file-based mechanism or the LiteFS Cloud client. External dependencies include Consul (for leader election), FUSE (required on Linux), and the underlying SQLite library (no cgo, pure Go via the ltxx package). The codebase is written in Go, with a heavy focus on correctness under concurrency—evidenced by extensive integration tests that exercise split-brain recovery, checkpoint races, and multi-node handoff. The project is in beta and follows a roadmap-driven contribution model where the maintainer rewrites feature PRs to ensure consistency.
Pluton is a self-hosted, open-source backup application that wraps restic and rclone to provide scheduled, encrypted backups to a wide variety of cloud and local storage providers. The system is built as a full-stack monorepo with a Node.js/Express backend (TypeScript) and a React frontend (Vite), communicating via REST APIs. It manages backup plans, devices, storage locations, and restore operations through a web dashboard, with support for replication, retention policies, integrity checks, and multi-channel notifications (email, Slack, Discord, push). The architecture follows a layered pattern: controllers handle HTTP requests, services orchestrate business logic, stores encapsulate database access (SQLite via Drizzle ORM), and managers/strategies handle execution of backup, restore, and system operations using restic and rclone as child processes. A job queue with cron scheduling drives automated backups, and event emitters decouple progress tracking and notification dispatch. The frontend provides a rich UI with virtualized file browsers, snapshot viewers, and step-by-step wizards for plan creation and restore. The codebase is well-structured with extensive unit tests, CI/CD pipelines, and cross-platform installers (Docker, Linux AppImage, macOS, Windows).
Pelican Wings is a production-grade game server management daemon (agent) built in Go. It serves as the control plane for the Pelican Panel, managing Docker containers that run game servers. The daemon exposes a comprehensive HTTP API for server lifecycle operations (start, stop, install, backup, transfer), real-time WebSocket streams for console output and server stats, and a built-in SFTP server for file management. It communicates with a remote Panel API to synchronize server configurations, validate user credentials, and report activity logs. The codebase demonstrates strong systems engineering with a focus on security (sandboxed filesystem using openat2 syscalls, token authentication, download IP filtering), performance (disk-space caching, atomic generics), and reliability (crash detection, state persistence, cron-based backups). It supports multiple backup backends (local disk, S3), parallel server bootstrapping, and self-update via GitHub releases. The architecture follows a modular pattern with clear separation between API routing, server management, filesystem operations, and integration adapters. While the code is a fork of Pterodactyl Wings with some incomplete rebranding, it remains actively developed and production-ready.
FastBack is a Minecraft mod that provides fast, incremental world backups using Git. It stores backup snapshots as orphan branches in a Git repository embedded within the world save directory, enabling efficient storage of only changed files. The mod supports local backups, remote backups to any Git server or file system, automatic scheduling on shutdown or autosave, snapshot pruning with configurable retention policies (daily, fixed, GFS, all), and restoration of snapshots. It requires native git and git-lfs to be installed, with a fallback to JGit for legacy backups. The architecture is modular: a common module contains all backup logic (commands, repo management, config, logging, mixins), while Fabric and NeoForge loader-specific modules handle platform integration. The codebase demonstrates strong engineering practices with clean interfaces, robust error handling, comprehensive logging, and a well-designed retention policy system.
Mail-Archiver is a production-grade ASP.NET Core 10 web application for archiving emails from IMAP, Microsoft 365 (Graph), and local imports (MBOX, EML). It provides a full MVC UI for searching, viewing, exporting, restoring, and managing archived emails, alongside a read-only REST API (v1) and a Model Context Protocol (MCP) server for AI agent integration. The system features background job processing for sync, batch operations, deduplication, retention policies, and audit exports, all backed by PostgreSQL with full-text search. Authentication supports local password (PBKDF2), OpenID Connect, and API keys, with two-factor authentication and role-based access control. The codebase is well-structured with clear separation of concerns: provider-specific services (IMAP, Graph, Mbox), shared utilities, and a core service layer. Extensive unit and integration tests (xUnit) cover critical paths, and Docker Compose provides a reproducible test environment. While some technical debt exists (raw SQL in core services, in-memory job tracking, German comments in places), the overall architecture is solid and the reusable components (static helpers, deduplication strategies, background job patterns) demonstrate high craftsmanship.
This repository is a curated awesome-list of open-source operations and DevOps tools, organized into 82 categories with 653 entries. It is not a software application; it is a structured dataset of YAML metadata files that are transformed into bilingual README tables via a GitHub Action using Go templates. The data lineage is: contributors add YAML files under items/<category>/<tool>.yaml with fields kind, owner, repo, desc, and desc_en; on push to main, the generator workflow runs the eryajf/yaml-readme action to fetch live GitHub metadata (license, stars, creation/update dates) and render README.md and resources/README-en.md from the provided templates. The repository serves as a discovery resource for engineers evaluating ops tools, answering questions like 'what are the best open-source tools for a given ops category?' and 'what is the license, star count, and last update of a tool?'. It is actively maintained with recent commits, a YAML linting CI, and a clear contribution guide. The reusable assets are the templating pattern and CI workflows, which can be adapted to any curated list or structured-data-to-document pipeline.
Vackup is a standalone bash script CLI tool for backing up and restoring Docker volumes. It provides four core commands: export (volume to tarball), import (tarball to volume), save (volume to Docker image), and load (Docker image to volume). The tool runs temporary busybox containers to perform file operations, ensuring compatibility across Linux and macOS. It features robust error handling with user-definable failure hooks, cross-platform path resolution, and a comprehensive Bats test suite. The tool is designed to be run from any host with Docker CLI access, with optional Docker image distribution via the provided Dockerfile.
This repository provides a Docker-based tool for automatically discovering and backing up databases running in Docker containers. The core logic resides in a single Python script (`db-auto-backup.py`) that uses the Docker SDK to list running containers, identifies database containers by image name (PostgreSQL, MySQL/MariaDB, Redis), and executes native backup commands (pg_dumpall, mysqldump, redis-cli SAVE). Backups are written to a configurable directory with optional compression (gzip, lzma, bz2, zstd) and can be scheduled via cron expressions using the pycron library. The tool also supports success hooks (e.g., healthchecks.io, Uptime Kuma) and environment variable configuration. It is designed to run as a standalone Docker container with access to the Docker socket, making it a lightweight, self-contained backup solution for Docker hosts. The codebase is well-structured with type hints, linting, and a comprehensive test suite that validates backup generation and compression across multiple database types.
TiddlyDesktop is a desktop application that serves as a custom web browser for working with locally stored TiddlyWiki files and folders. Built on nw.js (formerly node-webkit), it provides a native windowing system to open, edit, save, and manage multiple TiddlyWiki instances. The application includes a backstage configuration wiki for managing settings, a wiki list window for organizing wikis, and support for both single-file HTML wikis (TiddlyWiki 5 and Classic) and wiki folder formats. It implements file saving via a TiddlyFox-like protocol, automatic backups, and drag-and-drop wiki registration. The architecture is modular, with separate window classes for wiki files, wiki folders, and backstage tiddlers, all managed by a central window list. The codebase is well-structured and demonstrates solid engineering practices for an nw.js application.
This is a Home Assistant add-on that provides automated backup functionality using rclone. The system consists of a Go-based scheduler that runs configured backup jobs (sync, copy, move, delete) against local paths and remote storage services. It features dual services: a scheduler for executing backup jobs and an rclone web GUI for remote configuration. The addon integrates with Home Assistant's event system to report job success/failure and handles backup file naming conventions specific to Home Assistant's backup format. Configuration is managed through Home Assistant's add-on system with options stored in /data/options.json.
Xen Orchestra is a comprehensive management platform for XenServer/XCP-ng virtualization environments. It provides a web UI (legacy React and modern Vue 3), a REST API, CLI tools, and a backup system. The architecture is a monorepo with multiple packages: server-side plugins (authentication, performance alerts, SDN controller, backups), frontend apps (xo-web React, @xen-orchestra/web Vue 3, @xen-orchestra/lite lightweight), and shared libraries (filesystem abstractions, VHD manipulation, cron parsing, audit logging, etc.). The backend communicates with XenServer hosts via the Xen API (XAPI), while the frontend connects via WebSocket/JSON-RPC or REST. Backup operations can be executed remotely via a proxy. Key data flows involve the web UI sending commands through WebSocket to the server, which translates them into XAPI calls. Backup jobs are scheduled via cron and executed by the backup runner system, streaming VM disks to remote storage. Audit events are logged with cryptographic chaining for tamper evidence. The system includes an MCP server for AI-assisted management. The repo demonstrates high engineering quality in many packages with clean abstractions for async operations, event-driven collections, and modular plugin systems, though it contains legacy components and some incomplete transitions to newer architectures.
git-backup is a CLI tool written in Go that backs up all Git repositories from GitHub and GitHub Enterprise (including GitLab) to local storage. It uses the go-git library to perform bare or non-bare clones without requiring the git CLI. The configuration is driven by a YAML file that supports Go templating for environment variable injection. The tool discovers repositories via the provider APIs using configurable filters (owned, starred, collaborator, org member) and excludes specific repos or namespaces. Pagination is handled automatically. It supports Docker deployment with UID/GID mapping. The code is well-structured with clear separation of providers (GitHub, GitLab) behind a common RepositorySource interface. It is a focused, standalone backup solution that does not require external dependencies beyond the binary itself.
The system is a Home Assistant addon that enables backup creation and synchronization with OneDrive. Built with ASP.NET Core and Blazor Server, it provides a web UI for configuration and monitoring, scheduled backups with retention policies, file synchronization, and exposes Home Assistant sensor entities for status monitoring. The architecture follows a layered design with separate services for backup management, file synchronization, and external API integrations (Home Assistant and Microsoft Graph). Key entry points include Program.cs for application startup and the Orchestrator for coordinating scheduled tasks. Data flows involve creating backups via the Home Assistant API, uploading to OneDrive via GraphHelper, and synchronizing file changes bidirectionally.
K8up is a Kubernetes Operator designed for data protection, specifically managing backup and restore operations for Persistent Volume Claims (PVCs) and applications using the Restic backup tool. It functions as a control plane that reconciles Custom Resource Definitions (CRDs) such as Schedule, Backup, Restore, Prune, and Check into Kubernetes Jobs. The architecture is a monolithic binary (`cmd/k8up`) that can run as an operator or a CLI tool, utilizing the Kubebuilder framework and controller-runtime for the operator logic. It supports multiple storage backends (S3, Azure, GCS, Swift, etc.) and integrates with Prometheus for metrics and Grafana for dashboards. The codebase demonstrates a solid separation of concerns, with distinct modules for the operator logic (`operator/`), the Restic CLI wrapper (`restic/`), and the API definitions (`api/v1/`). It employs standard Kubernetes patterns like Job-based execution, RBAC, and Secret management for credentials. The project includes comprehensive testing, including unit tests, integration tests using envtest, and E2E tests using Bats. However, the codebase exhibits signs of technical drift, notably the `go.mod` file specifying a non-existent Go version (1.26), which suggests potential build instability or experimental state. Additionally, deprecated fields in CRDs (e.g., `keepJobs`) require migration attention. This repository is suitable for engineers needing a production-grade backup solution for Kubernetes clusters. It is particularly valuable for teams requiring declarative backup management, multi-cloud storage support, and application-aware backup capabilities via annotations or PreBackupPods. The high technical drift rating is primarily due to the Go version anomaly, but the architectural patterns and test coverage remain robust.
This repository provides a comprehensive Docker Compose configuration for building a self-hosted NAS (Network Attached Storage) system. It orchestrates over 20 services including media management (Sonarr, Radarr, Lidarr, Bazarr, Prowlarr, qBittorrent, Jellyfin, Jellyseer), a WireGuard VPN (PIA), reverse proxy (Traefik with Let's Encrypt), a homepage dashboard (Homepage), automatic updates (Watchtower), and optional services like AdGuard Home, Home Assistant, Immich, Vaultwarden, Paperless, and more. The architecture is a multi-container Docker stack with a shared network, designed to run on any Docker-capable Linux box. The primary data flow involves downloading media via qBittorrent through a VPN, managing it with *arr applications, and serving it via Jellyfin. The repository includes automation scripts for initial configuration (update-config.sh) and port forwarding updates (pia-shared/portupdate-qbittorrent.sh). It is not a software application with custom code but rather a well-documented, opinionated deployment template. The codebase is primarily configuration files (docker-compose.yml, environment files, nginx configs) with a few bash and JavaScript scripts. The quality is good for its purpose: it is organized, documented, and widely used, but it lacks reusable software components that could be extracted into a new project with minimal modification.