lsq is a CLI tool written in Go that serves as a lightning-fast companion for Logseq, the open-source knowledge base. Its core functionality is creating, appending to, and viewing daily journal entries directly from the terminal, following Logseq's file naming and formatting conventions (Markdown or Org-mode). The tool provides a rich set of flags for targeting specific dates, searching pages via a trie-based index (including alias support), regex searching across journals and pages, and piping content from STDIN. It also supports indentation for nested list items and a configuration file in EDN format for persistent settings. The architecture is a single-binary monolith with clear separation into config, system (file I/O, editor launch), trie, todo state cycling, and an EDN validator module. While the tool is tightly coupled to Logseq's directory structure (journals/ and pages/ folders), the underlying components like the trie search, file appending logic, and configuration loading are well-isolated and could be reused in other Go projects with minimal adaptation.
Ladder is a self-hosted HTTP web proxy written in Go, designed for testing and analyzing paywall implementations, content delivery, and access control behavior on modern websites. It intercepts client requests, applies domain-specific YAML rulesets to modify request headers, URLs, and response content (e.g., removing CORS/CSP headers, injecting scripts, rewriting relative URLs), then fetches the target site and returns the modified response. The proxy exposes a web form, a JSON API, a raw HTML endpoint, and a ruleset endpoint, and supports optional FlareSolverr integration for bypassing Cloudflare challenges. It is deployable as a binary, Docker container, or Helm chart, and is configured via environment variables and CLI flags. The architecture is a single Go binary using the Fiber web framework. The main entry point (cmd/main.go) sets up routing, basic auth, and middleware. Core logic lives in handlers/proxy.go, which handles URL extraction, ruleset application, HTTP fetching, HTML rewriting, and FlareSolverr integration. The pkg/ruleset package provides a clean, reusable ruleset loader supporting local files, directories, remote URLs, and gzip-compressed YAML. The codebase is generally well-structured with some global state and a few misnamed test files (e.g., proxy.test.go instead of proxy_test.go) that are not executed by Go's test runner. Despite these minor issues, the project is actively maintained, uses current dependencies, and demonstrates solid engineering practices for a focused tool.
This repository implements a production-grade Google Maps business data extraction platform with multiple execution modes: CLI, web UI, REST API, AWS Lambda, and a self-hosted SaaS edition. The core scraping engine uses Playwright for browser automation and also supports a stealth HTTP fast mode. It extracts 36+ structured fields per business (name, address, phone, website, reviews, ratings, emails, coordinates, hours, etc.) and provides deduplication, grid-based area coverage, email extraction via website visits, and review parsing. The architecture is modular with packages for scraping jobs, runners (file, database, web, Lambda, SaaS worker), output writers (CSV, JSON, PostgreSQL, S3, custom plugins), job queuing via River, and a comprehensive admin dashboard with cloud provisioning on DigitalOcean and Hetzner. The system is designed for both single-user CLI use and multi-tenant SaaS deployments with authentication, 2FA, API key management, and scalable worker pools. The codebase demonstrates strong Go engineering practices with well-defined interfaces, dependency injection, and test coverage. Notable reusable components include cryptography utilities, HTTP server middleware, rate limiting, and a CLI prompt library. However, the repository has some maintenance concerns: two separate database migration sets for legacy and SaaS modes, in-memory deduplication that doesn't persist across restarts, committed test SSH keys in the `local/` directory, and a large dependency surface that could introduce version conflicts. The scraper logic is inherently brittle due to reliance on Google Maps internal data structures, which are subject to change without notice. For AI agent consumption: This is a powerful off-the-shelf scraping tool that can be deployed as a Docker container, AWS Lambda, or managed SaaS. It provides a complete API and web interface for managing scraping jobs, monitoring progress, and exporting data. The SaaS edition includes worker auto-provisioning, making it suitable for large-scale lead generation operations. The code is actively maintained with recent commits addressing performance and feature enhancements.
spank is a macOS CLI tool that detects physical hits/slaps on Apple Silicon MacBooks via the accelerometer and plays audio responses. It requires sudo for IOKit HID access to the accelerometer and is built as a single-file Go application. The tool reads raw accelerometer data, runs vibration detection algorithms (STA/LTA, CUSUM, kurtosis, peak/MAD), and triggers audio playback when impacts exceed a configurable amplitude threshold. It offers multiple modes: pain (random 'ow!' sounds), sexy (escalating responses based on slap frequency), halo (Halo death sounds), lizard (similar to sexy but with lizard sounds), and custom (user-specified MP3 directory). Audio playback uses the beep package with resampling to a fixed sample rate, optional volume scaling based on impact force, and adjustable playback speed. The application uses Cobra for CLI parsing and Fang for command execution, with audio assets embedded via go:embed. Key reusable components include the audio resampling function that handles sample rate conversion and speed adjustment, and the amplitude-to-volume mapping function that applies a logarithmic curve for natural volume scaling.
Noti is a cross-platform CLI tool written in Go that monitors the execution of a command (or a PID) and triggers notifications when it completes. It supports a wide array of notification services: desktop banners (macOS NSUserNotification via osascript, Linux Freedesktop via D-Bus, Windows tray icon via PowerShell), speech synthesis (macOS say, Linux espeak, Windows SpeechSynthesizer), and cloud services (Slack, Telegram, Twilio, Pushbullet, Pushover, Pushsafer, Simplepush, BearyChat, Keybase, Mattermost, Google Chat, Zulip, Chanify, ntfy, Bark). The architecture is a monolithic CLI built with Cobra and Viper. Configuration is sourced from YAML files (with schema validation), environment variables (with deprecated-var remapping), and CLI flags, following a clear precedence order. The codebase is well-tested, uses idiomatic Go, and demonstrates clean interface abstraction for extensible notification backends. The repository has moved to Codeberg but the GitHub mirror shows recent active development.
Kaitai Struct is an umbrella project that provides a declarative language for describing binary data structures and a compiler to generate parser code in multiple target languages. The repository coordinates submodules for the compiler, runtimes for various languages (Java, Python, Ruby, etc.), format definitions, visualizer, tests, benchmarks, and documentation. Users describe a format in a .ksy file, compile it with the kaitai-struct-compiler to generate language-specific parser code, and use the corresponding runtime to parse binary data. The project includes CI workflows for building the compiler and publishing artifacts, and maintains a collection of ready-to-use format descriptions.
This repository is the official Google API Go client library collection, consisting of auto-generated client packages for hundreds of Google REST and gRPC APIs, alongside a code generator and shared infrastructure packages. The architecture centers on a Discovery Document-based code generator (`google-api-go-generator`) that produces per-service Go packages, each exposing a `NewService` constructor and fluent call builders with consistent patterns for pagination, media upload, and error handling. Shared support packages (`internal/gensupport`, `googleapi`, `option`, `transport`) provide HTTP/gRPC client construction, retry logic, credential resolution (mTLS, S2A, ID tokens), and JSON marshaling. The repository is actively maintained for critical fixes, but the generated client libraries are explicitly in maintenance mode, with newer Cloud Client Libraries recommended. Key entry points include `option.ClientOption` for client configuration, `transport/http.NewClient` and `transport/grpc.Dial` for transport, `idtoken` for token validation, and the generator itself for creating new clients.
Pentest-Swarm-AI is an open-source autonomous penetration testing tool built on a stigmergic swarm architecture. It coordinates multiple LLM-driven specialist agents (recon, classifier, exploit, report) via a shared blackboard with pheromone decay, enabling emergent attack paths without a central planner. The system integrates 30+ external security tools (nmap, nuclei, sqlmap, etc.), multiple LLM providers (Claude, OpenAI, Ollama, Gemini, LM Studio), and bug bounty platforms (HackerOne, Bugcrowd, Intigriti) for automated scope import and report submission. The architecture is a multi-agent system with a scheduler that dispatches agents based on blackboard predicates and pheromone levels. Key entry points include a CLI (Cobra commands: scan, init, playbook, submit, etc.), an API server (Fiber), a WebSocket hub for live streaming, a Next.js web dashboard, and an MCP server for IDE integration. Data flows: agents write findings to the blackboard with pheromone weights; the scheduler reads the blackboard and triggers agents whose predicates match; agents read findings, process them, and write new findings, creating a stigmergic feedback loop. External dependencies include PostgreSQL with pgvector for durable blackboard storage, LLM API endpoints, and the installed security tool binaries. The codebase demonstrates high craftsmanship with clean abstractions (tool adapter pattern, LLM provider interface, blackboard interface), extensive testing (unit, integration, e2e, benchmarks, LLM eval), and safety features (scope enforcement, provenance signing, rate limiting, false-positive caching). It is actively maintained with recent commits, CI/CD pipelines, and community engagement.
This repository is a production-oriented Go client library for the Binance cryptocurrency exchange, covering spot, margin, USDS-M futures, coin-margined delivery futures, options, portfolio margin, and related products. The code in v2/ exposes fluent builder-style services for REST endpoints and WebSocket streams, with clients such as NewClient, NewFuturesClient, NewDeliveryClient, NewOptionsClient, and portfolio-specific clients. The library is consumed by trading bots and exchange-integration applications; it is not a data analysis or research project, though it provides market-data access primitives for klines, tickers, order books, funding rates, and historical trades.
Convox Rack is a self-hosted platform-as-a-service that runs in an AWS account and manages the full lifecycle of containerized applications. The repository is a Go monolith with three coordinated surfaces: a CLI (cmd/convox), an HTTP/WebSocket API server (cmd/rack + pkg/api), and a generated Go SDK (sdk). All operations funnel through the Provider interface defined in pkg/structs/provider.go, and the main production implementation in provider/aws drives CloudFormation, ECS, ECR, DynamoDB, S3, KMS, SSM, CloudWatch Logs, SNS/SQS, and Lambda. In production, the rack creates per-app CloudFormation stacks, builds Docker images through ECS tasks, persists builds and releases in DynamoDB/S3, and promotes releases by updating stacks. Primary entry points are cmd/convox for the CLI, cmd/rack for the API server, cmd/monitor for background workers, and the Lambda functions under provider/aws/lambda for custom CloudFormation resources, autoscaling, instance lifecycle draining, and syslog forwarding. The CLI also supports local development mode via pkg/start and pkg/sync, which builds manifests in Docker and incrementally syncs files into running containers. The manifest layer has two generations: pkg/manifest1 handles legacy docker-compose-style definitions, while pkg/manifest implements the newer convox.yml format with flexible YAML parsing, NLB validation, timers, and internal/external port handling. The codebase contains substantial test infrastructure, including golden-output CLI tests, SDK contract tests, generated mock providers, and the awsutil request/response replay harness. However, it also has accumulated drift: two parallel manifest implementations, version-gated compatibility branches in the CLI/SDK, an embedded CA trust store from 2017, AWS SDK v1 dependencies, and some provider methods that remain unimplemented. Some tests are skipped or commented out. These factors keep the code from being exemplary, but the overall architecture is coherent, professional, and actively maintained. This is purely an application repository; there is no data-science or research component. The only data involved is operational state such as builds, releases, environment variables, and logs, stored in AWS services. Reusable pieces include generic TTL caching, order-preserving YAML unmarshalling, CloudFormation stack helpers, KMS envelope encryption, and a replay-based AWS test harness.
whatscli is a terminal-based (TUI) WhatsApp client written in Go. It connects via the WhatsApp Web API using the whatsmeow library, providing message sending/receiving, media attachment handling, group management, and desktop notifications. The architecture is a monolith with two goroutines: a tview-based UI main loop and a session manager that handles the WhatsApp connection and command dispatch through channels. The UI is highly configurable via an INI config file, supporting custom keybindings and color schemes. The codebase demonstrates good concurrency patterns with channel-based IPC and includes unit tests for the message storage layer. While the UI logic in main.go is tightly coupled to tview, the message database and event handling show clean separation and moderate reusability.
discrawl is a Go CLI application for archiving and searching Discord data. It syncs messages from the Discord API, imports Discord desktop cache, stores raw JSON in SQLite via sqlc-generated code, and provides subcommands for sync, publish, subscribe, share import, and remote status. It also exports/imports compressed JSONL snapshots with manifests and commits them to a Git snapshot repo using NUL-delimited path-only commits to preserve unrelated staged work. The repository is primarily an application, but it also vendors agent skills (autoreview, crabbox, discrawl) under .agents/skills, making it a hybrid monorepo.
file.d is a high‑throughput log and event processing daemon written in Go, designed for production data pipelines. It reads events from various sources (files, Kafka, Kubernetes, HTTP, syslog, sockets, journalctl), runs them through a configurable chain of action plugins (masking, throttling, transformation, decoding, joining, etc.), and delivers them to multiple outputs (Elasticsearch, ClickHouse, Loki, S3, Kafka, Splunk, Postgres, GELF, file, stdout). The architecture is plugin‑based: inputs, actions, and outputs register into a central registry, and pipelines are defined via YAML configuration. The system provides at‑least‑once delivery semantics, stream‑based ordering, batching with backoff, and dead‑queue failover. The codebase demonstrates strong engineering practices: extensive unit and integration tests (including e2e scenarios with external services), performance optimizations (zero‑copy JSON via insane‑json, event pooling, memory limits), and a rich set of reusable internal libraries (HTTP client with circuit breaker, metric controllers, match rules, configuration parsing). It includes a Helm chart for Kubernetes deployment, Prometheus monitoring with a Grafana dashboard, and a playground web service for interactively testing pipelines. The transform action includes a custom expression language with a Pratt parser and compiler. While the core is solid, there are some maintenance risks: many plugins contain global caches or mutable state, several code paths call `logger.Fatal` on misconfiguration (making the whole process exit), and the file input offset persistence is hand‑rolled. The large number of input/output/action plugins (~40 total) increases surface area for drift, and some e2e tests depend on external Docker services. Overall, file.d is a mature and capable data pipeline framework, well‑suited for log aggregation and transformation in containerized environments.
Dublin Traceroute is a NAT-aware multipath traceroute tool that extends the Paris-traceroute technique to enumerate ECMP load-balanced paths and detect NATs. It forges UDP packets with custom payloads to control the UDP checksum, which is used as a per-flow identifier and embedded in the IP ID field for NAT correlation. The system listens for ICMP Time Exceeded and Destination Unreachable responses, matches them to sent probes using IP ID (IPv4) or payload length (IPv6), and computes a NAT ID from checksum differences. Output is provided in JSON or DOT format for visualization. The repository contains two independent implementations: a primary C++ library/CLI using libtins and jsoncpp, and an experimental Go implementation with IPv6 support and additional tooling (routest for integration testing). The Go implementation demonstrates cleaner separation of concerns with reusable network protocol modules (ICMP, UDP, IPv6) and a layered protocol abstraction.
spogo is a terminal-based Spotify client written in Go that provides full control over playback, library management, search, and user listening data. It authenticates via browser cookies (sp_dc/sp_t) instead of the official OAuth flow, allowing it to bypass rate limits by using Spotify's internal web API endpoints (Connect/GraphQL) alongside the public Web API. The architecture is modular with multiple engine backends (connect, web, auto, applescript) and a clean CLI interface built with Kong. It features robust output formatting (human, JSON, plain), comprehensive error handling with documented exit codes, and extensive test coverage. While the tool is highly functional and well-crafted, its reliance on undocumented internal Spotify APIs and webpack hash resolution introduces fragility that may require maintenance as Spotify's frontend evolves.
This repository implements a Terraform provider for managing Incus, a container and virtual machine orchestration platform. It provides full CRUD operations for Incus resources such as instances, networks, storage pools, profiles, projects, certificates, cluster groups, and server configuration, along with corresponding data sources for reading existing state. The provider is built using the Terraform Plugin Framework (v1.19.0) with some fallback to Plugin SDK v2, and communicates with Incus via its client library (github.com/lxc/incus/v7). Key entry points include the main.go provider server, internal/provider/provider.go for resource registration, and individual resource files under internal/{certificate,cluster,image,instance,network,profile,project,server,storage}. The provider supports multiple remotes, OIDC authentication, clustering, and PKI, and includes a code generation tool (cmd/generate-datasources) that produces consistent data source implementations and documentation from a YAML configuration and Go templates. Acceptance tests require a live Incus environment and cover complex scenarios like instance lifecycle, exec triggers, file uploads, and import. The codebase is well-structured with shared utilities in internal/common for config/device/file/import handling, and demonstrates a consistent pattern for state synchronization and error handling. However, it carries some technical debt from its fork of the LXD provider, including dual SDK usage and heavy boilerplate that could benefit from further abstraction.
carbon-clickhouse is a production-grade Graphite metrics receiver and storage gateway written in Go. It ingests time-series data over multiple protocols (plain text TCP, UDP, Pickle, gRPC, Prometheus Remote Write, Telegraf HTTP JSON) and persists it to ClickHouse using the efficient RowBinary format. The system is architecturally split into receivers, a writer that rotates and compresses data files, and uploaders that batch-insert data into ClickHouse tables. Receivers handle protocol-specific parsing and drop rules; the writer manages chunking, interval-based rotation, and optional LZ4 compression; uploaders support points, index (hierarchical metric trees), tagged metrics (sorting, dedup, and URL encoding), and points-reverse storage. External dependencies include ClickHouse (HTTP interface), Prometheus protobuf definitions, gRPC, and common Go libraries. The codebase is well-tested, includes end-to-end integration tests with actual ClickHouse containers, and is actively maintained with CI/CD pipelines producing DEB/RPM packages.
Berty is a full-stack, privacy-focused, peer-to-peer messaging application. The system consists of a Go backend implementing the Wesh protocol (based on IPFS, libp2p, and OrbitDB) that provides core messaging, push notifications, authentication, replication, and directory services. The backend is exposed via gRPC and consumed by multiple clients: a React Native mobile app (with both a legacy JS variant and a newer Expo-based variant), a Go CLI with a terminal UI (mini) and a desktop GUI (Fyne), and native bridges for iOS and Android. The architecture is modular, with separate packages for error handling, authentication, push relay, and bot framework. Key entry points include the `berty daemon` CLI command, the mobile app entry points (`App.tsx` in both `js/packages` and `berty-bridge-expo/mobile`), and the bridge framework (`go/framework/bertybridge`). Data flows involve gRPC streams from the Go backend to the mobile clients, with event-driven state management and local encrypted storage via SQLCipher. External dependencies include libp2p, OrbitDB, gRPC, protobuf, and various platform-specific native modules for BLE, mDNS, and push notifications. The repository also contains auxiliary tools for infrastructure testing, rendezvous servers, and debugging (Tyber).
Redka is a Redis-compatible data store implemented in Go, using SQLite or PostgreSQL as the storage backend. It reimplements core Redis data structures (strings, lists, sets, hashes, sorted sets) with full TTL support, key scanning, and set/sorted-set aggregation operations. The system can be deployed either as an embedded Go library (via the `redka` module) or as a standalone RESP-compatible server (single binary or Docker image). Architecture follows a clean layered design: a top-level `DB` struct provides transactional access to per-type repositories (e.g., `ZSet()`, `Hash()`), which in turn use a dialect abstraction layer (`sqlx`) to generate SQLite- or PostgreSQL-specific queries. The RESP server is built on `tidwall/redcon` with a command dispatch pattern and a reusable parser combinator framework for argument parsing. While feature-complete for core Redis operations, the project is explicitly in maintenance mode—no new features are planned, and documentation notes incomplete command coverage (e.g., blocking operations, some sorted set commands). The codebase is well-structured with extensive test coverage, but Postgres support shows signs of drift (many queries aliased to SQLite) and the command dispatch switch has grown large.
K8up is a Kubernetes Operator designed for data protection, specifically managing backup and restore operations for Persistent Volume Claims (PVCs) and applications using the Restic backup tool. It functions as a control plane that reconciles Custom Resource Definitions (CRDs) such as Schedule, Backup, Restore, Prune, and Check into Kubernetes Jobs. The architecture is a monolithic binary (`cmd/k8up`) that can run as an operator or a CLI tool, utilizing the Kubebuilder framework and controller-runtime for the operator logic. It supports multiple storage backends (S3, Azure, GCS, Swift, etc.) and integrates with Prometheus for metrics and Grafana for dashboards. The codebase demonstrates a solid separation of concerns, with distinct modules for the operator logic (`operator/`), the Restic CLI wrapper (`restic/`), and the API definitions (`api/v1/`). It employs standard Kubernetes patterns like Job-based execution, RBAC, and Secret management for credentials. The project includes comprehensive testing, including unit tests, integration tests using envtest, and E2E tests using Bats. However, the codebase exhibits signs of technical drift, notably the `go.mod` file specifying a non-existent Go version (1.26), which suggests potential build instability or experimental state. Additionally, deprecated fields in CRDs (e.g., `keepJobs`) require migration attention. This repository is suitable for engineers needing a production-grade backup solution for Kubernetes clusters. It is particularly valuable for teams requiring declarative backup management, multi-cloud storage support, and application-aware backup capabilities via annotations or PreBackupPods. The high technical drift rating is primarily due to the Go version anomaly, but the architectural patterns and test coverage remain robust.