This repository provides an all-in-one CLI/TUI application for launching and managing security testing tools. It features a curated catalog of 215+ tools across 21 categories, defined declaratively in YAML files and loaded into Python tool classes. The interactive REPL supports commands like /run, /search, /tags, /ai, /goal, and /find, with a rich console interface using prompt_toolkit and Rich. A headless engagement mode runs YAML-defined pipelines (e.g., subfinder → httpx → nuclei) and generates structured findings and Markdown reports. AI features are opt-in and use OpenAI-compatible endpoints or local Ollama, with built-in safety guardrails (charter injection, sanitization, anti-fabrication checks) and graceful degradation to offline behavior. The architecture is monolithic but modular, with clear separation between the tool catalog, execution engine, AI integration, and discovery module. The codebase demonstrates strong engineering practices: safe subprocess execution (list-form, cwd, skip if present), declarative YAML overlays, a fixed tag taxonomy enforced by tests, and extensive test coverage including monkeypatching of subprocess and AI transport. While the catalog inherently drifts as upstream tools become archived, the project includes audit scripts to manage this. Overall, it is a well-crafted, production-quality application for authorized security testing.
MarkdownDB is a JavaScript library and CLI tool that parses markdown files to extract structured data (frontmatter, tags, links, tasks) and stores it in a queryable SQLite database (or JSON files). It provides a lightweight Node.js API for querying the indexed data and includes a command-line tool for indexing folders and executing scripts. The architecture follows a modular design with clear separation between markdown parsing, processing, storage, and querying layers. The system uses remark-family tools for markdown parsing, Knex.js for database operations, and Zod for schema validation. Key entry points are the CLI (`src/bin/index.js`) and Node.js API (`src/index.ts`).
Apache Polaris is a production-grade, multi-tenant Iceberg REST catalog server built on Quarkus. It provides a REST API for catalog management (namespaces, tables, views), credential vending for cloud storage (S3, GCS, Azure), fine-grained authorization via built-in RBAC or external OPA/Ranger, and event publishing (Kafka, CloudWatch). The system supports two persistence backends: a legacy relational JDBC (PostgreSQL, H2) and a new NoSQL abstraction with custom indexing, caching, and distributed invalidation. Multi-tenancy is implemented via isolated realms, each with its own configuration and persisted state. Key entry points are the Quarkus server (`polaris-server`), a CLI admin tool, and a Python client. External dependencies include Iceberg libraries, Quarkus, Jackson, Testcontainers, and optional integrations like Open Policy Agent, Apache Ranger, and Kafka. The architecture is modular with clear SPI boundaries, making extensions (persistence, auth, federation) pluggable.
InfluxDB 3 Core (IOx) is a production-grade, open-source time-series database written in Rust. It ingests data via line protocol (HTTP/gRPC), stores it in Apache Parquet on object storage (S3, Azure, GCP, local), and serves queries via SQL (DataFusion), InfluxQL, and FlightSQL. The architecture is a single binary (`influxdb3`) that combines an HTTP server (port 8181), gRPC FlightSQL endpoint, CLI, and an embedded Python processing engine for plugin-based triggers. The system is designed for low-latency last-value queries, high-throughput writes, and efficient columnar storage with catalog persistence as versioned binary logs and snapshots on object store. The codebase is a large Rust workspace (~40+ crates) with a clear separation of concerns: core libraries (`core/`) for query planning, Arrow utilities, authorization, and metrics; server crates (`influxdb3_*`) for catalog management, write buffering, processing engine, and system tables. Key entry points include `influxdb3/src/main.rs` (CLI), `influxdb3_server/src/unified_service/` (HTTP/gRPC routing), and `influxdb3_catalog/src/` (catalog operations). The query engine is built on a forked DataFusion with custom physical optimizers (gapfill, dedup, sort, union) and InfluxQL planning. External dependencies include object store backends, protobuf for gRPC, and Python (via PyO3) for plugins. The repository demonstrates high craftsmanship with several reusable patterns: a versioned binary record format for catalog persistence with forward-compatibility guarantees, a hook-based cache system with S3-FIFO eviction, a generic copy-on-write repository pattern, and a run-optimized merge sort for streaming sorted data. However, the large workspace, forked DataFusion pinned to a commit, and multiple catalog versions (v1–v4) introduce maintenance drift risks. The codebase is actively maintained with extensive tests, benchmarks, and CI/CD.
Apache Iceberg is a high-performance table format for analytic data lakes, providing ACID transactions, snapshot isolation, schema evolution, and time travel. The repository contains the Java reference implementation, including core libraries for table metadata management, file format support (Parquet, ORC, Avro, Arrow), expression evaluation, encryption, and a REST catalog protocol. It integrates deeply with Spark (versions 3.5, 4.0, 4.1, 4.2), Flink (1.20, 2.2, 2.3), Kafka Connect, Hive, and cloud object stores (AWS S3, Azure ADLS, Aliyun OSS, GCS). The architecture is modular: the `api` module defines public interfaces (Table, Catalog, Scan, FileIO), `core` provides implementations and utilities, and separate modules handle engine-specific integrations and cloud backends. Key entry points include the Table interface for DDL/DML operations, Scan for query planning, FileIO for storage access, and the REST catalog client for remote metadata management. Data flows involve reading/writing manifest files, applying expressions for predicate pushdown, and coordinating commits via optimistic concurrency control. External dependencies include Apache Parquet, ORC, Avro, Arrow, and various cloud SDKs, but the core is designed to be lightweight and extensible.
FirefoxCSS-Store is a static community catalog for discovering and sharing Firefox userChrome.css themes. It is built with Astro 7, TypeScript 6, and Zod 4. The site renders a gallery of published themes with search, tag filtering, and sorting, and generates detail pages and a /themes.json API endpoint. The repository includes a comprehensive set of automation scripts for theme submissions (issue-form → candidate PR → approval), monthly repository audits, and stats refresh. Theme entries are validated with a strict Zod schema. The codebase is well-maintained, uses modern tooling, and has active CI/CD via GitHub Actions. It is primarily an application (static site generator + automation workflows) rather than a data pipeline.
This repository is the official awesome list for OpenCode, a terminal AI coding agent. It maintains a curated catalog of plugins, themes, agents, projects, and resources as YAML files under data/. A small set of Node.js scripts validates the YAML against a JSON schema, generates the README.md from a template, and exports a normalized dist/registry.json for downstream consumers such as opencode.cafe. CI workflows validate PRs and automatically regenerate the README/registry whenever the YAML data changes. Architecturally this is a data-driven build tool: YAML files are the source of truth, and scripts/validate.js, scripts/generate-readme.js, and scripts/export-json.js are the functional entry points. They use standard dependencies (ajv, js-yaml, glob) and are organized cleanly. There is no runtime service, API, or library here; the repo is a curated content registry plus lightweight tooling to keep the catalog consistent and machine-readable. The code is straightforward, with no tests but with solid validation and a well-designed schema.
This repository is a comprehensive skill manager for AI agents, providing a CLI tool (Node.js and Python) to search, install, list, and update a large library of over 300 skill definitions. The skills are structured markdown files with YAML frontmatter, each defining a specific capability or workflow for an AI assistant (e.g., code review, security audit, database optimization, TDD orchestration). The application includes a catalog generation pipeline, validation scripts with baseline support, and CI checks to ensure skill quality and consistency. While the bulk of the repository is composed of skill content (instruction playbooks), the active software layer—the CLI, catalog builder, and validator—represents a well-engineered dual-language application with tests and automation.