Mollusk is a lightweight test harness for Solana programs, designed to execute program ELFs in a minified Solana Virtual Machine (SVM) without a full validator runtime. It provisions Agave's program cache, transaction context, and invoke context directly, enabling fast, deterministic testing of single instructions, instruction chains, and full transactions. The repository is organized as a Cargo workspace with core harness logic (mollusk-svm), a result validation crate (mollusk-svm-result), a benchmarking crate (mollusk-svm-bencher), a CLI tool (mollusk-svm-cli), fuzz fixture infrastructure, and bundled SPL program helpers (memo, token, token-2022). Key architectural patterns include a pluggable AccountStore trait for stateful testing, a flexible Check builder for result validation, and a Compare enum for cross-version execution comparison. The CLI supports fixture replay, benchmarking, and generating compute unit reports. The harness also supports fuzz fixture generation (both Mollusk and Firedancer formats) via protobuf schemas, enabling regression testing and cross-implementation comparison. While primarily a testing library, the inclusion of fuzz fixture infrastructure and benchmarking tools gives it a hybrid character similar to data research pipelines for test generation and analysis.
Zedis is a high-performance, Redis-compatible in-memory data store written in Zig. It implements the RESP protocol and supports core Redis data structures (strings, lists, TTL, pub/sub) along with advanced features: time-series with Gorilla compression, full-text search with BM25 and vector KNN (FLAT), probabilistic Bloom filters, and both AOF and RDB persistence. The architecture uses a multi-threaded server with a dedicated store thread for serialized command execution, custom budgeted allocators, and a lock-free client mailbox for message passing. While production-ready in capability, the project is explicitly designed for learning and experimentation and shows some documentation-to-code drift and a handful of unmaintained TODO items. It includes an extensive benchmark suite comparing against Valkey.
This repository is the official Kubernetes performance testing suite, housing multiple tools for benchmarking, load testing, and monitoring Kubernetes clusters. The core tool is clusterloader2 (CL2), a powerful framework that deploys scalable workloads, measures key performance indicators (API responsiveness, pod startup latency, scheduling throughput), and validates SLOs. It uses a pluggable measurement system, tuning sets for load patterns, and a provider abstraction layer supporting various environments (GKE, EKS, AKS, Kind, Kubemark). Other tools include a metric comparison utility (benchmark), a historical performance dashboard (perfdash), a real-time SLO Prometheus exporter (slo-monitor), DNS and network performance benchmarks, and data ingestion pipelines (perflens). The codebase is predominantly Go with Python scripts for data processing and extensive Prometheus/Grafana configurations. While primarily an application, it incorporates data pipelines for metric ingestion and SLO analysis, making it a hybrid toolset for performance engineering.
This repository contains three distinct research projects focused on language model development: pre-training (Autolab large-scale autoregressive model training), post-training (NanoChat synthetic task fine-tuning), and inference (llama.cpp optimization benchmarking). Each project follows a disciplined experimentation framework with Hugging Face Jobs integration, experiment tracking via TSV ledgers, and isolated variable modification policies. The pre-training project trains custom GPT architectures with value embeddings and MuonAdamW optimizer on climbmix-400b dataset, evaluating via bits-per-byte. Post-training uses a fixed NanoChat architecture for synthetic math/tool-use/multiple-choice tasks with staged SFT+RL training. Inference benchmarks GGUF model throughput via llama-server OpenAI-compatible API. All projects share common patterns for experiment workflow, HF job management, and results tracking while maintaining strict separation between research domains.
FastDFS is a mature, production-grade distributed file system designed for high-capacity file storage, access, and replication. It uses a two-tier architecture: lightweight tracker servers handle file metadata and load balancing, while storage servers manage actual file data in groups for horizontal scalability. The system exposes a lightweight protocol over TCP, with file access via file IDs (group/path) rather than traditional directory hierarchies. This repository includes the full server implementation (tracker and storage daemons), a comprehensive C client library, and a plethora of client SDKs in Go, Python, Rust, TypeScript, C++, JavaScript, Ruby, C#, and PHP, each implementing the same wire protocol. The codebase demonstrates strong engineering with features like connection pooling, retry logic with exponential backoff, metadata support, appender files (mutable uploads), slave files (thumbnails/transcodes), and trunk storage for small files. It ships with extensive operational tooling: CLI with interactive and batch modes, health checkers, Prometheus exporter, capacity planning, network diagnostics, file migration, deduplication, and benchmarks. The repository also includes Docker deployment scripts, systemd services, and Debian packaging for easy setup. Notable strengths include the consistent multi-language client ecosystem, the well-documented wire protocol, and the practical focus on operational readiness (monitoring, migration, repair). However, the codebase shows signs of technical drift: some client libraries (Ruby, C#) are incomplete or have stubbed methods, the core C code uses global state and manual memory management, and protocol constants are duplicated across all client implementations, creating maintenance overhead. The build system is a mix of legacy Makefiles and modern packaging, and some tools contain hardcoded paths or placeholder logic. Overall, FastDFS remains a robust solution for distributed file storage, particularly in environments where POSIX compatibility is not required and high throughput is paramount.
Youtu-Agent is a comprehensive AI agent framework developed by Tencent Cloud's Youtu Lab, designed for building, orchestrating, and evaluating LLM-powered agents. It provides a modular toolkit system, multi-agent orchestration patterns (simple, orchestra, orchestrator, workforce), a reusable configuration system (Hydra+YAML), and support for various environments (local shell, Docker, E2B sandbox, browser). The framework also includes a research-grade evaluation pipeline with benchmarks like GAIA, SWE-bench, WebWalkerQA, and a training-free GRPO module for agent self-improvement through experience distillation. Two front-end applications accompany the backend: a React-based chat UI and a Next.js evaluation analysis dashboard, both communicating via WebSocket and REST APIs. While the primary character is an application framework, Youtu-Agent has a strong data/research dimension: it ingests benchmark datasets, runs model-based judges, stores trajectories in SQL/PostgreSQL, and implements a meta-learning loop that extracts 'experiences' from agent behaviors. This hybrid nature makes it suitable both for production agent deployment and for conducting RL/benchmarking experiments. The codebase is well-organized into distinct modules (agents, tools, config, eval, practice, tracing, utils), uses modern Python packaging (uv, pyproject.toml), and includes extensive example configs and scripts. External dependencies include the openai-agents SDK, various LLM providers (OpenAI, Anthropic, etc. via litellm), search APIs (Serper, Jina, Google), document parsers (Chunkr, Unstructured), and MCP servers. Deployment supports Docker and multiple environment types. The repository is actively maintained with CI/CD, documentation (mkdocs), and a growing set of features.
The RDMA performance micro-benchmark suite (`perftest`) consists of command-line executables that measure bandwidth and latency for InfiniBand/RoCE verbs (SEND, WRITE, READ, ATOMIC) over multiple transport types (RC, UD, Raw Ethernet, etc.). The architecture follows a client-server model: parameters are negotiated via a TCP control channel, then RDMA transfers are executed, and results (bandwidth, latency percentiles, CPU utilization) are reported. The codebase is structured around shared modules in the `src/` directory: `libperftest.a` containing connection setup (`perftest_communication.c`), resource management (`perftest_resources.c`), CLI parsing (`perftest_parameters.c`), and data validation (`host_validation.c`). Memory backends (host, CUDA, ROCm, etc.) are abstracted through a vtable interface, and the negotiation protocol supports both legacy and key-value version. While not a data pipeline, the benchmarking methodology is rigorous. The code is professionally crafted but shows signs of long evolution: large monolithic files, extensive `#ifdef` gating, and manually maintained vendor-specific tables.
Perfolizer is a .NET performance-analysis toolkit extracted from BenchmarkDotNet. It provides statistical estimators (quantiles, MAD, variance), significance tests (Mann-Whitney, Welch), and specialized data structures such as Rqq for efficient range quantile queries. The repository is organized as a main class library, a demo executable, and two test projects, with CI that builds, tests, and publishes NuGet packages. Despite being labeled experimental, the codebase is actively developed, versioned, and shipped as a NuGet package.
OpenHGNN is a comprehensive research framework for Heterogeneous Graph Neural Networks (HGNNs) built on PyTorch and DGL. The repository implements 83+ graph neural network models across 17+ tasks including node classification, link prediction, recommendation, knowledge graph completion, and temporal graph learning. It serves as a benchmarking platform for academic research, providing standardized implementations of state-of-the-art architectures like HGT, MAGNN, CompGCN, and HERO. The framework uses a registration pattern for dynamic model/trainer discovery and supports distributed training, hyperparameter optimization via Optuna, and GraphBolt integration for large-scale graphs. While it has CLI entry points and setup.py, its primary purpose is research reproducibility rather than production deployment. The codebase demonstrates strong architectural patterns with BaseDataset, BaseTask, and BaseFlow abstractions that enforce consistent interfaces across 50+ model implementations. However, significant technical debt exists including version mismatches (setup.py declares 0.7.0 while README claims 0.9.0), hardcoded dataset paths, mixed DGL/PyG usage, and monolithic configuration files. The repository contains extensive training artifacts in output/ directories and documentation drift between README, setup.py, and Sphinx docs. Despite these issues, it remains a valuable resource for graph learning researchers seeking reproducible implementations of heterogeneous GNN architectures.
This repository is a data collection and presentation project for single-board computer (SBC) benchmarks maintained by Jeff Geerling. It consists of a set of automated benchmarking scripts written in Python using the pyinfra orchestration tool, which can be run against any SSH-accessible SBC to gather standardized performance metrics including Geekbench, HPL (LINPACK), power consumption, memory bandwidth, disk I/O, GPU benchmarks, and AI inference performance. A secondary set of Ansible playbooks provides a simpler but less comprehensive alternative. All collected results are manually curated into a YAML file (web/data/results.yml) and rendered into a static Hugo website with a sortable comparison table. The repository serves as a public reference for SBC performance comparisons. While primarily a data research project focused on benchmarking hardware, it also provides reusable infrastructure for running benchmarks on any Linux or macOS system.
TechEmpower Framework Benchmarks is a large-scale, multi-language benchmark suite for comparing the performance of web application frameworks. The repository contains hundreds of independent framework implementations (e.g., Go, Rust, Java, C#, Python, PHP) each implementing the same standard set of HTTP endpoints: JSON serialization, plaintext, single/multiple database queries, updates, fortunes (HTML), and cached queries. The core benchmark toolset (written in Python) orchestrates Docker-based builds, runs load tests via wrk, and verifies response correctness. The codebase is a federation of many small applications rather than a single system, with each framework's code living in its own directory under `frameworks/<language>/<name>/`. The architecture is highly standardized: every framework exposes the same routes, uses the same database schema (World and Fortune tables), and follows common optimization patterns like connection pooling, prepared statements, batch updates, and template caching. While the repository is actively maintained and the toolset is well-structured, the quality of individual framework implementations varies widely—some are expertly tuned for performance, while others are stubs or use outdated libraries. The primary value of this repository is as a reference for comparative performance numbers and as a collection of idiomatic, performance-focused code examples across many languages and frameworks.
This repository is a curated collection of recipes and tools for Go development, organized into categories such as testing, dependencies, code visualization, code generation, refactoring, errors, build, assembly, deployment, execution, monitoring, benchmarking, documentation, security, static analysis, and guides. The actual codebase is minimal: a Go module with a three-line main.go that generates the README.md from a structured YAML file (page.yaml) using the mdpage library. The primary value is the content itself, which serves as a reference for Go developers to discover and utilize various tools and techniques. The project is actively maintained, with up-to-date dependencies and a simple CI workflow for markdown link validation.
Renate is a Python library for continual learning that provides implementations of various algorithms (Experience Replay, DER, Super-ER, etc.) to mitigate catastrophic forgetting when updating neural networks on sequential data. It features a modular architecture separating models, data handling, update strategies, memory buffers, and drift detection, enabling users to customize continual learning pipelines. The framework supports local execution and cloud deployment via Amazon SageMaker, includes hyperparameter optimization through Syne Tune, and offers benchmarking tools for evaluating continual learning methods across diverse datasets and scenarios. Key reusable components include a memory buffer system with reservoir and class-balancing strategies, Maximum Mean Discrepancy (MMD) based drift detectors for monitoring data distribution shifts, and a composable loss component system for constructing complex continual learning objectives. The library integrates with PyTorch Lightning for training and provides abstract base classes for models and data modules to facilitate extension. Although the project is explicitly archived and no longer maintained, the codebase demonstrates solid engineering practices with well-defined interfaces and decoupled components. It has been tested on vision and NLP benchmarks (CIFAR, CLEAR, DomainNet, Hugging Face datasets) and includes synthetic scenario generators for controlled experimentation. The archival status indicates that users should expect no further updates or security patches, but the core functionality remains usable for continual learning research and applications.
This is the TigerGraph ecosystem repository, a large collection of tools, connectors, demos, benchmarks, and tutorials centered around the TigerGraph graph database platform. It is not a single application but an aggregation of community and in-house developed components. The dominant character is application: it includes production-grade ETL connectors (Spark DataSource V2, JDBC driver, Kafka Connect, Hadoop MapReduce), a multi-version GSQL CLI client, a Go-based REST API for remote GSQL management, a Kubernetes operator, and numerous demos showcasing graph analytics (PageRank, community detection, geospatial search, fraud detection, GraphRAG). There are also benchmarking suites (LDBC SNB) and sample data. The codebase exhibits high craftsmanship in the connector and client libraries, with robust retry logic, token management, streaming JSON parsing, and schema inference. However, it suffers from massive code duplication across GSQL client versions, incomplete JDBC driver implementations, hardcoded paths and credentials in demos, and outdated vendored libraries (Leaflet 0.7.7, moment-timezone 2016). Overall, it is a functional but uneven collection; the core connectors are reusable and well-engineered, while many demo scripts are fragile and require manual configuration.
This repository is the archived reference implementation of the CAI (Cybersecurity AI) framework by Alias Robotics. It is a comprehensive, open-source platform for building and deploying AI-powered offensive and defensive security automation. The codebase provides a full-stack agent framework including a Python SDK (Agent, Runner, tools, handoffs, guardrails, tracing, voice, MCP), a CLI/REPL with rich interactive features, a Textual-based TUI, a FastAPI REST API server, and a benchmarking suite (CAIBench) with cyber ranges and CTF challenges. The framework supports multiple LLM providers (OpenAI, LiteLLM, custom) and includes specialized agents for red teaming, blue teaming, bug bounty, DFIR, network analysis, and more. It also features continuous operations (24/7 autonomous loops), session management, cost tracking, and extensive refusal testing (400+ scenarios). While the repository is now archived and will not receive further updates, the SDK components (function_tool decorator, Handoff abstraction, tracing processors, voice pipeline) are well-crafted and could be extracted for reuse in other agent-based projects. The codebase is large and monolithic, with some global mutable state and test duplication, but overall demonstrates strong engineering practices in the cybersecurity AI domain.
This repository is a deprecated ONNX Model Zoo containing thousands of pre-trained models (mostly computer vision, some NLP and generative AI) exported to the ONNX format from PyTorch (timm, torchvision) and other frameworks. It also contains turnkeyml benchmark artifacts (turnkey_stats.yaml) that record export success, model metadata, ONNX operator counts, and system environment for each model across opsets 16–18. The primary purpose was to provide a centralized collection of ready-to-use ONNX models for inference and to validate cross-version ONNX export compatibility. However, the repository is explicitly deprecated (LFS disabled July 2025) and users are redirected to Hugging Face. The codebase consists almost entirely of binary model files and YAML metadata; the only reusable software components are a few preprocessing scripts (e.g., imagenet_preprocess.py, face_preprocess.py) in the validated/ directories. There is no application logic, no API, no training pipelines—this is purely a model distribution and benchmark data repository. Reproducibility is low due to hardcoded paths, frozen env snapshots that are out of date, and reliance on external model sources whose weights may have changed.
The Web Tooling Benchmark is a performance measurement suite designed to evaluate JavaScript-related workloads in common web developer tools. It executes benchmarks for tools like Babel, TypeScript, UglifyJS, and others, measuring their throughput in runs per second. The architecture consists of a Node.js CLI interface and a browser-compatible bundle, both driven by a benchmark suite that runs each tool's core functionality on standardized third-party code samples. Key entry points are src/cli.js (for Node.js) and src/bootstrap.js (for browser). The suite reads input files from the third_party directory, processes them via each tool's API, and aggregates results using geometric mean. External dependencies include the benchmarking framework, various JavaScript tools (acorn, babel, typescript, etc.), and utilities for file system virtualization and output formatting.
This repository is a comparative benchmark of web-to-desktop application frameworks (Electron, Tauri, NW.js, Wails, NodeGui, Neutralino, Flutter, .NET MAUI). It provides automated benchmarking infrastructure that builds empty apps for each framework, measures build size, build time, memory usage, and startup duration across multiple operating systems, and presents the results in formatted README tables and Mermaid graphs. The repository also tracks GitHub stars and forks over time. The codebase is primarily a data collection and presentation project, with the runner scripts orchestrating the benchmarks and generating the comparison documents. The benchmark apps themselves are minimal template applications. The project is actively maintained with recent commits and automated CI workflows, but the runner code contains several technical issues (undefined functions, deprecated patterns) indicating high technical drift. The data is bundled in the repository, making results immediately accessible without running benchmarks.