Heretic is a command-line tool that automatically removes censorship (safety alignment) from transformer-based language models without expensive post-training. It implements an advanced form of directional ablation (abliteration) combined with a TPE-based parameter optimizer (Optuna) to find high-quality ablation parameters that minimize refusals while preserving the original model's capabilities. The tool supports dense, MoE, hybrid, and multimodal architectures, and can produce decensored models that rival human-crafted abliterations. The architecture is a monolithic Python application built on PyTorch, Hugging Face Transformers, and PEFT. It features a plugin system for extensible scorers (e.g., keyword rate, KL divergence, benchmark scores) and modifiers (e.g., standard abliteration, Arbitrary-Rank Ablation). The workflow involves loading a model, computing residual directions from good/bad prompt datasets, running multi-objective optimization trials to tune ablation parameters, and exporting the modified model as a merged full model or LoRA adapter. The tool also includes comprehensive reproducibility support, allowing others to exactly reproduce generated models. Key entry points are the `heretic` CLI command (defined in `src/heretic/main.py`), which orchestrates the entire pipeline. The codebase demonstrates strong engineering practices: extensive type hints, pydantic-based configuration with CLI/env/TOML sources, a well-designed plugin contract, and thorough testing for cross-platform reproducibility. External dependencies include PyTorch, Transformers, Accelerate, bitsandbytes, Optuna, PEFT, Hugging Face Hub, and several others, all pinned via uv.lock.
This repository provides a build system for version-stable Rocker Docker images for R, including r-ver, rstudio, tidyverse, verse, shiny, shiny-verse, and geospatial. The core architecture uses R scripts to generate Dockerfiles from templates and docker-bake.json files from version matrices, enabling reproducible builds with fixed CRAN snapshots, RStudio Server versions, and CTAN dates. The CI/CD pipeline (GitHub Actions) automatically updates container definitions, builds multi-platform images (linux/amd64 and linux/arm64), and publishes them to Docker Hub and GitHub Container Registry. The repository also contains a collection of standalone installation scripts (rocker_scripts) that can be reused in downstream Dockerfiles to install R, RStudio Server, Shiny Server, tidyverse packages, geospatial libraries, Python, Julia, Jupyter, and more. These scripts are parameterized, idempotent, and follow best practices for Docker image layering. The build system itself is a well-engineered example of code generation and matrix-based CI, though it is tightly coupled to the Rocker project's specific versioning scheme.
This repository is a curated collection of 177+ domain-specific scientific 'skills' for AI agents, following the Agent Skills standard. Each skill is a self-contained module providing a CLI tool, documentation, and tests for a specific scientific or technical task—ranging from single-cell RNA-seq analysis (scanpy) and molecular docking (diffdock) to clinical report validation and ISO standards readiness. The architecture is a monorepo of independent skills, each with its own scripts/, references/, assets/, and tests/. Skills are designed to be safe, bounded, and reproducible: they validate inputs before execution, enforce resource limits, avoid network access unless explicitly required, and produce structured JSON reports. The repository also includes a shared test infrastructure (skill_contract) and a meta-skill (autoskill) that can automatically propose new skills from screen activity. While primarily an application library for AI agents, it has strong data/research characteristics due to its heavy use of scientific data formats and validation of research artifacts.
ClawBench is a rigorous execution-first benchmarking framework for evaluating AI agents (particularly OpenClaw) on multi-step tasks. The system orchestrates agent runs through a gateway or containerized native evaluation, scores performance across four axes (completion, trajectory, behavior, advisory judge), and generates detailed diagnostic reports. Its architecture includes a job queue with HuggingFace Dataset persistence, a Gradio web app for interactive leaderboards, and a comprehensive CLI for running evaluations and analyzing results. The framework supports multiple agent runtimes via a pluggable adapter pattern (OpenClaw, Hermes, Claude Code, Codex) and provides a canonical task schema for agent-agnostic task definitions. Beyond basic scoring, ClawBench applies dynamical-systems and statistical methods to agent behavior: trajectory regime classification (trapped, limit-cycle, diffusive), Kaplan-Meier survival analysis, constraint index computation, and variance decomposition. It includes a configuration-space benchmarking system (v0.5) that fingerprints plugin profiles for cold-start prediction and fANOVA factor analysis. The codebase is well-tested with over 60 test files, uses Docker for reproducibility, and includes CI/CD pipelines for automated evaluation campaigns. However, the system is complex and has some hardcoded paths and environment variables tied to specific deployment contexts (Crabbox, OpenClaw gateway), and its heavy dependency on external services introduces maintenance risk. Overall, it is a production-grade benchmark with sophisticated analysis capabilities.
Paper2Code is a research system that automatically generates runnable code repositories from machine learning research papers. The core pipeline, PaperCoder, uses a multi-agent LLM architecture (planning→analysis→coding) implemented as a sequence of Python scripts orchestrated via shell commands. It accepts papers in PDF (converted to JSON via s2orc-doc2json), JSON, or LaTeX format, and produces a complete Python project with config, model definitions, training loops, and evaluation scripts. The codebase supports both OpenAI API (o3-mini by default) and open-source models via vLLM (e.g., DeepSeek-Coder-V2-Lite-Instruct). It includes a debug stage that iteratively fixes runtime errors using LLM-generated SEARCH/REPLACE patches, and an evaluation module that scores generated repositories (reference-free or reference-based) using LLM judgment. The repository also bundles the Paper2Code benchmark dataset (90 papers from top ML conferences with gold repositories) and prompts for evaluation. Although primarily an application, it has research artifacts (evaluation scripts, prompt templates, benchmark data) that give it a hybrid character.