This repository is the official companion code for the book *Build a Large Language Model (From Scratch)* by Sebastian Raschka (105k+ stars). It provides a step-by-step, educational implementation of GPT-2 and its modern variants (Llama 3, Qwen3, Gemma 3/4, Olmo3) entirely in PyTorch, organized by book chapters (2–7 plus appendices). The codebase is a mix of Jupyter notebooks for interactive learning and Python modules that form a reusable pip package (`llms-from-scratch`), accompanied by extensive CI/CD pipelines, unit tests, and performance analysis scripts. Architecturally, the core is a modular transformer library that includes tokenization (BPE, tiktoken, custom), multi-head attention with multiple variants (GQA, MLA, SWA, DSA, cross-layer KV sharing), feedforward blocks, normalization (LayerNorm, RMSNorm), and LoRA. The code supports pretraining from scratch, classification finetuning (spam, IMDB), instruction finetuning (with/without LoRA), and preference tuning (DPO). It can load pretrained weights from OpenAI, Hugging Face, and local checkpoints. The package exposes clean interfaces (`GPTModel`, `generate`, `train_model_simple`) and includes bonus materials for KV cache implementations, mixture-of-experts, memory estimators, and user interfaces (Chainlit). While primarily educational, the code exhibits production-quality patterns: isolated attention modules, configurable model architecture, well-structured training loops, and robust weight loading with shape validation. The repository is actively maintained with recent commits, cross-platform support (Linux, macOS, Windows), and a strong emphasis on reproducibility (fixed seeds, deterministic settings, environment definitions via pixi/uv/conda). Duplication exists across bonus notebooks and a few hardcoded paths, but the core package and test suite are well-coordinated.
Deep-Live-Cam is a real-time face swapping desktop application that uses ONNX Runtime with insightface models to swap faces in live webcam feeds or pre-recorded images/videos. The core pipeline involves face detection (RetinaFace), face recognition (ArcFace), and a face swap model (inswapper_128), with optional face enhancement via GFPGAN or GPEN-BFR ONNX models. The application provides a PySide6 GUI for selecting source/target faces, toggling options (mouth mask, many faces, Poisson blending, sharpening, temporal interpolation), and managing camera selection. It supports multiple execution providers: CUDA (NVIDIA), CoreML (Apple Silicon), DirectML (AMD/Intel), OpenVINO (Intel), and CPU fallback. Video processing can run in-memory via FFmpeg pipes (avoiding disk I/O) or via traditional frame extraction to PNG files. The architecture is modular: frame processors (face swapper, enhancers) follow a common interface, and the codebase includes optimizations for Apple Silicon (CoreML subgraph optimization, Pad decomposition, Shape/Gather folding) and CUDA (IO binding, CUDA graphs). The project also includes a benchmark pipeline and unit tests for core components.
This repository is a multi-platform conference deadline tracking ecosystem centered on CCF (China Computer Federation) rankings, also covering CORE and THCPL. The primary production system is a Rust/Leptos web application compiled to WebAssembly and served as a static site (ccfddl.com), providing interactive filtering, countdowns, timezone conversion, favorites, calendar subscriptions, and email reminders. A Cloudflare Worker backend handles GitHub OAuth authentication, D1 SQLite database for user data (stars, message wall, email reminders), and serves a Model Context Protocol (MCP) endpoint for AI agent integration. Additional clients include a Python CLI, a Chrome extension (Manifest V3), and an iCal/RSS feed generator. The data layer consists of curated YAML files for conference metadata and historical acceptance rates, which are validated, merged, and transformed into JSON by a Python build pipeline. The repository also includes a static SEO page generator, a deterministic social media posting skill for X/Twitter, and agent skills for AI-assisted data maintenance. While the core value is the curated conference data, the dominant character is a software application with multiple user-facing interfaces and a serverless backend.
Langchain.rb is a Ruby gem for building LLM-powered applications with a unified interface to multiple LLM providers and vector databases. It provides a modular architecture supporting Retrieval Augmented Generation (RAG), AI agents with tool usage, document processing, and prompt templating. The gem includes a Rails engine for deep Rails integration and is actively maintained with frequent updates. The system enables developers to create interactive AI applications by combining LLMs, external tools, and vector search capabilities.
AutoGPT is a full-stack SaaS platform for building, deploying, and running AI agents. It provides a visual agent builder (blocks), a copilot chat interface that can generate and orchestrate agents from natural language, a marketplace for sharing agents and skills, team/experts management with scheduling and credential stacks, and a subscription billing system with Stripe. The architecture is a multi-service backend (FastAPI, PostgreSQL, Redis) with a Next.js frontend, supporting real-time WebSocket execution events, OAuth integrations with 45+ providers, and feature flags via LaunchDarkly and PostHog. The platform is dual-licensed (Polyform Shield for platform code, MIT for classic code) and supports both hosted SaaS and self-hosted deployment. The codebase is organized into two major areas: `autogpt_platform/` (the current platform) and `classic/` (legacy AutoGPT and Forge). The platform backend uses a block-based execution system where each integration (Google, Slack, GitHub, etc.) is a self-contained block with typed I/O schemas, cost tracking, and credential management. The frontend features a rich component library with dynamic form rendering (RJSF), responsive dialog/drawer patterns, and extensive state management via TanStack Query and Zustand. Key entry points include the copilot chat UI, the agent library, marketplace, team management, and admin panels. Primary data flows involve user input through the copilot or builder, which triggers graph execution via the execution manager, with events streamed over WebSocket. Credits are deducted per block execution, and billing is handled through Stripe subscriptions with trial management. Analytics events are sent to PostHog and DataFast for product metrics. The system relies on external dependencies including PostgreSQL (with pgvector), Redis, Stripe, LaunchDarkly, PostHog, Sentry, and numerous third-party APIs for integrations.
Manifold is an AI agent platform enabling multi-step workflow automation through teams of AI assistants. The system provides a chat interface for interacting with configurable specialist agents, a visual workflow editor (WARPP) for creating reusable automation, and an experimentation playground for testing prompts and models. Built with a Go backend and Vue.js frontend, it features extensible tool systems, layered memory architectures (belief, evolving, decision), MCP integration for external tools, and comprehensive observability via OpenTelemetry. The architecture emphasizes modularity with clear separation between agent execution, persistence, tool discovery, and memory subsystems.
This repository is a full-stack Next.js application that serves as the world's largest open-source prompt library for AI. It provides a platform for users to create, share, discover, and manage prompts for various AI models (ChatGPT, Claude, Gemini, etc.). The core functionality includes CRUD operations for prompts with types (text, image, video, audio, skill, taste), voting, commenting, versioning, change requests, collections, and moderation tools. It features AI-powered capabilities such as semantic search via OpenAI embeddings, prompt improvement, translation, quality checking, and media generation through plugin-based providers (Fal.ai, Wiro.ai). The application supports multi-tenancy for self-hosting with configurable branding, authentication plugins (OAuth, OIDC, GitHub, Apple, Google), and storage backends (S3, DO Spaces). Beyond the web app, it includes a CLI tool (packages/prompts.chat) for browsing and running prompts locally, a Raycast extension for quick access, and an MCP server for integration with AI coding assistants. Additionally, it bundles an interactive educational book on prompt engineering (28+ chapters) and a gamified kids learning section, both fully localized into 17 languages. The codebase is well-structured with a plugin registry pattern, configuration-driven design, and extensive use of modern Next.js patterns (App Router, server components, streaming).
claude-mem is a persistent memory system for AI coding assistants, primarily targeting Claude Code but also supporting Codex, Cursor, Windsurf, OpenClaw, and other platforms. It captures observations, summaries, and user prompts from coding sessions via lifecycle hooks (SessionStart, PostToolUse, etc.), stores them in SQLite (local) or Postgres (server), and injects relevant context into future sessions to maintain continuity across agent interactions. The system operates as a background worker daemon (Bun) or a multi-user server (Express + BullMQ + Valkey) with API key authentication, team/project scoping, and rate limiting. Architecturally, claude-mem is a hybrid of a CLI plugin and a standalone server. The CLI component integrates into IDEs via hook scripts and MCP servers, while the server component provides REST APIs for memory management, search, and cloud sync. It features a dual-storage backend (SQLite for local, Postgres for server) with vector search via ChromaDB and full-text search via SQLite FTS5. The system includes a React-based viewer for browsing memory feeds, telemetry (opt-out PostHog), and a Cloudflare Workers-based sync hub for cross-device synchronization. The codebase demonstrates strong engineering practices with well-abstracted components (search strategies, provider abstraction, atomic file operations, telemetry scrubbing) but carries some technical debt from its dual-runtime evolution (legacy worker vs. server-beta) and extensive Windows-specific workarounds. It is actively maintained with frequent releases and comprehensive test coverage.
This repository provides a minimal, from-scratch implementation of a Vision-Language Model (VLM) called MiniMind-V, designed for educational purposes and low-resource training. It ingests the ALLaVA-4V dataset (Parquet format with image bytes and JSON conversations) for pretraining (1.27M caption pairs) and supervised fine-tuning (2.9M mixed instruct/caption/text). The architecture follows a LLaVA-1.5 style: a frozen SigLIP2 visual encoder (256×256, 64 patch tokens) projects via a 2-layer MLP into a MiniMind LLM (with optional Mixture-of-Experts). Training freezes the visual encoder and trains only the projector and selected LLM layers, achieving ~3 RMB / 2 hours on a single RTX 3090. The codebase includes training scripts (pretrain and SFT), evaluation, a Gradio web demo, and model conversion to HuggingFace Transformers format. While primarily a research/educational project, it demonstrates a clean, modular VLM pipeline that can be adapted for other multimodal tasks.
byol-pytorch is a Python library that implements Bootstrap Your Own Latent (BYOL), a self-supervised learning method for image representations. The core contribution is the `BYOL` module, which wraps any PyTorch image backbone (e.g., ResNet) and trains it without labels by maximizing agreement between two augmented views of the same image. The library derives its behavior from code rather than promises: it computes a cosine loss between online predictions and target projections, where the target encoder is an exponential moving average of the online encoder (or the online encoder itself for the SimSiam variant). It also supports optional Simplicial Embeddings (SEM) and custom augmentation pipelines.
This repository is a comprehensive, curated collection of cybersecurity references, scripts, tools, code, labs, and training resources, primarily authored by Omar Santos. It serves as supplemental material for his books, video courses, and live training, covering a vast range of domains including offensive security, defensive security, application security, cloud/container security, hardware/embedded security, AI/ML security, and certification preparation (Cisco, CompTIA, ISC2, etc.). The repo is hybrid in nature: while it contains hundreds of runnable scripts and tools (e.g., network scanners, exploit examples, API security testers, AI-powered log analyzers), its dominant character is that of an educational knowledge base and reference library rather than a deployable application. The content is organized into clear domain directories, with extensive READMEs acting as curated indexes of tools, papers, talks, and labs. The AI security section is particularly well-developed, including course materials on agentic AI, prompt engineering, and the Foundry Security Spec. The repository is actively maintained, with recent commits and automated CI workflows, though the breadth of content introduces risks of link rot and technology churn in fast-moving areas like AI frameworks and certification blueprints.
rclip is a command-line semantic photo search tool that indexes local image collections using a CLIP model (OpenCLIP ViT-B/32-256) and supports natural-language, image-to-image, and combined queries with boolean operations. The core architecture is monolithic: a Python package with modules for model inference (ONNX/CoreML), SQLite-based feature caching, image preprocessing, and a Textual terminal user interface for interactive browsing. It is fully offline and privacy-preserving, supporting a wide range of image formats including HEIC, raw (CR2, DNG, ARW), and standard web formats. The tool is distributed via PyPI, Snap, AppImage, Homebrew, and Windows MSI, with careful compliance management for third-party licenses, codecs, and model artifacts. It includes an extensive test suite (unit and end-to-end), CI/CD via GitHub Actions, and a compliance toolkit that enforces license policies during builds. The codebase is well-structured and demonstrates good engineering practices, though it relies on a pinned version of textual-image that uses a private API, introducing some technical drift risk.
Triage is a Python framework for end-to-end machine learning workflows focused on temporal prediction problems, particularly in social good domains. It provides modular components for feature generation (architect), model training/evaluation (catwalk), experiment orchestration, model selection (audition), and post-modeling analysis. The system uses PostgreSQL for storing experiment metadata, feature matrices, and model predictions, with SQLAlchemy ORM and Alembic migrations managing the schema. Key entry points include the CLI (`triage experiment`, `triage audition`) and `manage.py` for development tasks. The architecture emphasizes reproducibility through configuration-driven experiments, temporal validation to prevent lookahead bias, and metadata tracking. While powerful, the system has complexity from multiple interdependent components and external service dependencies (PostgreSQL, Redis for RQ).
BitNet is a PyTorch library implementing the BitNet architecture for 1-bit transformers, as described in the paper "BitNet: Scaling 1-bit Transformers for Large Language Models". The library provides drop-in replacements for standard linear layers with BitLinear layers that use quantized weights and activations to enable efficient computation. It includes components such as BitMGQA (Multi-Grouped Query Attention), BitFeedForward, and full transformer models (BitNetTransformer), as well as adapters for other architectures like Mamba and Mixture of Experts. The library also offers utilities to convert existing Hugging Face or PyTorch models to use BitLinear layers and includes an optimized CUDA kernel for low-precision GEMM operations to accelerate inference. The architecture is modular, with the core BitLinear layer encapsulated in `bitnet/bitlinear.py`, which can be used independently. Higher-level components like attention and feed-forward networks are built using BitLinear and are designed to be reusable in other projects. The library supports training and inference workflows, as demonstrated by the provided example scripts and training routine for the enwik8 dataset. With active maintenance and a focus on performance through CUDA optimization, BitNet serves as a specialized tool for researchers and practitioners interested in extremely low-precision deep learning models.
SILENTCHAIN AI™ Community Edition is a Burp Suite extension that performs AI-powered passive vulnerability scanning of HTTP traffic. It intercepts requests/responses through Burp's proxy, sends them to configurable AI providers (Burp AI, Ollama, OpenAI, Claude, Gemini, Azure), and generates OWASP Top 10 vulnerability findings. The extension features a modular architecture with a thread pool for background AI processing, Swing-based UI integrated into Burp Suite, and persistence for settings via Burp's preferences API. Key capabilities include bidirectional data sanitization to prevent leaking secrets to AI providers, robust JSON parsing for handling LLM output inconsistencies, layered HTTP filtering (scope, content-type, rate limiting, URL deduplication), and real-time UI updates showing findings, task status, and statistics.
Xinference is a production-grade model serving platform that provides OpenAI-compatible REST APIs for a wide range of AI models including LLMs, embeddings, rerank, image generation, video, audio/TTS, and world models. It supports multiple inference backends (vLLM, SGLang, transformers, MLX, llama.cpp) and can be deployed as a single process or a distributed cluster using an actor-based supervisor/worker architecture (xoscar). The system includes a web UI for model management, authentication/authorization with OAuth2 scopes, token-aware routing for load balancing, and monitoring capabilities. It bundles many third-party model implementations for turnkey deployment, which reduces setup friction but introduces maintenance overhead from vendored code. The architecture is a hybrid of a monolithic FastAPI server and a distributed actor system. Key entry points are CLI commands (xinference, xinference-supervisor, xinference-worker) and the REST API. The frontend is a Next.js application statically exported and served by the backend. The codebase demonstrates strong patterns in modular route registration, plugin-style model registries, and streaming response handling, but also carries significant technical debt from heavy vendoring, duplicated code across third-party integrations, and version-sensitive patches to upstream libraries like vLLM.
This repository is a large curated dataset rather than a software application or data pipeline. It contains a multilingual, awesome-list style directory of 3,527 generative AI applications organized into 80 categories, with README.md as the canonical content store. Each entry follows a reproducible markdown record schema: application name, screenshot URL, short description, visit redirect link, and a separate pricing/information link. Categories span the generative AI ecosystem, including AI agents, chatbots, image/video generation, code assistants, legal, accounting, and gaming. The data_research classification reflects the fact that the primary asset is structured content, not executable code; there are no notebooks, model files, or ETL scripts in the repository.
pytorch-forecasting is a deep learning library for time series forecasting, providing implementations of state-of-the-art models such as Temporal Fusion Transformer, N-BEATS, N-HiTS, DeepAR, PatchTST, TiDE, TimeXer, and many others. Built on PyTorch and PyTorch Lightning, it offers a complete pipeline from data preprocessing (TimeSeriesDataSet, encoders, samplers) through model training (with GPU support, callbacks, hyperparameter tuning via optuna) to prediction and evaluation (point, quantile, and distributional metrics). The library supports multivariate forecasting, static and time-varying covariates, categorical features, and multiple targets. A major v2 refactoring is underway, introducing a new data module layer and simplified package interface, though the v2 API is explicitly marked as experimental and unstable. The codebase is well-tested with automated integration tests across all models and metrics, and employs a composition pattern for metrics (supporting weighting, aggregation, and distribution losses) and a layered architecture for models (base classes, package containers, building blocks).
This repository is a comprehensive, production-focused AI system design guide, not a software application. It contains over 170 markdown files organized into 19 chapters covering the full lifecycle of building and operating AI systems: foundations (transformers, tokenization), model landscape (selection, pricing), training and adaptation (fine-tuning, LoRA, RLHF), inference optimization (KV cache, speculative decoding, batching), prompting and context engineering, retrieval systems (RAG, chunking, vector databases, hybrid search, GraphRAG), agentic systems (ReAct, MCP, multi-agent orchestration, durable execution), memory and state management, frameworks and tools (LangChain, LangGraph, LlamaIndex, DSPy, Semantic Kernel), document processing, infrastructure and MLOps (CI/CD, AI gateways, FinOps), security and access control, reliability and safety (guardrails, ensemble methods, governance), evaluation and observability (LLM evaluation, benchmarks), design patterns and anti-patterns, and 20+ real-world case studies. It also includes interview preparation materials (question bank, answer frameworks, whiteboard exercises, job market trends) and transition guides for engineers moving into AI roles. The content is current as of mid-2026, referencing specific model versions, pricing, and protocols (MCP 2.0, A2A). While it contains no executable code, it serves as an authoritative reference for engineers designing and deploying AI systems in production.
This repository is a full-stack marketing demo application powered by Google Cloud generative AI (Vertex AI Gemini, Imagen). It provides an end-to-end workflow for generating campaign content (emails, social media posts, website copy, images), analyzing audiences via natural language to BigQuery, and managing assets across platforms. The frontend is a modern Angular 17 single-page application with Firebase authentication and hosting, while the backend is a FastAPI Python service deployed on Cloud Run. Terraform infrastructure-as-code provisions all GCP resources including BigQuery datasets, Vertex AI Search datastores, Dialogflow CX agents, and Google Drive folder templates for content export. The application showcases marketing use cases like trendspotting via GDELT news summarization, audience insights using structured and unstructured data, and interactive image editing with inpainting/outpainting. While primarily a demo application, the repository also contains several Jupyter notebooks exploring model tuning, text-to-SQL, and clustering explanations, making it a hybrid resource for both application developers and data scientists.