An Obsidian plugin (Gemini Scribe) that integrates multiple AI providers (Gemini, Anthropic, OpenAI, Ollama) for chat, agent-based automation, and knowledge management within the Obsidian vault. The system provides a modular architecture with an agent loop that executes tools (vault operations, web search, RAG semantic search, image generation, deep research) with confirmation flows and diff previews. It features a declarative settings UI with per-feature provider routing, a skill system following the agentskills.io spec, MCP server integration, scheduled tasks, lifecycle hooks, and an evaluation harness for benchmarking agent behavior. The codebase demonstrates strong engineering practices: thorough unit testing (73%+ line coverage), clean module boundaries with leaf modules to prevent circular imports, and reusable patterns like typed event buses, retry decorators, and cancellation-safe streaming.
This repository provides a production-ready, full-stack Retrieval-Augmented Generation (RAG) chat application built with Python FastAPI and React TypeScript. It demonstrates how to use Azure OpenAI (or OpenAI.com, or Ollama) to answer questions about data stored in Azure Database for PostgreSQL with the pgvector extension. The system supports both a simple RAG flow (embedding + direct search) and an advanced flow that uses OpenAI function calling to rewrite queries and apply column filters (brand, price). It features hybrid search combining full-text search and vector similarity via Reciprocal Rank Fusion. The backend is a monolith serving API endpoints and a React frontend, designed for deployment to Azure Container Apps using the Azure Developer CLI. The codebase includes comprehensive unit tests, end-to-end tests with Playwright, evaluation scripts, load testing with locust, and safety evaluation using Azure AI Red Teaming. The architecture is cleanly separated into reusable modules, making it a solid reference architecture for building RAG applications on Azure.
Langflow is a full-stack, open-source platform for building and deploying AI-powered agents and workflows. It provides a visual flow builder (React frontend), a Python backend (FastAPI/SQLModel), a CLI executor (LFX), an MCP server for AI tool integration, and a Python SDK. The codebase is a monorepo with multiple packages: the main `langflow` distribution, `langflow-base` (core backend), `lfx` (lightweight executor/CLI), `langflow-sdk` (API client), `langflow-stepflow` (step-based execution), and extension bundles (e.g., lfx-duckduckgo, lfx-arxiv, lfx-ibm, lfx-docling). The backend uses SQLModel with Alembic migrations, Celery for background tasks, FastAPI for REST endpoints, and a plugin/extension system for bundles. The frontend is a React/TypeScript app built with Vite, React Flow, Zustand, and Tailwind. The architecture supports multi-worker deployments, auto-login, JWT auth, rate limiting, MCP server integration, and a rich component library for LLMs, vector stores, tools, and data processing.
Chatbox is a cross-platform desktop AI chat client (Windows/macOS/Linux) built with Electron, React, and TypeScript. It supports 30+ AI providers (OpenAI, Claude, Gemini, DeepSeek, Ollama, etc.) via a unified provider registry and model abstraction layer. The application features agent mode with tool use, sandboxed code execution, MCP server integration, knowledge base RAG, web search, image generation, and a skills marketplace. It uses a hexagonal architecture with ports/adapters in a core package (`@chatbox/core`) that is host-agnostic, and React bindings in `@chatbox/react`. The renderer uses TanStack Router, Zustand/Jotai for state, and Mantine/MUI for UI. The main process handles IPC, OAuth flows, file system sandbox, and analytics. The codebase is large and well-tested (Vitest unit tests, integration tests, contract tests) with a focus on offline-first, local storage, and cross-platform compatibility (desktop, web, mobile via Capacitor). The application is actively maintained with a commercial 'Pro' version that shares code with this open-source Community Edition. Notable engineering patterns include a clean port/adapter separation, in-memory test doubles for all repositories, a session write coordinator that serializes updates and cancels stale queries, a draft tokenization system with web workers, and a provider registry that allows dynamic model discovery. The codebase has some technical debt: multiple UI libraries, legacy migration paths, and commented-out code, but overall demonstrates strong craftsmanship and reusability of its core abstractions.
ColossalAI is a comprehensive distributed training and inference framework for large-scale deep learning models, providing multiple parallelism strategies (data, tensor, pipeline, sequence, mixture-of-experts), ZeRO memory optimization, automatic parallelization via FX tracing, and high-performance custom CUDA/Triton kernels. The architecture is library-based with a plugin-style booster for orchestrating training and inference, a shardformer for model-level parallelism policies, and a legacy subsystem for older APIs. It includes application-level tools: ColossalChat for RLHF (PPO, DPO, GRPO), ColossalQA for retrieval-augmented question answering, ColossalEval for LLM evaluation with GPT judges, and Colossal-LLaMA for domain-specific LLM training. The codebase is actively maintained with recent commits, but has notable technical drift due to outdated dependency pins in some examples and a large legacy namespace that coexists with newer implementations.
ScrapeGraphAI is a Python library that extracts structured data from websites and local documents using LLM-driven graph pipelines. Instead of a traditional web scraper with hardcoded selectors, users define extraction through natural-language prompts and optional Pydantic schemas; the library then executes a directed graph of nodes such as fetching, parsing, chunking, and answer generation. The public API is a set of graph classes (SmartScraperGraph, SearchGraph, JSONScraperGraph, CSVScraperGraph, SpeechGraph, ScriptCreatorGraph, etc.) in scrapegraphai/graphs, each composing reusable nodes from scrapegraphai/nodes and running them on a BaseGraph execution engine. It is consumed programmatically in Python and supports many LLM providers (OpenAI, Ollama, Groq, Mistral, Anthropic, DeepSeek, xAI, NVIDIA, MiniMax) and multiple fetching backends (Playwright, Selenium, Chromium, BrowserBase, Scrape.do). The architecture is a lightweight DAG framework: nodes have typed inputs/outputs, can branch conditionally, iterate, merge results, and accumulate cost/token metadata; BaseGraph walks the graph, handles state, and collects execution information. The repository also contains document loaders, model token/cost registries, prompt templates, telemetry, proxy rotation, screenshot-scraping utilities, a GraphBuilder that constructs graphs from natural language, and integrations such as Burr and Indexify. While the README aggressively promotes a commercial hosted API at scrapegraphai.com, the repository itself is a self-contained open-source library with examples and tests. The codebase is generally well structured and modular, with clear separation between graphs, nodes, prompts, helpers, utils, and models. It shows signs of rapid growth: some typos, duplicated test fixtures, hardcoded model pricing tables, a Python version mismatch between pyproject and Dockerfile, and dependency fragility around LangChain and optional browser/OCR packages. These issues are manageable but give the codebase a medium technical drift. Overall it is an active, useful library for LLM-based extraction, and several utilities are good candidates for reuse outside the project.
Graphiti is a Python library and server framework for building temporal knowledge graphs that serve as persistent memory for AI agents. It ingests unstructured text episodes (messages, transcripts, JSON), uses LLMs to extract entities and relationships with bi-temporal validity windows, deduplicates against existing graph data, and provides hybrid search (semantic + BM25 + graph traversal) with configurable reranking. The codebase is structured around a clean driver abstraction (Neo4j, FalkorDB, Kuzu, Neptune) with per-provider operations classes, a prompt library supporting versioning and overrides, and a multi-provider LLM/embedder/cross-encoder client system. It is primarily an application library, though it includes evaluation infrastructure and research-grade deduplication algorithms (MinHash/LSH).
Activeloop's DeepLake is a production-grade, multi-vector database engine for AI data. At its core, it is a columnar, chunked, compressed storage system for tensors (multi-dimensional arrays) with native support for embeddings, images, video, text, and other complex AI data types. It exposes a Python library for dataset management, ingestion (COCO, Labelbox, CSV, Parquet), version control, and integration with PyTorch/TensorFlow/MMDetection. A high-performance C++ backend provides lazy NumPy-like array operations (nd::array), a composable async framework (async::promise), and a Tensor Query Language (TQL). The most distinctive component is a PostgreSQL extension (pg_deeplake) that allows SQL access to DeepLake datasets via custom table and index access methods, supporting vector similarity search (cosine, BM25) and hybrid search. Query execution is delegated to DuckDB through a custom scan node, enabling efficient analytical queries. The extension also includes WAL-based DDL replication across instances. This is primarily an application (a database system), with substantial research-oriented features such as medical image formats, compression codecs, and TPC-H benchmarking.
Infinity is a high-performance AI-native vector database designed for LLM applications, offering hybrid search over dense embeddings, sparse embeddings, tensors, and full-text. Its architecture is a distributed C++ core with a modular planner/executor, multiple index types (HNSW, IVF, BMP, DiskANN, EMVB, PLAID, full-text), and a transaction system with WAL and checkpointing. The database exposes a SQL interface, a Thrift RPC API, an HTTP API, and native SDKs for Python, Go, and C++. It also includes a web GUI (Next.js) for administration. The codebase is heavily optimized with SIMD instructions, custom analyzers for multiple languages, and advanced query optimization (fast rough filters, result caching). While the repository is actively maintained with a comprehensive test suite and CI/CD pipeline, there is some technical drift from deprecated KNN implementations and disabled unit tests, and the reliance on C++20 modules may introduce compiler compatibility risks.
This is a production-grade long-term memory plugin for the OpenClaw AI agent platform, built as a TypeScript/ESM module. It provides a complete memory lifecycle: LLM-powered extraction from conversation transcripts (6-category classification with L0/L1/L2 layering), hybrid vector+BM25 retrieval with cross-encoder reranking, Weibull decay-based importance scoring, tier management (core/working/peripheral), and multi-scope isolation (global, agent, user, project). The plugin integrates deeply into OpenClaw via lifecycle hooks, agent tools, CLI commands, and a dreaming engine for periodic consolidation. It stores all data in LanceDB and supports multiple embedding providers, LLM backends, and OAuth flows. The codebase is large and complex, with extensive test coverage (100+ tests) and many edge-case handlers, but exhibits significant technical debt in a single massive index.ts file and multiple fallback/legacy migration paths. It is not a research project; it is a highly functional, actively maintained plugin with a clear engineering focus on correctness, resilience, and performance.
53AIHub is an open-source enterprise AI knowledge management and agent orchestration platform. The codebase reveals a multi-tenant SaaS/self-hosted system that allows organizations to build, deploy, and operate AI agents across multiple backends (OpenAI, Coze, Dify, Gemini, etc.), manage knowledge bases with full RAG pipelines (document parsing, chunking, embedding, graph extraction, wiki generation), record and transcribe meetings with insight extraction, and provide a unified search and chat interface. The architecture is a Go monolith backend using Gin and GORM, with a monorepo frontend comprising three React apps (admin console, end-user portal, and embeddable agent plugin) sharing a common business logic and component library. Data flows center on document ingestion pipelines, real-time agent relay with tool execution, and recording/speech processing. The system integrates with vector stores (Qdrant), Elasticsearch, Redis, and various LLM providers. While primarily an application, it contains substantial data processing pipelines characteristic of a data_research project, particularly the RAG and recording insight modules.
Lumen is an open-source agent framework for chatting with data and building interactive dashboards, built on the HoloViz ecosystem (Panel, hvPlot, Bokeh). It provides a dual interface: a YAML-based declarative dashboard specification system for non-AI use cases, and an AI-powered natural language interface that generates SQL queries and Vega-Lite visualizations from user prompts. The architecture is modular, with clear separation between data sources (files, SQL databases, REST APIs, xarray), data pipelines (filters, transforms, SQL rewrites), views (hvPlot, Vega-Lite, DeckGL, tables), and an AI agent layer (Planner, Coordinator, specialized agents for SQL, visualization, validation, and source ingestion). The AI system supports multiple LLM providers (OpenAI, Anthropic, Google, Ollama, etc.), vector stores for RAG (Numpy, DuckDB, ChromaDB), and a tool system for external API integration. The codebase demonstrates high craftsmanship with reusable patterns like spec-driven component creation (MultiTypeComponent), composable SQL transforms, and a consistent agent/tool abstraction. While primarily an application framework, it includes data pipeline components that could be extracted for data engineering use. The project is actively maintained with extensive test coverage, though its large optional dependency surface and rapid feature velocity introduce some maintenance risk.
MetaGPT is a multi-agent orchestration framework designed to automate software engineering workflows and complex task execution. It functions as a Python library and CLI tool where autonomous agents (Roles like Product Manager, Architect, Engineer) collaborate via a message-passing environment to generate code, designs, and documentation. The system abstracts LLM interactions, supports Retrieval-Augmented Generation (RAG) for knowledge retrieval, and includes specialized environments for simulation (Minecraft, Stanford Town) and automation (Android, Web Browsing). It is primarily an application framework rather than a data research repository, though it incorporates research-grade components like Tree of Thought planning and AutoML evaluation. The architecture is modular, separating Roles, Actions, Tools, and Environments to allow for extensibility. Key entry points include the CLI command `metagpt` and the `metagpt.software_company` module for project generation. The codebase relies heavily on external dependencies, particularly LLM APIs (OpenAI, Azure, etc.) and browser automation tools (Playwright), which introduces maintenance complexity. While the core framework is well-structured with factory patterns and serialization support, the heavy coupling to external services and the presence of complex, niche extensions (e.g., Minecraft bot integration) contribute to a medium technical drift rating.
AnythingLLM is a full-stack, self-hosted AI chat platform designed for Retrieval-Augmented Generation (RAG) over private documents. It provides a web interface where users create workspaces, upload documents (PDF, DOCX, images with OCR, audio, etc.), which are then chunked, embedded, and stored in a vector database (default LanceDB, with support for PGVector, Pinecone, Qdrant, and others). Chat queries retrieve relevant document chunks and feed them to an LLM of choice (supporting 30+ providers including OpenAI, Anthropic, Ollama, and local models via KoboldCPP). The platform also features an extensible agent system (aibitat) with plugins for file system operations, SQL databases, Gmail, Google Calendar, web scraping, image generation, and scheduled jobs. A separate 'collector' microservice handles document ingestion and format conversion, while the 'open-computer' subproject provides a QEMU-based virtual machine for autonomous browser automation. The frontend is a React SPA with multi-user support, role-based access, internationalization, embeddable chat widgets, and a community hub for sharing agent skills, system prompts, and slash commands. The architecture is a monolith (Express server) with the collector as an isolated service, and the open-computer as a standalone CLI/service. It relies on external AI APIs, vector databases, and local storage, making it suitable for both cloud and on-premises deployment.
This repository is a collection of deployable templates and educational cookbooks for building real-time LLM applications using the Pathway framework. It provides ready-to-deploy RAG (Retrieval-Augmented Generation) solutions with live data synchronization from various sources (files, Google Drive, SharePoint, etc.), built-in vector and hybrid search, and integration with LangGraph for agentic workflows. The templates are production-oriented with Docker support, while the cookbooks demonstrate advanced patterns like Self-RAG with self-reflection and query rewriting. The codebase is well-structured but exhibits some duplication across templates, and the core logic relies heavily on the external Pathway library.
GraphRAG is a Microsoft Research project that implements a graph-based retrieval-augmented generation (RAG) system. It extracts structured knowledge (entities, relationships, claims) from unstructured text using LLMs, builds a hierarchical knowledge graph via Leiden community detection, generates community summaries, and supports multiple query strategies: local search (entity-centric), global search (map-reduce over community reports), DRIFT search (iterative multi-hop), and basic vector search. The system is designed as a Python library with a CLI (Typer) and a public API, plus a Streamlit demo app for comparing search modes. It provides extensive configuration via YAML/JSON with environment variable substitution, and supports multiple backends for storage (file, Azure Blob, CosmosDB), caching (SQLite, JSON, memory), vector stores (LanceDB, Azure AI Search, CosmosDB), and LLM providers (OpenAI, Azure OpenAI, LiteLLM). The codebase is organized as a monorepo with eight sub-packages (graphrag, graphrag-llm, graphrag-storage, graphrag-vectors, graphrag-cache, graphrag-input, graphrag-chunking, graphrag-common), each with its own pyproject.toml and versioning. The project is explicitly in maintenance mode, accepting only bug fixes and dependency updates, with no new features planned. Despite this, the architecture is well-structured with clean abstractions, factory patterns, middleware pipelines, and a composable design that makes many components reusable in other projects.
Tabby is a self-hosted AI coding assistant that provides code completion, chat, and code search capabilities through multiple interfaces: a web dashboard (Next.js/React), IDE plugins (VS Code, IntelliJ, Eclipse, Vim), and a Language Server Protocol (LSP) agent. The backend is primarily written in Rust, using Axum for HTTP services, Tantivy for indexing, and llama.cpp for local model inference. It supports enterprise features like SSO (OAuth, LDAP), team management, access policies, and background job scheduling for repository indexing, web crawling, and third-party integration (GitHub/GitLab). The system is designed to be self-contained, running without external cloud dependencies, and can operate on consumer-grade GPUs. The architecture follows a monorepo pattern with Rust crates for core services (inference, indexing, git operations, download) and TypeScript/JavaScript for the agent and UI. The web dashboard uses GraphQL (Juniper) for API communication, while IDE plugins communicate via LSP. Data flows include: code completion requests from editors → agent → backend model server → response with postprocessing; chat queries → retrieval-augmented generation (RAG) pipeline using indexed code and documentation; and background jobs for syncing repositories, crawling web docs, and indexing commits/issues/PRs. External dependencies include SQLite (via SQLx), llama.cpp, tree-sitter, and various cloud provider APIs for OAuth. The codebase demonstrates strong engineering practices with modular design, extensive testing (unit tests, golden tests), and reusable patterns such as a background job abstraction, config merging system, and a postprocessing pipeline for code completions. However, the polyglot nature (Rust, TypeScript, Java, Kotlin) and multiple client implementations introduce maintenance complexity and potential drift. The repository is actively maintained with frequent releases and a large community (33k+ stars).
Vanna is an open-source Python framework that generates SQL queries from natural language using retrieval-augmented generation (RAG). It provides a modular agent architecture with a core `Agent` class that orchestrates LLM interactions, tool execution, and rich UI component streaming. The system supports multiple LLM providers (OpenAI, Anthropic, Gemini, Ollama, etc.) and vector store backends (ChromaDB, FAISS, Qdrant, Pinecone, etc.) for storing and retrieving training data (question-SQL pairs, DDL, documentation). It includes a comprehensive evaluation framework for comparing agent variants, a legacy Flask API for backward compatibility, and a modern Lit-based web component frontend that renders rich components (tables, charts, cards, progress bars) via SSE streaming. The architecture is highly extensible with dependency injection for lifecycle hooks, LLM middlewares, error recovery strategies, context enrichers, conversation filters, and observability providers. Key entry points include the `Agent.send_message()` async generator, the `ToolRegistry` for tool management, and server factories for Flask and FastAPI. The system relies on external dependencies for LLM APIs, vector databases, and SQL databases, all configured as optional extras. The codebase demonstrates strong engineering practices with clean abstractions, though it carries significant legacy code from a previous version and some placeholder implementations (e.g., MD5-based embeddings in Qdrant).
VeritasGraph is a comprehensive, governed GraphRAG and agent framework for building enterprise-grade knowledge graphs with verifiable attribution. The repository contains a core Python library (VeritasReason) for knowledge graph construction, semantic extraction, reasoning, provenance tracking, and export, along with multiple domain-specific applications: a HIPAA-safe clinical knowledge graph pipeline (clinical-kg), a floorplan extraction and validation web app (floorplan-kg), a municipality incident reporting chatbot, and a Model Context Protocol (MCP) server for IDE integration. The architecture is modular, with a plugin-based method registry pattern, unified configuration management, and a pipeline orchestration engine. While the primary character is an application framework, it also includes research-oriented components such as clinical benchmarks (MedKG-Signal) and evaluation suites, making it a hybrid project with a strong application focus.
Kotaemon is a production-grade Retrieval-Augmented Generation (RAG) application for chatting with documents. It provides a full-stack Gradio web UI with user management, document indexing (PDF, DOCX, HTML, images, web URLs), hybrid retrieval (vector + full-text), reranking, and multi-step reasoning pipelines (simple QA, question decomposition, ReWOO agent). The architecture is a monolith split into two Python libraries: `kotaemon` (core components: embeddings, LLMs, indices, storages, loaders, agents) and `ktem` (application layer: UI pages, reasoning pipelines, database models, settings). It supports multiple LLM providers (OpenAI, Azure, Anthropic, Gemini, Cohere, Ollama, LlamaCpp), embedding models, vector stores (Chroma, LanceDB, Milvus, Qdrant), and document stores. The system is highly configurable via environment variables and a settings UI. While primarily an application, it includes data/research elements such as document loaders, OCR pipelines, graph-based indexing (GraphRAG, LightRAG, NanoGraphRAG), and a training dataset for question rephrasing.