$ show microsoft/graphrag
GraphRAG is a Microsoft Research project that implements a graph-based retrieval-augmented generation (RAG) system. It extracts structured knowledge (entities, relationships, claims) from unstructured text using LLMs, builds a hierarchical knowledge graph via Leiden community detection, generates community summaries, and supports multiple query strategies: local search (entity-centric), global search (map-reduce over community reports), DRIFT search (iterative multi-hop), and basic vector search. The system is designed as a Python library with a CLI (Typer) and a public API, plus a Streamlit demo app for comparing search modes. It provides extensive configuration via YAML/JSON with environment variable substitution, and supports multiple backends for storage (file, Azure Blob, CosmosDB), caching (SQLite, JSON, memory), vector stores (LanceDB, Azure AI Search, CosmosDB), and LLM providers (OpenAI, Azure OpenAI, LiteLLM). The codebase is organized as a monorepo with eight sub-packages (graphrag, graphrag-llm, graphrag-storage, graphrag-vectors, graphrag-cache, graphrag-input, graphrag-chunking, graphrag-common), each with its own pyproject.toml and versioning. The project is explicitly in maintenance mode, accepting only bug fixes and dependency updates, with no new features planned. Despite this, the architecture is well-structured with clean abstractions, factory patterns, middleware pipelines, and a composable design that makes many components reusable in other projects.
The LLM identified 4 files worth highlighting for reuse:
Implements a composable middleware pipeline for LLM completion and embedding functions. Middleware components (caching, retries, rate limiting, metrics, logging, error injection) are wrapped around the core LLM call in a configurable order. Each middleware is a function that takes the next callable and returns an async wrapper. This pattern allows clean separation of cross-cutting concerns and is easily extensible with custom middleware. The pipeline is built via a simple list of middleware functions, making it reusable for any async function that follows the LLMCompletion/LLMEmbedding protocol.
Provides a fluent filter expression DSL for vector store metadata queries. Defines a base `Expression` class with `AndExpr`, `OrExpr`, `NotExpr`, and `Condition` subclasses, plus a `F` builder for constructing conditions (e.g., `F('field') == 'value'`). Supports evaluation against dictionaries, JSON serialization/deserialization, and compilation to backend-specific query languages (LanceDB SQL, OData). The design is generic and can be reused in any project requiring composable, backend-agnostic filtering for search or storage systems.
A generic Factory base class that supports registration of strategies by type key and creation with singleton or transient scoping. Uses a class-level registry (`_strategies`) and a `register` classmethod for extensibility. The `create` method instantiates the registered strategy with provided config, optionally caching singletons. This pattern is used consistently across all sub-packages (storage, cache, vector store, LLM, input reader, etc.) and can be extracted as a standalone utility for any project needing a lightweight, type-safe factory with registration.
Defines an abstract `Table` class for row-by-row streaming I/O, with async iteration (`__aiter__`), write methods (`upsert`, `delete`), and a `child` method for namespace isolation. This abstraction allows workflows to process large datasets without loading everything into memory, and supports multiple backends (Parquet, CSV, CosmosDB) via the `TableProvider` interface. The design is clean and reusable for any data pipeline that needs incremental, streaming table operations with pluggable storage.