$ show KnowledgeXLab/LeanRAG
LeanRAG is a Retrieval-Augmented Generation (RAG) framework that leverages knowledge graph structures with semantic aggregation and hierarchical retrieval to generate context-aware, concise, and high-fidelity responses. The system processes documents through a pipeline: chunking, LLM-based triple extraction, hierarchical graph construction with clustering, and structure-guided retrieval from fine-grained entities upward through the graph to reduce redundancy and improve evidence quality. It targets question-answering tasks across domains (general, CS, legal, agriculture) and uses LLMs (OpenAI, Qwen, DeepSeek) for extraction and generation, with Milvus for vector search and MySQL for storing graph entities, relations, and communities. Evaluation is performed via an external LLM API scoring answers on comprehensiveness, empowerment, diversity, and overall quality.
The LLM identified 3 files worth highlighting for reuse:
Implements a load-balanced LLM API client with automatic failover, retry logic, and token usage tracking. Manages multiple API keys to distribute requests and handle rate limits, demonstrating non-obvious skill in resilient API management. Self-contained with minimal dependencies (standard library only), making it easily extractable for any project requiring robust LLM API interaction.
Provides Gaussian Mixture Model-based multi-level clustering for knowledge graph summarization. Implements hierarchical entity clustering to create community summaries at different granularity levels, reducing redundancy in retrieval. The module is isolated (depends only on numpy and scikit-learn) and reusable for any clustering task requiring hierarchical summarization of vector embeddings.
Abstracts Milvus vector search and MySQL CRUD operations for storing and querying hierarchical knowledge graph data. Encapsulates complex database interactions into simple methods for entity/relation/community management and vector similarity search. Demonstrates skill in designing a clean persistence layer that could be adapted to other graph-based applications with minimal changes.