$ show chaitin/PandaWiki
PandaWiki is a production-ready, self-hosted AI knowledge base and wiki platform. The actual codebase implements document management with versioned releases, RAG-based Q&A and search, AI-assisted document summarization and editing, multi-source content ingestion (URL, RSS, Sitemap, Notion, Feishu, DingTalk, Confluence, EPUB, Wiki.js, Yuque, Siyuan, etc.), a customizable public landing page builder, analytics, comments, contribution review, and multi-channel bot integrations for WeChat, WeCom, DingTalk, Feishu/Lark, and Discord. It is not a data research repository, but it contains substantial document-ingestion and analytics pipelines that feed the RAG system. The architecture is a full-stack monolith-style application split across a Go backend (Echo + GORM + PostgreSQL + Redis + MinIO/S3 + NATS JetStream), a React admin panel (Vite/MUI/Redux), a Next.js public site, and a pnpm workspace of shared UI/icon/theme packages. The Go backend follows a layered design with handlers, use cases, repositories, middleware, and dependency injection via generated Wire code. A standalone MQ consumer process handles async RAG document updates, crawler exports, and cron-driven statistics aggregation. The public app is served separately from the admin panel, with an embeddable widget and an H5 chat interface. In production, documents are imported through crawler tasks or direct upload, converted to HTML/Markdown, chunked, and upserted into a configurable vector/RAG service. User chat requests rewrite the query, retrieve ranked chunks, and stream LLM responses via SSE with citations. Statistics are rolled up hourly into PostgreSQL. Authentication supports local accounts, JWT sessions, GitHub OAuth, LDAP, CAS, and enterprise SSO. Feature availability is gated by commercial license editions, with a private pro submodule adding enterprise functionality.
The LLM identified 6 files worth highlighting for reuse:
Deterministic finite automaton based sensitive-word matching and replacement. Self-contained trie/state-machine implementation suitable for content moderation in any text pipeline.
Validates URLs and resolved IPs to reject private/reserved ranges, enforce scheme allowlists, and block embedded credentials. Easily extracted for any service that fetches remote resources.
Parses RSS, Atom, and JSON Feed formats into a common model. Useful for ingestion systems that need to normalize syndicated content from multiple feed providers.
HTML-to-Markdown transformer with custom renderers for attachments, task lists, and flowchart-to-Mermaid conversion. Demonstrated in document ingestion before chunking and vectorization; portable to other RAG pipelines.
React hook for chat-style auto-scroll that stops when the user scrolls up and resumes on user intent. Uses ResizeObserver and refs, with no app-specific dependencies.
Configurable rate limiter used to protect auth and public endpoints. Self-contained package that can be reused as middleware or as a standalone limiter in other Go services.