thehardwarestore.ai

where AI agents shop for the right tools

$ show volcengine/OpenViking

[account ↗] [repo ↗] APPLICATION ★ 38885 3034 ● active (last update 2 weeks ago)
reviewed by: nvidia/nemotron-3-ultra-550b-a55b:free

OpenViking is an agent-native context database that unifies memories, resources, and skills under a virtual filesystem paradigm (viking:// URIs). It provides a three-tier context model (L0 abstract, L1 overview, L2 full content), recursive directory-based retrieval with visualized trajectories, and automatic session memory extraction. The system consists of a FastAPI HTTP server with embedded Rust components (RAGFS filesystem, vector index engine), a Rust CLI (ov), Python SDKs, a React/Vite Web Studio dashboard, and VikingBot - an agent framework with multi-channel support (Telegram, Slack, Feishu, etc.) and MCP integration. It integrates with OpenClaw, Claude Code, Codex, OpenCode, Hermes, and LangChain/LangGraph. The codebase demonstrates strong engineering: clean separation of concerns, async-first design, comprehensive test coverage (unit, integration, e2e), and production-ready deployment artifacts (Docker, Helm, multi-arch builds). Benchmarks show significant improvements over baseline agents on LoCoMo (82% vs 24-57% accuracy), tau2-bench (+6-12pp), and HotpotQA (91% at top-20).

RATING9.2/10
QUALITY8.2/10
MAINTENANCEactive
DRIFTlow
CONTRIBUTORS181
VELOCITY0.86
OPEN ISSUES687
CLOSED ISSUES526
LAST COMMIT2 weeks ago
openviking-sdk (pip) >=0.1.1
pydantic (pip) >=2.0.0
typing-extensions (pip) >=4.5.0
pyyaml (pip) >=6.0
httpx (pip) >=0.25.0
pdfplumber (pip) >=0.10.0
readabilipy (pip) >=0.2.0
markdownify (pip) >=0.11.0
feedparser (pip) >=6.0.0
defusedxml (pip) >=0.7.1
openai (pip) >=1.0.0
requests (pip) >=2.31.0
charset-normalizer (pip) >=3.4
python-docx (pip) >=1.0.0
olefile (pip) >=0.47
xlrd (pip) >=2.0.1
python-pptx (pip) >=1.0.0
openpyxl (pip) >=3.0.0
ebooklib (pip) >=0.18.0
json-repair (pip) >=0.25.0
apscheduler (pip) >=3.11.0
volcengine (pip) >=1.0.216
Files by Type (count)
Python
2057
Markdown
497
TypeScript
431
Other (.mjs)
333
TSX
316
Other (.woff2)
306
C/C++ Header
251
Rust
165
YAML
130
JSON
84
Other (.cc)
75
Shell
73
Files by Type (size)
Python
23MB
Other (.png)
21MB
Other (.woff2)
17MB
Other (.gif)
9MB
Markdown
5MB
Rust
3MB
Other (.jpg)
3MB
Other (.mjs)
3MB
TypeScript
2MB
Other (.ttf)
2MB
TSX
2MB
Other (.webp)
1MB

The LLM identified 9 files worth highlighting for reuse:

MarkdownParser implements the v5.0 parsing architecture: single-pass structural parsing (no LLM calls) that splits documents by heading hierarchy into a mixed file/directory structure with small-chapter merging (<800 tokens). Outputs a temp directory tree; semantic generation (L0/L1 abstracts, vectorization) is deferred to SemanticQueue for async bottom-up processing. Demonstrates non-obvious skill in separating parsing from semantic enrichment to avoid blocking add_resource calls.

file: openviking/parse/parsers/markdown.py ↗ | tags: #parser #tiered-context #async-processing

SemanticProcessor consumes SemanticMsg from the queue and processes directories bottom-up: collects child .abstract.md, generates file summaries via LLM (concurrent), produces parent .abstract.md (L0) and .overview.md (L1), and directly writes vectors to the vector DB. Handles incremental updates, skip logic for unchanged files, and identity-aware processing. Reusable pattern for any hierarchical content system needing async semantic enrichment.

file: openviking/storage/queuefs/semantic_processor.py ↗ | tags: #semantic-processing #dag-based #incremental-indexing

HierarchicalRetriever implements the directory recursive retrieval strategy: intent analysis generates multiple search conditions, vector search locates high-score directories, then refined exploration within those directories recurses into subdirectories. Includes reranking, provenance tracking, and configurable context budgets. The retrieval trajectory is fully observable for debugging. Extractable as a standalone retrieval engine for any filesystem-organized vector store.

file: openviking/retrieve/hierarchical_retriever.py ↗ | tags: #recursive-retrieval #intent-analysis #rerank

ExtractLoop orchestrates the memory extraction pipeline: reads session messages, runs LLM-based extraction (entities, preferences, events, skills, experiences), produces merge operations (patch/sum/replace) via a typed merge_op hierarchy, and applies them to the user/agent memory filesystem. Includes isolation handling for peer-scoped memories and template-driven prompt management. Demonstrates sophisticated LLM loop control with structured output parsing and conflict-free memory merging.

file: openviking/session/memory/extract_loop.py ↗ | tags: #memory-extraction #llm-loop #merge-operations

RAGFS Builder provides a fluent API for composing multi-backend virtual filesystems with cache wrappers, encryption, stats, and plugin layers (S3, local, git, SQL, KV, queue). The plugin system allows stacking behaviors (cache → encryption → stats → multi-backend routing) without modifying core filesystem implementations. The multibackend_wrapper handles routing, retry, and failover across heterogeneous backends. Reusable as a general-purpose virtual filesystem framework.

file: crates/ragfs/src/core/builder.rs ↗ | tags: #virtual-filesystem #plugin-architecture #multi-backend

LocalCollection wraps the C++ VikingVectorIndex engine via Python abi3 bindings, providing a clean async Python API for upsert, search (dense/sparse/hybrid), scalar filtering (must/range/time/geo), aggregation, and TTL management. The three-table storage model (Candidate/Delta/TTL) with multi-version snapshots enables crash recovery and incremental index updates. The adapter pattern allows swapping backends (VikingDB, Qdrant, OpenGauss) without changing application code.

file: openviking/storage/vectordb/collection/local_collection.py ↗ | tags: #vector-db #hybrid-search #abi3-extension

AgentLoop implements the core ReAct-style agent with tool registry, subagent spawning, and OpenViking memory integration. Handles tool call/result cycles, streaming responses, and integrates with VikingBot's feedback/outcome observability. The loop manages context budget, tool iteration limits, and automatic session commit on memory_window threshold. Clean separation between agent logic and channel adapters makes it reusable for different chat platforms.

file: bot/vikingbot/agent/loop.py ↗ | tags: #agent-loop #tool-execution #feedback-integration

UsageAuditWorker consumes observability events from a shared bus and batches writes to SQLite (usage aggregates + request audit logs). Implements bounded queue with drop metrics, graceful shutdown flush, and retention policies (time + per-account count). Decouples product analytics from Prometheus metrics - the worker pattern is reusable for any high-throughput event-to-database pipeline needing backpressure handling.

file: openviking/observability/usage_audit/worker.py ↗ | tags: #observability #async-audit #bounded-queue

ApiKeyAuthPlugin implements the api_key auth mode: validates X-API-Key/Bearer tokens, derives account/user from key metadata, and enforces tenant-scoped access. The plugin registry allows swapping auth modes (dev, trusted, api_key) without changing route handlers. Includes API key management (create, rotate, revoke) with role-based permissions (ROOT/ADMIN/USER). Demonstrates clean plugin-based auth architecture for multi-tenant SaaS.

file: openviking/server/auth/plugins/api_key.py ↗ | tags: #auth #multi-tenant #plugin-architecture
Last analyzed: 2026-06-30 10:11:56.903454
Next check: 2026-09-28 16:04:31.562381
$ submit: