$ show chubbyguan/chubbyskills
chubbyskills is a personal knowledge management CLI toolkit that ingests content from over a dozen platforms (YouTube, Bilibili, X, Xiaohongshu, WeChat, Douyin, Podcasts, etc.), transcribes audio/video locally or via cloud providers, and outputs structured Markdown notes with YAML frontmatter (schema v1). The system is architected as a collection of modular "skills" (one per platform) orchestrated by a unified CLI (`tools/chubby.py`), with shared utilities in `chubby_common/`. It includes an incremental knowledge base indexer using SQLite FTS5 and optional semantic embeddings, an MCP server exposing six tools for AI agent integration, and a deterministic evidence brief generator with integrity validation. Operation is local-first: transcription uses SenseVoice and faster-whisper, video downloading uses yt-dlp, and all processing is designed to work offline. Cloud transcription (Atlas/MuAPI) and LLM enrichment (DeepSeek) are optional extensions. The repository also provides a skill installer, platform smoke testing, output validation, and retry mechanisms with credential redaction. Despite the breadth, the codebase maintains clean separation of concerns, with the shared library standing out as a reusable zero-dependency core.
The LLM identified 5 files worth highlighting for reuse:
The core shared module used by all skills. Provides config loading (YAML), environment checks, yt-dlp wrapper with platform-specific UA/referer, SenseVoice transcription wrapper, Markdown frontmatter generation, VTT subtitle parsing, and LLM JSON fallback. Designed to be imported without heavy dependencies (funasr is lazy-imported). This is the most reusable component—it could be extracted into a standalone library for any content-to-markdown pipeline.
Exports structured evidence briefs (topic passages) from the knowledge base. Builds excerpts using line ranges, validates SHA-256 hashes against original files, and rejects path traversal or symlink attacks. Ensures export integrity by re-validating hashes at export time. The pattern of deterministic, verifiable excerpt extraction is reusable for any document-based research workflow.
SQLite-based indexer with FTS5 full-text search and optional semantic embeddings (OpenAI or local lite). Implements incremental syncing: only re-indexes changed notes, preserves embeddings for unchanged content, uses atomic commits and rollback on failure. The design patterns for incremental indexing and embedding caching are reusable for any local-first knowledge base.
Abstraction layer for cloud transcription providers (Atlas, MuAPI). Manages job submission, status polling, credential redaction, and state locking to prevent concurrent write conflicts. Jobs persist to disk for recovery from crashes. The provider interface pattern and resilient job management are reusable for any cloud API integration with long-running tasks.
Installs skills as self-contained, portable directories with bundled dependencies and an installation manifest (`installation.json`). Supports `--all` flag, refuses to overwrite existing skills, and generates minimal virtual environment scaffolds. The pattern of creating isolated, relocatable tool kits from a monorepo is reusable for distributing modular CLI tools.