$ show ECNU-ICALK/AutoSkill
AutoSkill is a comprehensive framework for extracting, managing, and evolving reusable 'skills' from conversational logs and documents. It operates as a modular application with multiple entry points: a web UI for interactive chat, a CLI for batch processing, and an OpenAI-compatible proxy server that injects relevant skills into LLM context. The system employs LLM-driven and heuristic pipelines to extract skill candidates, deduplicate and merge them, and persist them in a skill store with vector and BM25 retrieval. It also includes a self-evolution subsystem (SkillEvo) that replays experiences, evaluates performance, and mutates skills for improvement. The codebase is hybrid, containing both application components (web server, proxy, libraries) and data/research pipelines (document ingestion, trajectory extraction), but the dominant character is an application SDK designed for integration with agentic workflows. Notable craftsmanship is seen in standalone utilities like a CJK-aware text unit counter, an AST-based prompt patcher, a persistent BM25 index, and a LibreOffice socket shim, though the repo also suffers from significant code duplication (e.g., identical validators across docx/pptx/xlsx) and a large surface area of environment variables and external dependencies.
The LLM identified 7 files worth highlighting for reuse:
Provides CJK-aware text unit counting (characters for CJK, words for others) and truncation utilities. This is a self-contained module with no internal dependencies, usable in any text processing pipeline that needs mixed-language tokenization.
Thread-safe persistent BM25 inverted index with atomic file writes, upsert, delete, search, and validation. Implements the VectorIndex interface but with classic BM25 scoring. Can be extracted and used as a standalone local text search library.
A LibreOffice shim that detects blocked AF_UNIX sockets (common in containerized or sandboxed environments) and injects an LD_PRELOAD C shim compiled on-the-fly via gcc to bypass the restriction. Demonstrates a novel, low-level workaround for a specific platform issue. Reusable as a standalone library for any project needing to call LibreOffice headless in hostile environments.
Implements a configurable skill taxonomy system from YAML/JSON files. Normalizes asset types, resolves family candidates, builds hierarchical asset paths, and renders prompt guidance for document extraction. Self-contained class with file loading, validation, and query methods; can be extracted for any project needing configurable hierarchical tagging.
Uses Python's AST to perform in-place replacement of a function body in `prompts.py`. This is a careful, non-obvious pattern for programmatic prompt updates without resorting to string templates or config files. It handles function signature preservation, indentation, and multiple definitions. Reusable for projects that need to dynamically modify source code within the same process.
Standardized utility that builds LLM and embedding provider configuration dictionaries from environment variables. Handles multiple providers (OpenAI, Anthropic, GLM, etc.) with secure credential defaults and provider-specific key mappings. Though duplicated in AutoSkill4Doc, the pattern is self-contained and could be published as a small library for multi-provider LLM config management.
A self-contained local web server for interactive chat with AutoSkill, built entirely on Python stdlib (http.server, threading, json). Manages multiple sessions with disk persistence, streaming responses, and a REST API for skill management. Demonstrates a reusable pattern for building lightweight development UIs without external dependencies. The _SessionManager class and helper functions (json response, file serving, boolean coercion) are extractable.