$ show stanfordnlp/dspy
DSPy is a Python framework for programming language models (LMs) rather than prompting them. It provides a declarative API for defining LM programs using typed signatures, predictors (e.g., Predict, ChainOfThought, ReAct, RLM), and adapters (Chat, JSON, XML) that format and parse LM interactions. The framework includes a rich optimization suite (teleprompters) that can bootstrap demonstrations, optimize instructions (MIPROv2, COPRO), fine-tune models, and perform code-level optimization (GEPA) for Flex modules. It supports multiple LM backends (LiteLLM, native lm15) with caching, streaming, error handling, and cost tracking. The architecture is modular: clients manage LM connections, engines abstract execution backends, adapters handle message formatting/parsing, predictors implement reasoning patterns, signatures define I/O contracts, and utilities provide caching, callbacks, parallelism, and other cross-cutting concerns. The codebase demonstrates high cohesion and separation of concerns, with extensive test coverage and documentation. However, it is a large project with some experimental features, deprecation warnings, and a vendored lm15 library that adds maintenance overhead. Overall, it is a production-ready research framework widely used in the NLP community for building and optimizing LM programs.
The LLM identified 4 files worth highlighting for reuse:
Implements a two-tier (memory + disk) cache with restricted pickle deserialization for safety. Provides a `request_cache` decorator that wraps functions (e.g., LM calls) and caches results based on a hash of arguments, ignoring sensitive keys. The cache supports registration of safe types for serialization and graceful fallback to memory-only on disk failures. This component is self-contained and could be extracted as a standalone caching utility for any Python application that needs persistent, secure caching of function outputs.
Provides `ParallelExecutor` for multi-threaded execution of functions over iterables with configurable parallelism, progress reporting, and error accumulation. Supports timeout per task and straggler limits. The pattern is cleanly abstracted with a sequential fallback and is used in evaluation and optimization workflows. This component is reusable for any batch processing task that requires concurrent execution with robust error handling and progress tracking.
Implements a pure-function HTTP/1.1 codec for request serialization and response parsing. Includes chunked transfer encoding, Content-Encoding decompression (gzip, deflate), and connection management primitives. The module is entirely self-contained with no external dependencies beyond Python standard library. Demonstrates deep understanding of HTTP protocol internals and could be reused in any project needing low-level HTTP/1.1 message handling without a full HTTP client library.
Implements a generic callback system with a `BaseCallback` class defining lifecycle hooks (e.g., on_lm_start, on_lm_end, on_module_start, on_evaluate_start) and a `with_callbacks` decorator that wraps instance methods to fire callbacks. Uses context variables to propagate a unique call ID and support nested callbacks. This pattern is cleanly separated and usable in any framework that needs to add instrumentation, logging, or tracing to method calls without modifying the core logic.