LLM Foundry is a production-grade framework for training, finetuning, evaluating, and serving large language models, with a focus on the MPT architecture and HuggingFace integration. It provides a full pipeline from data preparation (MDS streaming), to model definition (custom MPT, HF wrappers, embedding models), to distributed training (Composer, FSDP, TP), and to evaluation (ICL benchmarks, gauntlets). The architecture is modular, using YAML-driven configuration and a central registry pattern for extensibility. The codebase includes dozens of callbacks for monitoring and training stabilization, custom optimizers, and efficient data collation (bin packing). While tightly integrated with MosaicML's Composer and platform, the core registry and layer abstractions are reusable. The repository is actively maintained with comprehensive test suites, though some features remain experimental and several dependencies (flash attention, megablocks) require careful version management.
This repository implements and evaluates Function Vectors (FVs) for steering large language models (LLMs) without fine-tuning. The core workflow involves computing task-conditioned mean head activations from in-context learning examples, identifying causally influential attention heads via indirect effect, summing their outputs to form the FV, and applying it during inference to elicit target behaviors (e.g., translation, inflection). The system provides a modular Python library with utilities for model loading, prompt construction, activation extraction, intervention, and evaluation, accompanied by a demo notebook and evaluation scripts for running experiments on various linguistic datasets. While the codebase demonstrates clean separation of concerns and contains reusable components for model intervention and prompt handling, it shows signs of technical drift due to dependency rigidity and architecture-specific hooks, and appears to have stagnated in maintenance with no recent updates.