$ show ohmeow/blurr
BLURR is a library that integrates Hugging Face transformers with fastai for NLP tasks. It provides high-level, mid-level, and low-level APIs to train and deploy transformer models for text classification, token classification, question answering, summarization, translation, and language modeling. The library leverages fastai's data block API and learner system, wrapping Hugging Face models to work seamlessly within the fastai training loop. Key components include custom data blocks for text processing, model wrappers for Hugging Face integration, specialized loss functions, and callbacks for optimization. The repository includes extensive examples and notebooks demonstrating usage with standard datasets like GLUE, SQuAD, and CNN/DailyMail.
The LLM identified 4 files worth highlighting for reuse:
Provides TextBlock, BatchTokenizeTransform, and BatchDecodeTransform for handling text data in fastai's data block API. This component tokenizes and decodes text batches using Hugging Face tokenizers, enabling seamless integration of transformer models with fastai's data pipeline. It is self-contained, reusable in any fastai text processing task, and demonstrates effective use of fastai's transform system for NLP.
BaseModelWrapper and BaseModelCallback work together to integrate Hugging Face models with fastai Learners. The wrapper adapts the model's forward pass, while the callback handles loss extraction from model outputs. This pattern allows any Hugging Face model to be used in fastai's training loop with minimal boilerplate, showcasing a clean abstraction for external model integration.
PreCalculatedCrossEntropyLoss and related loss functions extract loss from Hugging Face model outputs that compute loss internally. This enables the fastai training loop to use the model's native loss calculation, avoiding redundant computation. The component is a focused, reusable solution for a common integration challenge between Hugging Face models and fastai's loss API.
Implements gradient checkpointing to reduce GPU memory usage during training by trading compute for memory. This callback can be added to any fastai Learner to train larger models or batches on limited hardware. It is a self-contained optimization technique with broad applicability beyond NLP, demonstrating effective resource management in deep learning training loops.