$ show roboflow/rf-detr
RF-DETR is a production-grade computer vision library by Roboflow for real-time object detection, instance segmentation, and keypoint estimation. It implements a DETR-family architecture (LW-DETR, Deformable DETR) with a DINOv2 backbone, providing a complete training-to-deployment pipeline. The library is published as the `rfdetr` PyPI package and serves as the open-source core of Roboflow's detection platform, with larger models available under a separate license. The system is structured as a Python library with a CLI entry point (`rfdetr`). The main model class `RFDETR` (and variants like `RFDETRNano`) provides `predict()`, `train()`, `export()`, and `evaluate()` methods. Training is built on PyTorch Lightning with custom `LightningModule` and `LightningDataModule`, plus callbacks for EMA, COCO evaluation, GPU memory monitoring, and drop-path scheduling. The export subsystem uses a registry pattern with format-specific exporters (ONNX, TensorRT, TFLite, CoreML, OpenVINO, ExecuTorch, LiteRT) that share preprocessing and decoding utilities. Dataset loaders support COCO, YOLO, WebDataset, and Roboflow formats with configurable augmentation backends (torchvision, Albumentations, Kornia). The primary data flow: input images → preprocessing (resize, normalize) → DINOv2 backbone → multi-scale feature projector → deformable attention transformer encoder-decoder → task-specific heads (detection, segmentation, keypoints) → post-processing (top-k selection, NMS-free decoding). Training uses COCO-style datasets with Hungarian matching for label assignment. External dependencies include PyTorch, PyTorch Lightning, HuggingFace Transformers, and optional backends for export. The library integrates with Roboflow's platform for dataset management and deployment. The codebase is actively maintained with extensive test coverage (pytest, CI workflows for CPU/GPU/XLA), backward compatibility testing for legacy checkpoints, and a formal changelog. It demonstrates high craftsmanship in areas like safe checkpoint loading, atomic file operations, custom fused optimizers (Triton), and export robustness. The large optional dependency surface and reliance on some private PyTorch APIs introduce maintenance risk, but overall it is a well-engineered library suitable for production use.
The LLM identified 5 files worth highlighting for reuse:
Implements a formal registry for export formats using dataclasses and lazy imports. Each format registers an `ExporterEntry` with metadata (name, dependencies, capabilities). The `resolve_exporter` function dynamically imports and instantiates the correct exporter based on format string. This pattern is cleanly decoupled, easily extensible, and could be extracted for any plugin-based system.
Provides `_safe_torch_load` with a three-stage fallback: first tries `weights_only=True` (PyTorch 2.6+), then `safe_globals` for allowed classes, and finally an opt-in `trust=True` path with explicit warning. This pattern is critical for secure checkpoint loading and is reusable in any PyTorch project that loads serialized models.
Implements a custom fused optimizer that combines global gradient clipping, AdamW, and EMA updates into a single Triton kernel for BF16 training. Includes a routing mechanism to fall back to standard PyTorch when the kernel is unavailable. This demonstrates advanced performance engineering and is a reusable pattern for fusing multiple training operations.
Provides `decode_image` and `decode_image_bytes` with a decoder fallback chain: `simplejpeg` for fast JPEG decoding, then Pillow with draft mode, and finally standard Pillow. Includes decompression bomb protection and optional color conversion. This is a self-contained utility that can be extracted for any project needing fast, safe image loading.
Implements GroupPose-style keypoint prediction with custom loss functions (`compute_l1_keypoint_loss`, `compute_keypoint_matching_cost`) that include per-keypoint masking, non-finite input sanitization, and Cholesky-based Gaussian NLL. The code is well-documented with numerical stability considerations and is reusable for any keypoint detection task.