Ultralytics is a production-grade computer vision library built around the YOLO model family. In production it provides a unified Python/CLI API for loading pretrained and custom models, running inference, training, validation, export, benchmarking, and object tracking across detection, instance and semantic segmentation, pose estimation, oriented bounding boxes, classification, and depth estimation. The architecture is a modular Python package: an engine layer (Model, BasePredictor, BaseTrainer, BaseValidator), a unified AutoBackend inference dispatcher supporting 20+ formats from PyTorch to TensorRT/ONNX/OpenVINO/CoreML/TFLite, a YAML-driven model configuration system, dataset loaders with caching and augmentation, pluggable experiment-tracking callbacks, and a family of ready-to-use solution classes for counting, heatmaps, queue management, and similar vision applications. The CLI entrypoint (`yolo`) and Python classes (`YOLO`, `SAM`, `RTDETR`, etc.) are the primary entry points, backed by active CI, Docker images, and extensive examples in Python, C++, and Rust.
This repository is <code>timm</code> (PyTorch Image Models), a high-quality library providing a vast collection of image classification model architectures (100+), pretrained weights, training utilities, and data pipelines. It is not an application but a reusable library/framework, designed for research and production use. The codebase is well-structured with clear separation of concerns: <code>timm/layers</code> for building blocks (activations, normalizations, attentions, pooling), <code>timm/models</code> for model definitions with a consistent registry pattern, <code>timm/data</code> for data loading and augmentation (including AutoAugment, RandAugment, Mixup, CutMix), <code>timm/optim</code> for 30+ optimizers, <code>timm/scheduler</code> for learning rate schedulers, and <code>timm/task</code> for training task abstractions. The library supports distributed training, mixed precision, ONNX export, Hugging Face Hub integration, and dynamic input sizes (NaFlex). It is actively maintained by Hugging Face and the community, with frequent releases and a large number of pretrained models.
vit-pytorch is a comprehensive PyTorch library implementing dozens of Vision Transformer (ViT) architectures and extensions, along with self-supervised pretraining methods (MAE, DINO, SimMIM, MP3, LeJEPA) and video transformers (ViViT, NaViT-3D, CCT-3D). It is a research-oriented codebase offering a consistent API across models, all accepting standard hyperparameters like image_size, patch_size, depth, and heads. The library covers a wide spectrum of attention mechanisms (cross-attention, shifted windows, channel attention, sparse gating), tokenization strategies (patch merging, region-based, t2t, SPT), and training utilities (recorders, extractors, distillation wrappers). While primarily a collection of model implementations, it also includes action transformers for robotics (VAT, VAAT) and N-dimensional ViTs, demonstrating both breadth and depth of recent transformer research. The code is highly modular, relying on shared building blocks such as Attention, FeedForward, and Transformer classes, with heavy but well-organized use of einops for tensor manipulation. Some variants are experimental and may lack the polish of the core models, but the overall architecture is clean and serves as a valuable reference for implementing and comparing ViT variations. The repository is actively maintained, with recent commits and a large user base, but its sheer size and variety mean that not every model is exhaustively tested—core components are covered by basic forward-pass tests, while many advanced variants rely on the underlying patterns being sound.
YOLOv5 is a production-oriented PyTorch framework for object detection, instance segmentation, and image classification, with a hybrid research/educational flavour from its tutorial notebooks. The dominant character is application: the repository provides CLI tools (`train.py`, `val.py`, `detect.py`, `export.py`, plus `segment/` and `classify/` variants), a Flask REST API, Docker images, PyTorch Hub support, CI smoke tests, benchmarking scripts, and experiment-tracking integrations for W&B, ClearML, and Comet. It is not just an experiment repo; it is designed to be deployed and exported to edge and cloud inference targets. Core architecture is config-driven: model YAMLs are parsed by `models/yolo.py` into `DetectionModel`, `SegmentationModel`, or `ClassificationModel` geometries, while `models/common.py` holds reusable layers and the central `DetectMultiBackend` inference wrapper for PyTorch, TorchScript, ONNX Runtime, OpenVINO, TensorRT, CoreML, TensorFlow, TFLite/Edge TPU, Paddle, and Triton. The primary data flow is dataset YAML -> dataloaders with augmentation (`utils/augmentations.py`) -> model forward -> task-specific loss (`utils/loss.py`, `utils/segment/loss.py`) -> train/val loops. `export.py` converts trained checkpoints into 12+ deployment formats using a decorated `@try_export` pattern, and `data/` embeds dataset download/conversion scripts for COCO, ImageNet subsets, VOC, VisDrone, and others. Maintenance is active, but the codebase sits in a legacy/support position relative to the unified `ultralytics` package. Technical drift is medium: some loggers are deprecated, many `models/hub/` configs are experimental, and the code depends heavily on the external `ultralytics` package, which creates version-compatibility risk. The code is generally clean and modular, though it has task-level duplication across detection, segmentation, and classification paths and limited unit test coverage outside smoke tests.
X-AnyLabeling is a cross-platform desktop application (PyQt6) for AI-assisted annotation of images, videos, and 3D point clouds. It serves as a unified interface for a vast model zoo including YOLO (v5-v26), SAM (1/2/3), Grounding DINO, Florence-2, PP-OCR, and many others, enabling auto-labeling for detection, segmentation, pose estimation, OCR, tracking, and more. The application features a plugin-based model architecture, abstracted inference engines (ONNX, TensorRT, PyTorch), and a comprehensive settings system. It also integrates an Ultralytics YOLO training pipeline and a secure in-app update mechanism. While primarily an application, it has a strong hybrid nature due to its deep integration of research code (vendored Grounding DINO, SAHI) and data pipeline utilities (format converters, COCO evaluators). The codebase demonstrates high craftsmanship in its model abstraction layer and inference engine design, though the sheer volume of integrated models introduces a medium risk of technical drift. It is actively maintained with a large community and extensive test coverage.
AlbumentationsX is a production-grade Python library for fast, flexible image augmentation in deep learning and computer vision pipelines. It provides a comprehensive catalog of 70+ transforms covering pixel-level (blur, noise, color, weather effects), geometric (flip, rotate, crop, resize, distortion, perspective), dropout (coarse, grid, mask, pixel), mixing (mosaic, copy-paste, overlay), and domain adaptation (FDA, histogram matching, stain normalization). The library supports 2D and 3D data, including images, masks, bounding boxes (HBB and OBB), keypoints, and volumes, with seamless integration into PyTorch via tensor transforms. Its architecture is built on a clean inheritance hierarchy (BasicTransform → DualTransform/ImageOnlyTransform/Transform3D) with Pydantic-based parameter validation, serialization to JSON/YAML, and deterministic replay via ReplayCompose. The codebase is backed by an extensive test suite (property-based, regression, contract, and golden vector tests), ASV benchmarks, and a sophisticated CI/CD system with path-aware planning, performance budgets, and legal integrity checks. Telemetry (opt-out) collects anonymous usage statistics to guide development. While primarily a software library, it incorporates research-level transforms (e.g., PlanckianJitter, HEStain, dithering) and is used in both academic and production settings.
This repository is a comprehensive survey paper on the intersection of knowledge graphs and multi-modal learning. It organizes the literature into two main directions: KG-driven multi-modal learning (KG4MM) and multi-modal knowledge graphs (MM4KG). The survey covers various tasks such as visual question answering, image classification, multi-modal named entity recognition, and knowledge graph completion, among others. Each section includes a list of relevant papers with links to their sources (e.g., arXiv, conference proceedings). The repository is maintained through updates that include adding new papers and fixing links, as evidenced by the recent commit in 2026 and the news items up to 2024-12.
NSFWJS is a client-side JavaScript library for detecting NSFW (Not Safe For Work) content in images using TensorFlow.js. It provides a simple API to load pre-trained models (MobileNetV2, MobileNetV2Mid, InceptionV3) and classify images into five categories: Drawing, Hentai, Neutral, Porn, and Sexy. The library is designed for both browser and Node.js environments, with backend selection guidance for optimal performance (WebGL, WebGPU, WASM, CPU). The core architecture consists of a lightweight `NSFWJS` class that handles model loading, inference, classification, and disposal, with support for tree-shaking via separate model imports. The repository also includes several demo applications: a full-featured React + Vite demo with Web Worker offloading, camera support, and file drop; a Node.js Express server example; and a minimal HTML/script tag demo. While primarily an application library, it is hybrid in nature as it bundles pre-trained model weights (quantized and base64-encoded) and includes model conversion scripts, making it both a software tool and a model distribution package. The library relies on TensorFlow.js as a peer dependency and uses IndexedDB for model caching in the browser.
This repository implements the EViT (Expediting Vision Transformers via Token Reorganizations) method for accelerating Vision Transformers by dynamically reducing the number of tokens processed in intermediate layers through token discarding or fusion. It provides training, evaluation, and visualization scripts for ImageNet classification, leveraging the timm library for core utilities. The codebase includes dynamic token rate scheduling via cosine annealing, throughput and MAC measurement tools, and knowledge distillation support. While the core EViT model is specialized, the repository contains reusable components such as helper functions for scheduling and efficiency metrics (helpers.py) and a knowledge distillation loss wrapper (losses.py). These components are self-contained and demonstrate non-obvious solutions to common problems in efficient deep learning. However, the codebase suffers from technical drift due to outdated dependencies (e.g., torch 1.9.0, timm 0.4.12) and appears inactive based on the commit history.
This repository contains the reference implementation of AugMix, a data augmentation technique for improving robustness and uncertainty calibration in image classification models. It provides training scripts for CIFAR-10/100 and ImageNet using PyTorch, along with evaluation on their corrupted versions (CIFAR-C, ImageNet-C). The core augmentation logic is implemented in NumPy and PIL, and the training code applies a Jensen-Shannon Divergence consistency loss between augmented and clean images. The codebase is research-oriented, with scripts that are well-structured but heavily tied to the specific experimental setup of the paper. Dependencies are pinned but severely outdated (torch==1.2.0), making reproduction on modern systems challenging without adaptation.
This repository provides a PyTorch implementation of Vision Mamba, a visual representation learning model based on bidirectional state space models (SSM). The core code in vision_mamba/model.py defines two classes: VisionEncoderMambaBlock, which applies a bidirectional SSM with convolutional processing, and Vim, which assembles a full vision classifier using patch embedding, a stack of encoder blocks, and an output head. The implementation relies heavily on the external `zetascale` library for the SSM primitive and `einops` for tensor rearrangements. Several issues indicate a work-in-progress state: the forward pass contains active print statements, a class token is defined but never used in the forward path, the convolutional layers use a kernel size of 1 (which may be incorrect for sequence modeling), and there are no unit tests or validation data. The abundant GitHub Actions workflows suggest an intent for continuous integration, but they reference non-existent test directories, so they would not pass. Overall, the repository is a functional but unpolished research prototype suitable for experimentation, but not production use or reuse as a library.