This repository from Google DeepMind implements a family of point tracking models (TAP-Net, TAPIR, BootsTAP, TAPNext, TAPNext++, TRAJAN) and the associated TAP-Vid and TAPVid-3D benchmarks. It is primarily a research codebase for tracking arbitrary points in video, with both JAX/Haiku and PyTorch implementations. The code includes training pipelines (Jaxline), evaluation scripts, data generation for synthetic (Kubric) and real datasets (DAVIS, Kinetics, ADT, DriveTrack, Panoptic Studio), and interactive Colab demos. While it has application-like components such as live webcam demos and a VOT challenge tracker, the dominant character is research: the code is organized around benchmarking and model experimentation rather than production deployment. The repository is actively maintained with recent commits and provides extensive documentation for reproducing results.
This repository is an awesome list curating research papers on Video-LMM Post-Training for video reasoning. It maintains a JSON dataset of paper metadata (title, release date, links, tags, etc.) and uses a Python script to generate a categorized README.md with markdown tables organized by research paradigm (Reinforced Video-LMMs, SFT for Reasoning, Test-Time Scaling, Benchmarks, Surveys). The repository serves as a reference tracker for the latest advancements in enhancing reasoning capabilities of video-focused multimodal models through post-training techniques. The hybrid nature lies in its dual role as both a data collection (the JSON paper metadata) and a lightweight data presentation system (the generation script).
P-HAR is a video analysis application that performs human action recognition specifically tailored for adult content. It processes input videos through three complementary modalities: visual frames (using TimeSformer), human pose sequences (using PoseC3D), and audio features (using a ResNet-based model). The system extracts features from each modality, runs inference through specialized deep learning models, and fuses the results using a weighted late fusion approach that handles heterogeneous output spaces. The architecture follows a modular pipeline: video input undergoes frame extraction and preprocessing, then each modality is processed independently through its respective model pipeline. The RGB stream processes spatiotemporal visual features, the skeleton stream first performs human detection and pose estimation before action recognition, and the audio stream extracts loudness-normalized spectral features. Results from all three streams are aligned to a common label space and combined using weighted averaging to produce final action predictions. Output can be generated as an annotated video with overlayed action labels and confidence scores, or as a JSON file containing temporal predictions for further processing. Key entry points include the multimodal demo script for end-to-end video processing, utility scripts for dataset preparation and model evaluation, and specialized tools for audio filtering and data pipeline management. The application relies on the MMAction2 framework ecosystem (including MMDetection and MMPose) for core computer vision capabilities, supplemented by standard Python libraries for media processing and data handling.
This repository contains the research code for Video-SALMONN-2, a family of audio-visual large language models for video understanding, developed by Tsinghua University and ByteDance. It includes three generations: Video-SALMONN-2 (based on LLaVA), Video-SALMONN-2+ (based on Qwen2.5-VL), and Video-SALMONN-2 Pro (based on Qwen3-VL). The core contribution is a caption-enhanced training paradigm that tightly aligns audio and visual signals. The codebase provides full training pipelines for supervised fine-tuning (SFT) and direct preference optimization (DPO), along with evaluation scripts, a demo mode, and a vLLM-based serving plugin for deployment. While the project is predominantly research-focused, it includes application-like components such as CLI inference and OpenAI-compatible serving. The code exhibits high duplication across the three model variants, hardcoded paths to internal cluster storage, and brittle dependency pinning, making it challenging to reproduce outside the original environment. However, several components—such as the multimodal batch sampler, audio-video processing utilities, and QFormer-based fusion—are well-structured and potentially reusable for other multimodal research projects.
CameraBench is a benchmark and research project for evaluating camera motion understanding in videos, comparing classical Structure-from-Motion (SfM) methods and modern Vision-Language Models (VLMs). The repository serves as a landing page with download scripts for the test dataset (videos and labels from Hugging Face), a leaderboard showing model performance, and links to fine-tuned Qwen2.5-VL models. The actual evaluation and training code resides in a separate repository (t2v_metrics). The repo lacks a self-contained pipeline, dependency management, or data processing scripts; it is essentially a static site with download helpers. A notebook template for a ReLU activation function exercise is included but is unrelated to the core project. The benchmark is novel and published at NeurIPS 2025, but the codebase itself is minimal and not reusable beyond data retrieval.