$ show rlleshi/phar
P-HAR is a video analysis application that performs human action recognition specifically tailored for adult content. It processes input videos through three complementary modalities: visual frames (using TimeSformer), human pose sequences (using PoseC3D), and audio features (using a ResNet-based model). The system extracts features from each modality, runs inference through specialized deep learning models, and fuses the results using a weighted late fusion approach that handles heterogeneous output spaces. The architecture follows a modular pipeline: video input undergoes frame extraction and preprocessing, then each modality is processed independently through its respective model pipeline. The RGB stream processes spatiotemporal visual features, the skeleton stream first performs human detection and pose estimation before action recognition, and the audio stream extracts loudness-normalized spectral features. Results from all three streams are aligned to a common label space and combined using weighted averaging to produce final action predictions. Output can be generated as an annotated video with overlayed action labels and confidence scores, or as a JSON file containing temporal predictions for further processing. Key entry points include the multimodal demo script for end-to-end video processing, utility scripts for dataset preparation and model evaluation, and specialized tools for audio filtering and data pipeline management. The application relies on the MMAction2 framework ecosystem (including MMDetection and MMPose) for core computer vision capabilities, supplemented by standard Python libraries for media processing and data handling.
The LLM identified 3 files worth highlighting for reuse:
Implements logic to combine predictions from multiple models with different output spaces (different numbers/classes) by aligning them to a common superset space. The core alignment algorithm is self-contained and dependency-free, making it highly reusable in any ensemble learning scenario where models predict different class sets.
Filters audio files based on EBU R128 loudness standard, keeping only those above a specified percentile threshold. Demonstrates proper use of loudness measurement (not simpler metrics like RMS) and provides a clean, reusable implementation for audio data preprocessing pipelines that need to remove quiet or irrelevant audio clips.
Converts lists of data files (e.g., video frames) to lists of corresponding feature files (e.g., extracted features), handling path transformations, missing files, and class ID mapping. Represents a reusable pattern for data pipeline construction where raw data needs to be mapped to processed features, particularly useful in feature extraction workflows.