$ show KaihuaTang/VQA2.0-Recent-Approachs-2018.pytorch
This repository implements multiple state-of-the-art Visual Question Answering (VQA) models from 2018, including bottom-up top-down attention, Bilinear Attention Networks (BAN), inter-intra modality attention, learning to count, and conditioned graph structures. It is built on Cyanogenoid's vqa-counting framework and provides a clean, modular codebase for training and evaluating VQA models on the VQA v2 dataset. The code includes data preprocessing scripts (feature extraction, vocabulary building), a flexible training pipeline with various loss functions and learning rate schedules, and evaluation scripts that compute per-category accuracy. While the code is well-structured for research purposes and contains reusable components like the counting module and fusion layers, it relies on deprecated PyTorch APIs (e.g., Variable), hardcoded paths, and lacks unit tests or environment definitions, making reproducibility challenging. The repository is primarily a research artifact for comparing attention mechanisms and counting strategies in VQA.
The LLM identified 3 files worth highlighting for reuse:
Implements a reusable counting module (Counter, PiecewiseLin) that can be plugged into any VQA model. It uses Intersection over Union (IoU) to deduplicate object proposals and applies attention-weighted summation to produce a count feature. This module is self-contained, well-documented, and can be extracted for other counting tasks in vision-language models.
Provides generic neural network building blocks: FCNet (fully connected layers with weight normalization, dropout, and activation) and Fusion (a non-linear fusion operation combining element-wise subtraction and ReLU). These components are used across multiple models and can be reused in any PyTorch project requiring modular, weight-normalized layers.
Contains TextProcessor, a padding-aware GRU encoder initialized with GloVe embeddings. It handles variable-length question sequences and produces a fixed-size representation. This module is self-contained and can be reused for any NLP task requiring a simple, effective text encoder with pre-trained embeddings.