$ account allenai
This repository contains the research code for the paper 'What's hidden in a randomly weighted neural network?' (arXiv 1911.13299). It implements methods for discovering high-performing subnetworks within randomly initialized neural networks without training the weights, building on the lottery ticket hypothesis. The codebase is organized around PyTorch model definitions (ResNet, custom ConvNets), config-driven experiment management (YAML), and modular trainers for different pruning strategies (default, lottery, sample-based sparsification). The core contribution is a set of custom convolution layers (SubnetConv, SampleSubnetConv, FixedSubnetConv) that learn binary masks over frozen weights via straight-through gradient estimators. The repository supports experiments on CIFAR-10, ImageNet, and MNIST, with extensive configuration files for small-scale and large-scale experiments. While the code is well-structured for research reproducibility, it relies on outdated dependencies (PyTorch 1.3.0, torchvision 0.4.1) and contains hardcoded paths, making it primarily useful as a reference implementation rather than a reusable library.