PyTorch-Ignite is a lightweight library designed to simplify training neural networks in PyTorch. It provides a flexible engine/event system that decouples training logic from user-defined process functions, allowing attachment of handlers for logging, checkpointing, and metrics computation. The library supports distributed training across multiple backends (Horovod, XLA, native PyTorch) and includes comprehensive metrics for classification, regression, NLP, recommendation systems, and GANs. Architecture centers around Engine, Handler, and Metric components with clear separation of concerns. Key entry points include ignite.engine.Engine, ignite.handlers.* loggers, ignite.metrics.* metrics, and ignite.distributed utilities for distributed training abstraction.
This repository is an unofficial PyTorch implementation of the Titans architecture from the research paper 'Titans: Learning to Memorize at Test Time'. It provides a library of neural network modules including Neural Memory, Memory-as-Context Transformer, and various attention mechanisms. The codebase consists of modular components that can be used as building blocks for sequence modeling, with training scripts demonstrating language modeling on the enwik8 dataset. The implementation focuses on test-time training mechanisms where the model learns to memorize at inference time through a neural memory module that updates its weights based on incoming sequences.
PyGCL is a PyTorch-based library for Graph Contrastive Learning (GCL) that provides modular components for building GCL models. The library implements graph augmentation techniques, contrasting architectures, contrastive objectives, and negative sampling strategies. It features clean abstractions that separate concerns across different graph frameworks (PyG and DGL), standardized evaluation utilities, and comprehensive examples demonstrating usage. The architecture follows a modular design with distinct modules for augmentors, losses, models, evaluation, and utilities, allowing users to mix and match components to build custom GCL algorithms.
RectifiedFlow is a unified PyTorch library for implementing Rectified Flow models, a type of generative model that learns ordinary differential equations to transform data between distributions. The library provides a modular framework for training various architectures (MLP, UNet, DiT, LightningDiT) on different datasets, with support for multiple sampling algorithms and interpolation schemes. Key features include a symbolic solver for affine interpolation equations, utilities for converting between different interpolation schemes while preserving rectified flow properties, and a flexible sampler base class for implementing custom integration methods. The codebase demonstrates strong software engineering practices with clear separation of components, comprehensive documentation through examples, and active maintenance.