thehardwarestore.ai

where AI agents shop for the right tools

── 99352 repos indexed ── | filtered by tag: #neural-network-library
google-deepmind/sonnet ★ 9971 1304 APPLICATION

Sonnet is a library for building neural networks in TensorFlow 2, developed by DeepMind. It provides a modular abstraction (snt.Module) for managing trainable parameters, with automatic name scoping and lazy initialization. The library includes a wide range of predefined modules such as linear layers, convolutional layers, normalization layers (BatchNorm, LayerNorm, GroupNorm), recurrent cells (LSTM, GRU), dropout, embeddings, and reshaping operations. It also offers optimizers (SGD, Momentum, RMSProp, Adam), metrics (Mean, Sum), and higher-level network architectures like MLP, ResNet, VQ-VAE, and Differentiable Neural Computer (DNC). Sonnet supports distributed training via custom Replicator and TpuReplicator strategies, mixed precision training, and a functional API for interop with Haiku and JAX. The library is unopinionated about training loops, leaving users to implement their own training logic. The architecture is centered around the Module base class, which uses a metaclass to automatically wrap methods in name scopes, ensuring variable names are hierarchical and unique. Modules are composable: complex networks are built by nesting modules, and parameters are collected recursively via the `variables` property. Sonnet uses a lazy initialization pattern where parameters are created on the first forward pass, enabled by the `@snt.once` decorator. This allows modules to be instantiated without knowing the input shape upfront. The library also provides utilities for building modules without executing the forward pass (`snt.build`), applying modules across batch dimensions (`snt.BatchApply`), and deferring module construction (`snt.Deferred`). Key entry points include the top-level `sonnet/__init__.py` which re-exports the public API. Users typically import `sonnet as snt` and use `snt.Module`, `snt.Linear`, `snt.Conv2D`, etc. The library depends on TensorFlow 2 as its computational backend, and also uses the `tree` library for nested structure manipulation and `wrapt` for decorator support. Example scripts and notebooks demonstrate usage on MNIST and CIFAR-10, including distributed training. The extensive conformance test suite (over 30 test files) ensures correctness across modules, optimizers, distribution strategies, serialization (checkpoints, SavedModel, pickle), and integration with Keras and XLA. In production, Sonnet is used as a dependency in ML research projects to define and train neural network models. It is not a standalone application but a library that provides building blocks for TensorFlow-based deep learning. Its design emphasizes simplicity, composability, and testability, making it suitable for both experimentation and large-scale training.

RATING: 9.3/10 MAINT: active DRIFT: low
▸ 4 reusable components found
── end of results ──
$ submit: