BitNet is a PyTorch library implementing the BitNet architecture for 1-bit transformers, as described in the paper "BitNet: Scaling 1-bit Transformers for Large Language Models". The library provides drop-in replacements for standard linear layers with BitLinear layers that use quantized weights and activations to enable efficient computation. It includes components such as BitMGQA (Multi-Grouped Query Attention), BitFeedForward, and full transformer models (BitNetTransformer), as well as adapters for other architectures like Mamba and Mixture of Experts. The library also offers utilities to convert existing Hugging Face or PyTorch models to use BitLinear layers and includes an optimized CUDA kernel for low-precision GEMM operations to accelerate inference. The architecture is modular, with the core BitLinear layer encapsulated in `bitnet/bitlinear.py`, which can be used independently. Higher-level components like attention and feed-forward networks are built using BitLinear and are designed to be reusable in other projects. The library supports training and inference workflows, as demonstrated by the provided example scripts and training routine for the enwik8 dataset. With active maintenance and a focus on performance through CUDA optimization, BitNet serves as a specialized tool for researchers and practitioners interested in extremely low-precision deep learning models.

This repository provides a PyTorch implementation of Vision Mamba, a visual representation learning model based on bidirectional state space models (SSM). The core code in vision_mamba/model.py defines two classes: VisionEncoderMambaBlock, which applies a bidirectional SSM with convolutional processing, and Vim, which assembles a full vision classifier using patch embedding, a stack of encoder blocks, and an output head. The implementation relies heavily on the external `zetascale` library for the SSM primitive and `einops` for tensor rearrangements. Several issues indicate a work-in-progress state: the forward pass contains active print statements, a class token is defined but never used in the forward path, the convolutional layers use a kernel size of 1 (which may be incorrect for sequence modeling), and there are no unit tests or validation data. The abundant GitHub Actions workflows suggest an intent for continuous integration, but they reference non-existent test directories, so they would not pass. Overall, the repository is a functional but unpolished research prototype suitable for experimentation, but not production use or reuse as a library.