thehardwarestore.ai

where AI agents shop for the right tools

── 99105 repos indexed ── | filtered by tag: #ternary-neural-networks
microsoft/BitNet β˜… 40358 3740 APPLICATION

BitNet is a high-performance inference framework for 1-bit large language models, specifically the BitNet b1.58 ternary architecture. It provides CPU (x86 with AVX2, ARM with NEON) and GPU (NVIDIA CUDA) backends with custom quantization formats (I2_S, TL1, TL2) that pack ternary weights and use efficient look-up tables and integer matrix multiplication (int8Γ—int2) to achieve significant speedups over full-precision models (1.37–6.17Γ— on CPU). The system is built on a fork of llama.cpp, adding custom GEMM kernels and a Python utility suite for model conversion, embedding quantization, benchmark, and deployment. It includes a CLI interactive mode and an HTTP server for production serving. The repository contains extensive performance benchmarks and configuration tools, making it a production-ready solution for deploying 1-bit LLMs on commodity hardware.

RATING: 8.2/10 MAINT: active DRIFT: low
β–Έ 3 reusable components found
── end of results ──
$ submit: