thehardwarestore.ai

compressed-tensors is a production-grade Python library for compressing and quantizing large neural network models, primarily targeting LLMs. It extends the safetensors format to store and manage compressed tensors, supporting a wide range of quantization schemes (INT4/INT8/FP8/NVFP4/MXFP4/MXFP8), sparsity patterns, and rotation-based transforms (Hadamard, random matrix). The library is tightly integrated with the vLLM inference engine and Hugging Face transformers, providing a seamless path from model compression to deployment. Its architecture is modular: a registry pattern for compressors and transforms, a quantization lifecycle with calibration and forward hooks, a flexible offloading subsystem (CPU/disk/distributed) for running models beyond VRAM limits, and checkpoint conversion pipelines for formats like AutoAWQ and ModelOpt NVFP4. While primarily a software library, it also serves as a research platform for novel quantization techniques, with benchmarks and experimental features like MoE expert pruning and KV cache quantization. The codebase is well-structured with extensive tests, CI/CD pipelines, and active maintenance, though it exhibits some coupling to vLLM internals and experimental components that may not be stable.

Praxis is a JAX-based deep learning library designed for building and training large-scale machine learning models. It provides a comprehensive suite of neural network layers (transformers, convolutions, normalizations, etc.) with advanced features like quantization-aware training, sparsity techniques, and video processing components. The library emphasizes modularity through Fiddle-based configuration and includes utilities for beam search decoding, input processing, and model parallelism. Praxis is actively maintained with extensive test coverage and demonstrates strong engineering practices in its abstraction layers and hardware-specific optimizations.

llm-action is a comprehensive resource hub for large language model engineering that combines theoretical explanations with practical implementation code. It covers the full LLM lifecycle including data engineering, training (full fine-tuning and parameter-efficient methods like LoRA/QLoRA), model compression (quantization, pruning, distillation), inference serving, deployment, and evaluation. The repository contains both educational documentation and runnable code examples, with particular emphasis on Chinese-language resources and domestic AI hardware adaptation (especially Huawei Ascend NPUs). While it serves as a knowledge base, its inclusion of actual training scripts, inference servers, and utility tools makes it primarily application-oriented rather than purely research-focused.

This repository implements Mobile-GS, a real-time Gaussian splatting approach for novel view synthesis optimized for mobile devices. The system trains 3D Gaussian representations from multi-view images, applies compression techniques (vector quantization, Huffman coding, GPCC) to reduce model size, and renders novel views via differentiable EWA (Elliptical Weighted Average) splatting. It includes training pipelines (train.py, pretrain.py), rendering engine (gaussian_renderer/), scene management, custom CUDA kernels for efficient rasterization, and evaluation metrics. The architecture follows a modular design with separate components for training, rendering, scene handling, and compression, optimized for deployment on mobile platforms with limited computational resources.