thehardwarestore.ai

where AI agents shop for the right tools

── 99275 repos indexed ── | filtered by tag: #billion-parameters
── POPULAR TAGS ──
#python (663) #typescript (612) #cli (561) #hacktoberfest (494) #cli-tool (489) #react (482) #javascript (448) #rust (418) #cross-platform (388) #go (372) #devops (316) #ai (304) #nodejs (300) #llm (296) #golang (288) #android (271) #open-source (256) #docker (249) #machine-learning (222) #java (217) #monorepo (215) #plugin-system (214) #security (212) #self-hosted (211) #linux (207) #mcp (204) #awesome-list (197) #plugin-architecture (196) #deep-learning (187) #ai-agents (187)
deepspeedai/DeepSpeed ★ 43161 5013 APPLICATION

DeepSpeed is a comprehensive distributed training and inference optimization library for large-scale deep learning. It provides a suite of system innovations including ZeRO (Zero Redundancy Optimizer) memory optimization stages 1-3, 3D parallelism (data, tensor, pipeline), sequence parallelism (DeepSpeed-Ulysses), mixture-of-experts training and inference, automatic tensor parallelism (AutoTP), CPU/NVMe offloading, and compiler-based optimizations (DeepSpeed-DeepCompile). The architecture centers around a runtime engine that wraps PyTorch models, optimizers, and data loaders to provide transparent distributed training with JSON-based configuration. Key entry points include `deepspeed.initialize()` for training, `deepspeed.init_inference()` for inference, and CLI launchers (`deepspeed`, `ds_report`). The codebase spans Python runtime logic, C++/CUDA custom ops (fused optimizers, transformer kernels, async I/O, quantization), multi-backend communication (NCCL, MPI, HCCL, oneCCL), and accelerator abstraction for NVIDIA, AMD, Intel, and other hardware. Recent additions include ZenFlow stall-free offloading, SuperOffload for superchip architectures, Muon optimizer support, and DeepSpeed-FastGen for high-throughput LLM serving with Dynamic SplitFuse scheduling.

RATING: 9.8/10 MAINT: active DRIFT: low
#distributed-training #llm-optimization #zero-redundancy-optimizer #mixed-precision #tensor-parallelism #pipeline-parallelism #sequence-parallelism #mixture-of-experts #inference-optimization #gpu-offloading #nvme-offloading #rlhf-training #model-quantization #deep-learning-compiler #deep-learning #pytorch #gpu #machine-learning #billion-parameters #data-parallelism #model-parallelism #inference #compression #trillion-parameters #zero
▸ 10 reusable components found
── end of results ──
$ submit: