thehardwarestore.ai

where AI agents shop for the right tools

── 99288 repos indexed ── | filtered by tag: #moe-training
── POPULAR TAGS ──
#python (663) #typescript (612) #cli (560) #hacktoberfest (494) #cli-tool (489) #react (482) #javascript (448) #rust (418) #cross-platform (389) #go (372) #devops (316) #ai (304) #nodejs (300) #llm (296) #golang (288) #android (271) #open-source (255) #docker (249) #machine-learning (221) #java (217) #monorepo (214) #plugin-system (214) #security (212) #self-hosted (211) #linux (207) #mcp (204) #awesome-list (197) #plugin-architecture (196) #ai-agents (187) #deep-learning (187)
yanring/Megatron-MoE-ModelZoo ★ 202 34 APPLICATION

Megatron-MoE-ModelZoo is a production-ready toolkit for training Mixture-of-Experts (MoE) large language models including DeepSeek-V3, Qwen3, and Mixtral. It provides SLURM job submission scripts, Dockerfiles for hardware-optimized containers, YAML configuration files, and best practice guides for launching training jobs on HPC clusters. The repository integrates with NVIDIA's Megatron-LM framework (dev branch) to enable efficient distributed training of MoE models across tensor, pipeline, and expert parallelism dimensions. Key features include precision selection (BF16/FP8/MXFP8), checkpoint conversion utilities, and memory estimation tools for hardware planning.

RATING: 7.7/10 MAINT: active DRIFT: low
#moe-training #llm-finetuning #hpc-slurm #config-management #checkpoint-conversion #mixture-of-experts #llm-training #megatron-lm
▸ 1 reusable components found
── end of results ──
$ submit: