Unsloth is a full-stack, local-first AI platform for fine-tuning and inference of large language models (LLMs), diffusion models, and audio models. It consists of two primary components: (1) the core `unsloth` Python library, which provides highly optimized LoRA/QLoRA/RL training with custom Triton kernels, and (2) the Unsloth Studio application, a Tauri desktop app with a React frontend (TanStack Router, Zustand) and a FastAPI backend (SQLite, structlog). The platform enables users to discover models from the Hugging Face Hub, download and manage them locally, run inference with chat, image/video generation, and RAG, fine-tune with a visual recipe editor, and export in multiple formats (GGUF, NVFP4). It also integrates external LLM providers (OpenAI, Anthropic, Ollama) and supports agentic tool loops via MCP and Codex transports. The codebase is extensively tested with both unit tests and source-level contract tests, and includes cross-platform desktop installers for Windows, macOS, and Linux.
H2O LLM Studio is a web-based application for fine-tuning large language models (LLMs) without requiring coding experience. It provides a GUI built on H2O Wave and CLI tools for dataset import, experiment configuration, model training, evaluation, and export to Hugging Face Hub. The system supports multiple problem types including causal language modeling, classification, regression, sequence-to-sequence, and Direct Preference Optimization (DPO). The architecture centers around a Wave server handling both UI and backend logic, with distributed training capabilities via DeepSpeed. Key components include a configuration system using dataclasses, utilities for conversation chain handling, model loading abstractions, and factory patterns for training components.
TinyLLaVA Factory is a modular Python framework for building, training, evaluating, and serving small-scale Large Multimodal Models (LMMs). It provides a pluggable architecture with separate components for vision towers (CLIP, SigLIP, DINOv2), language models (Phi, TinyLlama, Gemma, Qwen2, etc.), and connectors (MLP, QFormer, Resampler), all accessible via factory registries. The training pipeline supports two-stage pretraining (alignment) and finetuning with full, LoRA, or QLoRA recipes using DeepSpeed. It includes a comprehensive evaluation suite for standard benchmarks (VQA, GQA, SQA, POPE, MMMU, etc.) and a Gradio-based web demo. The codebase is well-organized and follows clean architectural patterns, though many modules are tightly coupled to the internal framework. It is actively maintained, with recent additions like a visualization tool and support for new models.
LMFlow is a comprehensive toolkit for fine-tuning, aligning, and deploying large foundation models. It provides a modular library with support for various efficient fine-tuning methods (LoRA, QLoRA, LISA), alignment techniques (DPO, RAFT), and inference backends (vLLM, SGLang). The architecture separates concerns into models, pipelines, optimizers, and utilities, with factory patterns for extensibility. Key features include conversation templating for multiple model families, custom optimizers, and distributed training integration. The toolkit is actively maintained with recent commits and addresses current research in model alignment and efficiency.
Megatron-MoE-ModelZoo is a production-ready toolkit for training Mixture-of-Experts (MoE) large language models including DeepSeek-V3, Qwen3, and Mixtral. It provides SLURM job submission scripts, Dockerfiles for hardware-optimized containers, YAML configuration files, and best practice guides for launching training jobs on HPC clusters. The repository integrates with NVIDIA's Megatron-LM framework (dev branch) to enable efficient distributed training of MoE models across tensor, pipeline, and expert parallelism dimensions. Key features include precision selection (BF16/FP8/MXFP8), checkpoint conversion utilities, and memory estimation tools for hardware planning.
This repository contains educational labs for MIT's Introduction to Deep Learning course (6.S191). It consists of Jupyter notebooks covering deep learning fundamentals (Lab1: music generation with RNNs), computer vision (Lab2: MNIST classification and facial detection with VAEs), and LLM fine-tuning (Lab3: style adaptation using LoRA). The repository also includes extra labs for reinforcement learning and uncertainty quantification. While structured as educational material, the labs involve substantive data processing, model training, and evaluation pipelines. The mitdeeplearning package provides reusable utilities for training visualization and data handling. The code assumes execution in Google Colab with GPU access and relies on external APIs (Comet ML, OpenRouter) and datasets hosted on third-party services (Dropbox).
This is a research repository from the SCIR-HI lab at Harbin Institute of Technology, focused on instruction-tuning large language models (LLaMA, Alpaca, Bloom, Huozi) with Chinese medical knowledge. The core contribution is the construction of Chinese medical instruction datasets using structured knowledge bases (cMeKG) and medical literature via GPT3.5, followed by LoRA-based fine-tuning. The repository provides scripts for fine-tuning, inference, and model export, along with pre-trained LoRA weights. While it demonstrates a complete research pipeline (data generation, training, evaluation), it is primarily a research artifact rather than a production application. The code is functional and includes a clean, reusable Prompter class for template management, but lacks software engineering rigor (no tests, no CI). Reproducibility is moderate: dependencies are pinned, but the data generation process is not fully automated, and external base models and LoRA weights must be obtained separately.
This repository is a personal cookbook and project showcase centered on the MiniCPM family of large language models (MiniCPM, MiniCPM-V, MiniCPM3). It contains over a dozen distinct application projects, including multi-modal OCR + visual grounding (OCR_VG), cross-modal image-text search, ReAct-style agent planning (agent_auto_plan), MBTI personality role-play, low-memory RAG (4G_memory_rag), AWQ quantization, embedding extraction, SFT/DPO fine-tuning pipelines, and Windows agent demos. Each project includes source code, shell scripts, and accompanying Chinese/English tutorials in markdown format. The codebase is primarily a collection of independent, loosely coupled modules rather than a unified application. It demonstrates practical usage of MiniCPM models for fine-tuning, deployment with vLLM/Ollama/llama.cpp, and integration with LangChain. The quality is functional but inconsistent: many modules contain hardcoded paths, duplicated logic, and lack tests, yet several components (ReAct data generation, image slicing utilities, LangChain wrapper) exhibit sufficient isolation and craftsmanship to be reusable across projects. The repo is actively maintained with a recent commit, but dependency on unmerged forks and version-specific model files poses a medium technical drift risk.