$ show datawhalechina/self-llm
This repository is a comprehensive collection of tutorials and code examples for deploying, fine-tuning, and integrating open-source large language models (LLMs), with a strong focus on the Chinese LLM ecosystem. It covers over 50 models (Qwen, ChatGLM, DeepSeek, InternLM, etc.) and provides step-by-step guides for setting up FastAPI services, Streamlit web demos, LangChain knowledge base assistants, and LoRA/QLoRA fine-tuning using PEFT. The repository also includes example applications such as role-playing chatbots and digital persona creation. While primarily educational, the code patterns are directly applicable to production scenarios, including model serving with vLLM/SGLang, experiment tracking with SwanLab, and deployment on diverse hardware (NVIDIA, AMD ROCm, Ascend NPU, Apple Silicon). The codebase is highly repetitive across models, with significant copy-paste, but the underlying patterns (fine-tuning pipeline, deployment scaffold, LangChain wrapper) are well-established and reusable.
The LLM identified 5 files worth highlighting for reuse:
Provides a BaseBackend abstract class and create_backend factory function that abstracts away the differences between MLX and Transformers backends for LLM inference. This pattern allows the Gradio app to switch between backends without code changes, demonstrating clean separation of concerns and extensibility.
A self-contained utility for downloading models from Hugging Face or ModelScope with progress tracking and error handling. It encapsulates the download logic in a single function, making it reusable across different models and projects.
A complete notebook for fine-tuning BGE-M3 embedding models using contrastive learning with in-batch negatives and temperature scaling. It includes a validation step using C-MTEB retrieval evaluation, demonstrating a rigorous training pipeline for embedding models.
A production-grade fine-tuning script for large models (up to 427B parameters) using DeepSpeed ZeRO-3 with CPU offloading and streaming. It includes environment setup, model patching, and tmux-based execution, showcasing advanced distributed training techniques.
A utility module for processing image and video inputs for multimodal VLMs. It handles resizing, padding, and frame extraction, providing a clean interface for preparing visual data for model inference.