$ show liguodongiot/llm-action
llm-action is a comprehensive resource hub for large language model engineering that combines theoretical explanations with practical implementation code. It covers the full LLM lifecycle including data engineering, training (full fine-tuning and parameter-efficient methods like LoRA/QLoRA), model compression (quantization, pruning, distillation), inference serving, deployment, and evaluation. The repository contains both educational documentation and runnable code examples, with particular emphasis on Chinese-language resources and domestic AI hardware adaptation (especially Huawei Ascend NPUs). While it serves as a knowledge base, its inclusion of actual training scripts, inference servers, and utility tools makes it primarily application-oriented rather than purely research-focused.
The LLM identified 5 files worth highlighting for reuse:
Self-contained Jinja2 template for formatting supervised fine-tuning data according to model-specific chat formats. Handles special tokens and role markers correctly for various LLMs (Baichuan, ChatGLM, etc.). Can be extracted and used in any LLM fine-tuning pipeline with minimal modification, demonstrating proper prompt engineering practices for chat models.
Standalone script that merges LoRA adapter weights with a base LLM model to produce a unified model for inference. Takes base model path and LoRA adapter path as inputs, outputs merged model. Demonstrates correct mathematical implementation of LoRA weight merging and can be used in any PEFT workflow without dependencies on other repository components.
Self-contained GPU memory monitoring utility using NVIDIA Management Library (NVML). Provides accurate tracking of GPU memory utilization, including reserved vs. allocated memory. Can be dropped into any GPU computing project to monitor memory usage during training or inference, showing proper NVML usage patterns and cross-process memory tracking.
Complete training script integrating Parameter-Efficient Fine-Tuning (LoRA) with DeepSpeed ZeRO Stage 3 optimization and CPU offloading. Shows how to combine PEFT with advanced distributed training techniques for efficient large model fine-tuning. Can be adapted to different models/datasets by modifying configuration parameters, demonstrating best practices for memory-efficient LLM adaptation.
Self-contained Jupyter notebook for visualizing and analyzing quantization effects on LLM activations. Includes hooks to capture layer inputs/outputs before and after quantization, with 3D visualization of activation distributions. Demonstrates activation analysis techniques for quantization research and can be applied to any quantized model with minimal modification.