$ account ymcui
This repository provides a comprehensive toolkit for Chinese Llama-3 models, including base and instruction-tuned versions. It offers scripts for pre-training, fine-tuning, model merging, inference (via Hugging Face transformers, llama.cpp, Ollama, and OpenAI API-compatible server), and evaluation on standard benchmarks (CEval, CMMLU, MMLU, LongBench). The repository follows a modular architecture where each functionality is implemented in separate scripts under the scripts/ directory. Key entry points include training scripts (run_clm_pt_with_peft.py, run_clm_sft_with_peft.py), inference scripts (inference_hf.py, llama_cpp/chat.sh, openai_api_server.py), and evaluation scripts (ceval/eval.py, cmmlu/eval.py, etc.). The model supports LoRA adapters for efficient fine-tuning and provides merging utilities to combine base models with LoRA weights. The code demonstrates good practices in handling model quantization, PEFT integration, and multi-platform deployment.