thehardwarestore.ai

This repository is the official release hub for the MiniCPM family of compact large language models (1B to 8B parameters) from OpenBMB. It provides model weights, deployment scripts for 10+ inference backends (vLLM, SGLang, llama.cpp, Ollama, etc.), fine-tuning recipes for 5 frameworks (TRL, LLaMA-Factory, unsloth, etc.), quantization tools (AWQ, GPTQ, BNB), and application demos (survey generation, MCP tool calling, code interpreter). The repo is primarily a model card and cookbook collection rather than a single integrated application; it serves as a developer toolkit for deploying and customizing MiniCPM models in production and edge scenarios. The repository also includes research artifacts such as training datasets (UltraData suite) and technical reports, but the dominant character is applied machine learning engineering.

The QwenLM/Qwen3 repository provides resources for the Qwen3 family of large language models, including model cards, documentation, and usage examples. It serves as an application-focused hub for inference, deployment, training, and evaluation of Qwen3 models. The repository contains extensive multilingual documentation, inference demos (web and CLI), deployment guides for various frameworks (vLLM, SGLang, etc.), training examples (using Axolotl, LLaMA-Factory, etc.), quantization techniques, and evaluation harnesses (notably for ARC-AGI). While the core model weights are hosted externally, this repo enables end-to-end usage through well-organized examples and scripts.

This repository is a comprehensive toolkit for analyzing, optimizing, and transforming ONNX models. It provides a core library (onnx_tool) for loading models, performing graph analysis (shape/value inference), profiling computational complexity (MACs, parameters, memory), applying optimizations (operator fusion, constant folding), quantizing weights, and managing memory for efficient inference. The toolkit supports a wide range of model architectures including LLMs (BERT, GPT, LLaMa, MPT, Qwen), diffusion models (Stable Diffusion), computer vision (ResNet, EfficientNet), and multimodal models. Key features include a custom inference engine (GraphInfer) with memory pooling for zero-copy tensor views, a plugin-based node registry system for extending ONNX operator support, and utilities for model conversion and evaluation. The architecture centers around the onnx_tool package, which defines fundamental abstractions like Graph, Tensor, and Node. The Graph class handles model loading, shape inference, and transformation pipelines. The Node registry system allows dynamic registration of operation-specific handlers for shape/value inference and profiling. Optimization modules (fusion, quantization) work on the graph representation, while the inference engine (inference/infer.py) executes optimized graphs using a memory pool to minimize allocation overhead. Additional components include benchmarking scripts for performance analysis, model configuration management, and specialized support for LLM KV-cache handling. While the core library demonstrates solid engineering with reusable patterns like the node registry and graph rewrite framework, there are areas of technical drift due to loose dependency versioning and the broad scope requiring continuous maintenance to keep pace with evolving model architectures and ONNX specifications. The project appears maintained based on recent commit activity and community engagement (stars, forks, issue templates), though dependency management without version constraints poses a risk for long-term stability.

OpenCV Zoo is a curated collection of pre-trained computer vision models optimized for OpenCV's DNN module, accompanied by inference demo scripts (Python and C++), benchmarking harness, evaluation tools, and quantization utilities. The repository serves as a practical resource for deploying models on a wide range of hardware (x86, ARM, RISC-V, NPUs) by providing ready-to-run code for tasks such as face detection/recognition, object detection, segmentation, text detection/recognition, pose estimation, and more. While primarily an application-oriented model zoo, it also includes research-grade evaluation metrics (e.g., LFW, WIDERFace) and quantization workflows, giving it a hybrid character that bridges model deployment and performance characterization. The benchmarking framework and quantization tools are the most architecturally coherent parts; model-specific demos are utilitarian but tightly coupled to their respective ONNX graphs.

ExLlamaV2 is a high-performance inference library for running quantized large language models (LLMs) on consumer GPUs, supporting GPTQ and the custom EXL2 quantization formats. It provides a Python library with custom CUDA/C++ extensions for efficient attention, MLP, sampling, and KV cache management, along with advanced features like dynamic batching, paged attention, speculative decoding, LoRA, tensor parallelism, and multimodal (image/video) support. The library includes a WebSocket server for remote inference and a suite of example scripts demonstrating various use cases. While the codebase exhibits strong engineering with clean abstractions (e.g., module system, filter pattern, multi-backend attention) and highly optimized CUDA kernels, the repository has been archived in favor of ExLlamaV3, meaning no further updates or bug fixes are expected. This makes it a valuable reference for understanding quantized LLM inference internals but risky for new projects requiring ongoing support.

This repository is a personal cookbook and project showcase centered on the MiniCPM family of large language models (MiniCPM, MiniCPM-V, MiniCPM3). It contains over a dozen distinct application projects, including multi-modal OCR + visual grounding (OCR_VG), cross-modal image-text search, ReAct-style agent planning (agent_auto_plan), MBTI personality role-play, low-memory RAG (4G_memory_rag), AWQ quantization, embedding extraction, SFT/DPO fine-tuning pipelines, and Windows agent demos. Each project includes source code, shell scripts, and accompanying Chinese/English tutorials in markdown format. The codebase is primarily a collection of independent, loosely coupled modules rather than a unified application. It demonstrates practical usage of MiniCPM models for fine-tuning, deployment with vLLM/Ollama/llama.cpp, and integration with LangChain. The quality is functional but inconsistent: many modules contain hardcoded paths, duplicated logic, and lack tests, yet several components (ReAct data generation, image slicing utilities, LangChain wrapper) exhibit sufficient isolation and craftsmanship to be reusable across projects. The repo is actively maintained with a recent commit, but dependency on unmerged forks and version-specific model files poses a medium technical drift risk.