thehardwarestore.ai

llama.cpp is a high-performance C/C++ inference engine for large language models (LLMs) and vision-language models (VLMs). It loads models in the GGUF format and runs them on a wide range of hardware, including CPU, CUDA, Metal, Vulkan, SYCL, OpenCL, Hexagon DSP, and more. The project provides a core library (`llama.cpp` and `ggml`), a CLI client (`llama-cli`), an OpenAI-compatible HTTP server (`llama-server`), a Svelte-based web UI, and a suite of tools for quantization, perplexity evaluation, and model conversion. It supports dozens of model architectures (LLaMA, Mistral, Qwen, Gemma, DeepSeek, etc.) and advanced features like speculative decoding, grammar-based sampling, flash attention, and LoRA adapters. The architecture is modular: the `ggml` tensor library provides a pluggable backend abstraction, while model-specific graph builders in `src/models/` construct computation graphs. The project is actively maintained with frequent releases and broad community adoption.

ncnn is a high-performance neural network inference framework designed for mobile, embedded, and desktop deployment. It provides a self-contained C++ library with no mandatory external dependencies, supporting CPU inference with extensive SIMD optimizations (ARM NEON, x86 SSE/AVX/AVX512, RISC-V Vector, MIPS MSA, LoongArch LSX/LASX) and GPU inference via Vulkan. The framework includes a comprehensive set of neural network layers (convolution, pooling, normalization, activation, recurrent, attention, etc.) and supports multiple data types (float32, float16, bfloat16, int8). The companion pnnx tool converts models from PyTorch (TorchScript) and ONNX into ncnn format using a sophisticated graph rewriting pipeline. Python bindings and a model zoo are provided for rapid prototyping. The codebase is production-grade, used in Tencent applications like QQ and WeChat, and is actively maintained with support for modern architectures and operations (e.g., scaled dot-product attention, rotary embeddings).

This repository contains the public-facing source code for Draw Things, a production-grade AI image and video generation application for macOS/iOS/Linux. The codebase implements a comprehensive suite of diffusion models (Stable Diffusion, FLUX, HiDream, Qwen, MiniMax, etc.), samplers, LoRA training, model conversion, and a gRPC-based distributed inference server. It is written primarily in Swift, using Bazel for builds, and includes extensive use of FlatBuffers for data serialization, SQLite for persistence, and a custom neural network compiler (NNC) for on-device inference. The architecture is modular, with separate libraries for tokenization (CLIP, Tiktoken, SentencePiece), model zoos (downloadable model specifications), image generation pipelines, and cloud authentication. The repository also includes CLI tools (draw-things-cli, gRPCServerCLI, model converters) and server management scripts for deploying GPU-backed inference services with Tailscale, Envoy, and Docker. While the core application is a consumer-facing app, the codebase also contains research-grade components such as instruction count estimators for performance profiling, formal verification tools (Z3, Lean), and web search integration with browser fallback. The code is well-structured with many reusable patterns, though it carries some maintenance overhead from vendored dependencies and generated code.

O! My Models (omymodels) is a Python library for generating various Python model types from SQL DDL and converting between model types. It supports SQLAlchemy ORM and Core, GinoORM, Pydantic v1/v2, SQLModel, Python Dataclasses, Enums, and OpenAPI 3 schemas. The architecture follows a pipeline: SQL DDL parsing → normalization → type conversion → model generation → Jinja2 template rendering. Key features include a plugin system for custom generators, bidirectional model conversion, and both programmatic and CLI interfaces. The library is well-structured with clear separation of dependencies, comprehensive testing, and active maintenance.

LunaVox is a high-performance, cross-platform text-to-speech inference engine specialized for Qwen3-TTS models. It provides a unified C++ core with Python bindings, offering multiple interfaces: a CLI (`lunavox`), a desktop GUI (customtkinter), an HTTP/WebSocket server (FastAPI), and a Python runtime API. The architecture separates model loading, tokenization, LLM-based talker/predictor generation (via llama.cpp), and neural audio codec decoding (via ONNX Runtime) into modular components. Voice modes include base, voice cloning from reference audio, custom speaker selection, and text-described voice design. The system is designed for low-latency streaming synthesis, with benchmarks showing real-time factors below 0.2 on consumer GPUs. It includes a comprehensive model conversion pipeline from HuggingFace to GGUF/ONNX formats, a build toolchain for compiling the C++ engine, and production-ready serving with Prometheus metrics and auto-batching. The codebase demonstrates strong engineering practices with cross-platform support, structured stats contracts, and bilingual documentation, though it carries some technical debt from vendored GGUF code and platform-specific workarounds.