$ show unslothai/unsloth
Unsloth is a full-stack, local-first AI platform for fine-tuning and inference of large language models (LLMs), diffusion models, and audio models. It consists of two primary components: (1) the core `unsloth` Python library, which provides highly optimized LoRA/QLoRA/RL training with custom Triton kernels, and (2) the Unsloth Studio application, a Tauri desktop app with a React frontend (TanStack Router, Zustand) and a FastAPI backend (SQLite, structlog). The platform enables users to discover models from the Hugging Face Hub, download and manage them locally, run inference with chat, image/video generation, and RAG, fine-tune with a visual recipe editor, and export in multiple formats (GGUF, NVFP4). It also integrates external LLM providers (OpenAI, Anthropic, Ollama) and supports agentic tool loops via MCP and Codex transports. The codebase is extensively tested with both unit tests and source-level contract tests, and includes cross-platform desktop installers for Windows, macOS, and Linux.
The LLM identified 5 files worth highlighting for reuse:
Provides VRAM estimation logic for loading models in 4-bit (bnb) and fp16 precision. Computes layer counts, parameter sizes, and overhead. A self-contained module that could be reused in any PyTorch model loading pipeline requiring memory budgeting.
A vendored copy of the `truststore` package that enables native platform TLS certificate verification (macOS Security Framework, Windows CertStore, OpenSSL). This entire subpackage is isolated and can be extracted to provide secure HTTPS without certifi in any Python application.
Implements a generic single-flight pattern that deduplicates concurrent asynchronous requests for the same key. The first request executes and subsequent callers receive the same result. Reusable in any frontend or Node.js codebase needing to collapse redundant API calls.
Handles end-of-stream detection for chat completions by analyzing chat template stop tokens, tool call states, and edge cases like deepseekr1 think blocks. Encapsulates complex EOS logic independently of any specific backend (llama.cpp, MLX). Could be extracted for any chat inference pipeline.
Computes API monitor statistics (tokens per second, request counts, duration) from raw event data. A self-contained module with no UI dependencies, usable for any latency/throughput monitoring widget.