$ show sophgo/sophon-demo
This repository is a comprehensive collection of AI model demos and sample applications for Sophgo's Sophon TPU hardware (BM1684X, BM1688, CV186X). It provides both Python and C++ implementations for running inference on a wide range of models including object detection (YOLO variants, CenterNet), segmentation (SAM, YOLACT), OCR (PP-OCR, LPRNet), large language models (Qwen, ChatGLM, MiniCPM), speech recognition (Whisper, WeNet), text-to-speech (ChatTTS, VITS), and multi-modal models (CLIP, BLIP, GroundingDINO). The repository also includes full application demos such as a document QA chatbot (ChatDoc), an audio assistant pipeline (ASR+LLM+TTS), multi-channel video processing with Qt GUIs, and OpenAI-compatible API servers. The primary architecture is a set of independent, self-contained demos, each with its own model compilation scripts (using TPU-MLIR), inference code, and evaluation tools. While the codebase is functional and covers a broad spectrum of AI models, it suffers from significant code duplication across samples, hardcoded paths, and some outdated dependency versions, making it more of a demo showcase than a production-ready library.
The LLM identified 2 files worth highlighting for reuse:
A clean, reusable Python class that wraps the Sophon SAIL engine for TPU inference. It handles model loading, input/output tensor management, and supports both numpy arrays and bm_image inputs. The class provides methods like `infer_numpy` and `infer_numpy_dict` that abstract away the low-level SAIL API, making it easy to integrate TPU inference into any Python application. It is self-contained (only depends on `sail` and `numpy`) and could be extracted into a standalone library with minimal modification.
A standalone C++ `TimeStamp` class for performance profiling. It supports named timestamps, loop averaging, and summary output. Used consistently across many C++ demos, it provides a simple, reusable way to measure inference latency without external dependencies. The class is header-only and could be extracted into any C++ project.