llama-benchy is a CLI tool for benchmarking OpenAI-compatible LLM endpoints, providing detailed performance metrics similar to llama-bench. It measures prompt processing and token generation speeds at various context depths, supports concurrent requests, prefix caching analysis, and multiple output formats. The tool features a modular architecture with separate modules for HTTP client logic, result processing, and progress reporting. It uses asynchronous HTTP requests to efficiently handle concurrent benchmarking and includes sophisticated logic for handling streaming responses, token timing reconstruction, and latency measurement. The tool is actively maintained with recent commits and up-to-date dependencies.
This repository contains a PotPlayer subtitle translation plugin written in AngleScript. It provides real-time subtitle translation by sending subtitle text to OpenAI-compatible chat completion APIs. Two variants are included: one with context (sends recent subtitles for better accuracy) and one without (faster, no context). The plugin handles configuration via a dual-layer system: installer-injected defaults that can be overridden by PotPlayer's settings panel. It supports retry logic, request delays, hallucination detection, small model mode, and context caching (including OpenAI prompt caching and Gemini cached content). The installer is a separate component built with Inno Setup that provides a GUI for selecting variant, configuring API settings, and injecting defaults into the script files. The code demonstrates solid engineering with careful error handling, configuration precedence, and extensibility for multi-provider support.
This repository is an offline speech-to-text application that uses the faster-whisper model to transcribe audio and video files into text, JSON, or SRT formats. It provides a web interface built with LayUI and a REST API (including an OpenAI-compatible endpoint) for easy integration. The application supports CPU and CUDA acceleration, caches loaded models for efficiency, and handles media conversion via FFmpeg. The architecture consists of a Flask web server served by gevent, with a background thread processing transcription tasks from a queue. Configuration is managed via an INI file, and utilities for FFmpeg execution, update checking, and browser launching are encapsulated in a helper module. The model caching mechanism avoids reloading large models for multiple requests, improving performance in a server setting.
This repository is a Python client SDK for the OGX AI inference API, which is OpenAI-compatible. It provides both synchronous (`OgxClient`) and asynchronous (`AsyncOgxClient`) clients with typed resources for chat completions, embeddings, vector stores, models, prompts, responses, batches, files, and more. The SDK is auto-generated by Stainless from an OpenAPI spec, ensuring the API surface stays aligned with the backend. It includes a CLI tool (`ogx-client`) for interacting with the OGX server, and a higher-level agent library (`lib/agents/`) for building conversational agents with tool use, turn/step event models, and ReAct-style reasoning. The architecture follows a resource pattern: each API endpoint group (e.g., `chat/completions`, `vector_stores`) is a class inheriting from `SyncAPIResource`/`AsyncAPIResource`, registered as properties on the client. The base client handles HTTP transport via `httpx`, retries with exponential backoff, streaming via SSE, cursor-based pagination, and error classification. The agent library wraps the Responses API, manages conversation state, and synthesizes structured events from raw stream data. The CLI uses `click` for command groups covering models, vector stores, eval, datasets, providers, and inference. Key data flows: users instantiate a client, call resource methods (e.g., `client.chat.completions.create()`), which serialize parameters via Pydantic models, send HTTP requests, and parse responses. Streaming responses are handled by `Stream`/`AsyncStream` classes that decode SSE events. The agent library consumes stream events and emits turn/step lifecycle events for UI or logging. External dependencies are minimal: `httpx`, `pydantic`, `typing-extensions`, and optional `aiohttp` for async transport. The codebase is well-structured but heavily generated; the custom agent code is the most reusable and demonstrates non-obvious skill in event synthesis and tool orchestration.
This repository contains demonstration scripts and documentation for the GPT_API_free service, a forwarding proxy that provides free access to various large language models including GPT, DeepSeek, Claude, Gemini, and others. The actual forwarding server is not included; instead, the repository offers Python examples using the OpenAI library and standard HTTP requests to illustrate how to interact with the service's API endpoints for chat completions, image generation/editing, audio transcription, speech synthesis, and file analysis. The code is minimal and educational, focusing on replacing the base URL and authentication header to route requests through the proxy. The codebase is primarily a collection of usage examples tightly coupled to the external service's address and API key format. There is no production logic, no tests, and no independent reusable components. The demos follow standard patterns for calling OpenAI-compatible APIs and serve as a quick-start guide for developers integrating the service. As such, the repository does not represent a standalone application but rather a documentation supplement for the GPT_API_free project.