OGX (formerly Llama Stack) is a production-grade, open-source AI inference platform that provides a drop-in replacement for the OpenAI API. It is a multi-provider API gateway supporting chat completions, embeddings, the Responses API (agentic turns with tool use), vector stores, file processing, conversations, prompts, skills, and connectors. The system is built on a clean provider plugin architecture where inline (built-in) and remote providers (30+ including Anthropic, Bedrock, Gemini, Ollama, vLLM, NVIDIA) are registered via ProviderSpec and dynamically routed. The FastAPI server auto-discovers API protocols and generates routes with telemetry (OpenTelemetry), authentication, multi-tenancy, and ABAC access control. The repository includes a Next.js admin UI, a Python CLI, OpenAPI-generated client SDKs (Python, TypeScript), benchmarking tools (RAG eval, vertical scaling), and comprehensive integration tests with recording/replay. It is designed for self-hosted or cloud deployment with Kubernetes and Docker support.

This repository is a Python client SDK for the OGX AI inference API, which is OpenAI-compatible. It provides both synchronous (`OgxClient`) and asynchronous (`AsyncOgxClient`) clients with typed resources for chat completions, embeddings, vector stores, models, prompts, responses, batches, files, and more. The SDK is auto-generated by Stainless from an OpenAPI spec, ensuring the API surface stays aligned with the backend. It includes a CLI tool (`ogx-client`) for interacting with the OGX server, and a higher-level agent library (`lib/agents/`) for building conversational agents with tool use, turn/step event models, and ReAct-style reasoning. The architecture follows a resource pattern: each API endpoint group (e.g., `chat/completions`, `vector_stores`) is a class inheriting from `SyncAPIResource`/`AsyncAPIResource`, registered as properties on the client. The base client handles HTTP transport via `httpx`, retries with exponential backoff, streaming via SSE, cursor-based pagination, and error classification. The agent library wraps the Responses API, manages conversation state, and synthesizes structured events from raw stream data. The CLI uses `click` for command groups covering models, vector stores, eval, datasets, providers, and inference. Key data flows: users instantiate a client, call resource methods (e.g., `client.chat.completions.create()`), which serialize parameters via Pydantic models, send HTTP requests, and parse responses. Streaming responses are handled by `Stream`/`AsyncStream` classes that decode SSE events. The agent library consumes stream events and emits turn/step lifecycle events for UI or logging. External dependencies are minimal: `httpx`, `pydantic`, `typing-extensions`, and optional `aiohttp` for async transport. The codebase is well-structured but heavily generated; the custom agent code is the most reusable and demonstrates non-obvious skill in event synthesis and tool orchestration.