$ show abi/screenshot-to-code
This is a full-stack web application that converts screenshots, images, text descriptions, and video recordings into functional code—primarily HTML with Tailwind CSS, but also supports React, Vue, Bootstrap, and Ionic stacks. The backend is a FastAPI server with a WebSocket-based streaming architecture. It orchestrates multiple AI models (OpenAI, Anthropic, Gemini) through a modular provider abstraction layer, using an agent system with tools such as create_file, edit_file, generate_images, extract_assets, and screenshot_preview. The frontend is built with React, Zustand for state management, Tailwind CSS, and shadcn/ui components. Users can upload images, paste URLs, type descriptions, import existing code, or record screen videos. Generated code is presented with live preview (desktop/mobile), version history (commits and variants), design system management, and an interactive select-and-edit mode for making targeted updates. The application also includes a sophisticated evaluation infrastructure for benchmarking different models and prompts, with agent run logging, cost tracking, and prompt reports that help developers compare outputs across configurations. Although primarily an application, the extensive eval system gives it a hybrid character with data-research elements.
The LLM identified 6 files worth highlighting for reuse:
A clean adapter pattern that abstracts OpenAI, Anthropic, and Gemini provider sessions behind a common ProviderSession interface. Each implementation handles API-specific streaming formats, tool call deltas, token usage, and cost extraction. This module is self-contained and could be reused in any multi-LLM orchestration system with minimal changes.
AgentToolRuntime dispatches canonical tool definitions to concrete implementations (create_file, edit_file, generate_images, etc.) using a clean command pattern. Each tool returns a structured ToolExecutionResult with multimodal parts. Designed for extensibility and testability, it could be extracted as a reusable tool execution engine for agent-based systems.
AgentRunRecorder provides robust, non-blocking logging of every LLM call, tool execution, and streaming delta. Uses SQLite indexing, content-addressed asset snapshots (SHA256), and careful exception swallowing to ensure logging never breaks generation. This pattern is reusable for any system requiring auditable, inspectable agent runs with asset persistence.
A context-based image lightbox supporting zoom, pan, keyboard navigation, and gallery browsing. Written in TypeScript with React Context, it wraps any image with lightweight modal behavior. Self-contained and stylable, it can be dropped into any React project needing image inspection.
A recursive JSON tree renderer that supports markdown rendering, media previews (images/video), and expandable/collapsible nodes. It handles large payloads gracefully and uses optional chaining to avoid crashes on missing fields. This component is reusable for any JSON visualization with rich media—debug logs, API responses, or structured data.
A well-typed Zustand store managing complex UI state for code generation: commits, variants, agent events, and generation settings. Demonstrates clean separation of concerns, memoized selectors, and structured update functions. While somewhat coupled to the app's domain, the store pattern and its typed actions could guide similar state management designs.