This repository is a large monorepo for a multimodal AI agent platform developed by ByteDance. It includes a desktop Electron application (UI-TARS Desktop) that allows users to control their computer using natural language via vision-language models (UI-TARS, GPT-4o, Claude, etc.). Additionally, it provides a CLI tool (Agent TARS) for running agents in browser/desktop environments, a server for multi-user agent sessions, a web UI for interacting with agents, MCP servers for browser/filesystem/commands, and a comprehensive SDK for building custom agents and operators. The architecture is heavily componentized with multiple packages for agent core, tool-calling engines (native, prompt-engineering, structured-outputs), browser automation (local, remote, sandbox), desktop automation (NutJS, ADB), and UI components. While the codebase demonstrates strong engineering with clean abstractions (event streams, plugin systems, strategy patterns), it also shows signs of rapid growth with some duplication (two agent server implementations, overlapping operator packages) and ongoing migrations. The repository is actively maintained with frequent releases and comprehensive test coverage across unit, integration, and e2e tests.
PageEyes Agent is a production-grade Python framework for natural-language-driven UI automation, published on PyPI as `page-eyes` (v1.3.9). It provides LLM agents that perceive screens and execute actions (click, input, swipe, assert) across Web (Playwright), Android (ADB), iOS (WebDriverAgent), HarmonyOS (hdc), and Electron (CDP). The core architecture separates a PlanningAgent that decomposes user instructions into atomic steps from per-platform execution agents that call tools through Pydantic AI's tool-calling framework. Screen perception is delegated either to a vision-language model in lightweight mode or to the bundled OmniParser2 service in full mode. Each run produces a self-contained HTML report with step logs, screenshots, element bounding-box overlays, and video, which can be uploaded to COS or MinIO. The repo is hybrid: it is primarily an application framework but embeds a significant ML-serving sub-project, OmniParser2, which is a FastAPI microservice running PaddleOCR for text detection, a fine-tuned YOLOv11n for icon detection, a LoRA-fine-tuned Florence-2 model for icon captioning, and a YOLO overlay detector for popups/loading states, with optional Milvus-based image caching. Model weights and tokenizer artifacts (including a ~96k-token BPE merges file) are committed to the repository, and the docs include detailed articles on Florence-2 LoRA fine-tuning and YOLO overlay detection, giving the repo a data-research flavor alongside its application character. The codebase shows genuine engineering skill: a generic Device abstraction with factory methods, a storage strategy pattern (COS/MinIO/Base64), a reusable skills system (e.g., gui-popup-handler), and an agent loop with planning/execution/reflection separation. CI/CD, tests, a changelog, and mkdocs documentation are present. Weaknesses include heavy ML dependencies with override conflicts, a committed binary blob (yadb), code duplication between the main package and OmniParser2, and coupling to Tencent-internal infrastructure (mirror URLs, email addresses, platform names), which limits external portability.