$ show microsoft/UFO
UFO (UI-Focused agent) and Galaxy constitute a multi-agent AI framework for automating GUI-based tasks across Windows, Linux, and mobile platforms. The system uses LLM-driven agents (AppAgent, HostAgent, ConstellationAgent) that interact with desktop applications via UI automation (UIA, COM) and execute distributed task workflows modeled as DAGs (task constellations). It includes a client-server architecture with WebSocket-based Agent Interaction Protocol (AIP), MCP tool servers, a configurable RAG knowledge substrate, and a Web UI (React+FastAPI) for real-time monitoring and control. The framework is primarily an application with strong research elements: it supports trajectory analysis, evaluation agents, demonstration learning, and configurable LLM backends (OpenAI, Claude, Gemini, DeepSeek, Ollama).
The LLM identified 3 files worth highlighting for reuse:
The Agent Interaction Protocol (AIP) package provides a clean layered protocol stack for agent communication over WebSockets: typed messages, protocol handlers (registration, heartbeat, task execution, device info), transport adapters (mock, websockets, FastAPI), endpoint abstractions (client, server, constellation), and resilience components (exponential backoff reconnection, heartbeat manager, timeout handling). This is a self-contained, reusable communication framework that could be extracted and applied to any multi-agent system requiring structured, resilient messaging.
The constellation editor implements a full command pattern with ICommand/IUndoableCommand interfaces, CommandInvoker, CommandRegistry, CommandHistory, and concrete commands (AddTask, RemoveTask, AddLine, SetProperty, etc.). It supports undo/redo, observer notifications, and JSON serialization of commands. This is a textbook implementation of the command pattern that can be reused in any project requiring transactional, undoable modifications to a structured data model (e.g., graph editors, workflow designers).
Provides a safe_get() HTTP client with comprehensive SSRF protection: DNS pinning (resolves hostname to IP and blocks private/loopback/multicast ranges), proxy rejection, redirect hop validation (max 5, same-origin check), authentication stripping, and configurable timeout. The UrlValidationPolicy dataclass encapsulates blocking rules and can be reused in any Python application needing secure HTTP calls to user-supplied URLs.