Peekaboo is a mature macOS desktop automation toolkit that provides a CLI, Mac app, and MCP server for screen capture, UI element detection, input synthesis, window management, and AI agent integration. The architecture is modular: a Swift CLI (peekaboo) exposes ~33 commands for interaction, system control, and AI agent orchestration; a Mac app provides a menu bar interface, session management, and visualizer overlays; a bridge daemon (PeekabooBridge) enables secure remote automation via a versioned socket protocol with operation allowlisting. The MCP server exposes native automation tools to AI clients (Claude, Cursor, etc.) via stdio transport. The agent runtime (PeekabooAgentService) supports multiple AI providers (OpenAI, Anthropic, Ollama, etc.) with streaming, tool calling, and session persistence. The codebase is written in Swift 6.2 with strict concurrency, extensive unit/integration tests, and a focus on safety (fail-closed target validation, honest effect reporting). It is not a data research project; it is a production-grade application for macOS automation.
MacosUseSDK is a Swift library and collection of command-line tools for macOS accessibility automation. It enables traversing application accessibility trees to inspect UI elements, simulating user input (mouse/keyboard), opening applications, and providing visual feedback for debugging. The architecture centers around a core library with specialized modules for traversal, input simulation, visual feedback, and action orchestration, with six CLI tools exposing different subsets of functionality. Primary use cases include UI testing, accessibility verification, and automated interaction with macOS applications.
This repository provides a toolkit for developing JavaScript for Automation (JXA) scripts on macOS using TypeScript. It includes packages for running JXA code from Node.js (@jxa/run), an interactive REPL (@jxa/repl), type definitions for macOS applications (@jxa/types and @jxa/global-type), and a tool to generate TypeScript definitions from Apple's SDEF files (@jxa/sdef-to-dts). The monorepo structure (managed by Lerna) separates concerns into focused packages, enabling type-safe and efficient automation of macOS applications.
SYSTEM is an AI-powered macOS automation assistant enabling natural language control of Mac functions through a distributed architecture. It consists of a Tauri-based desktop app for local management, a CLI for setup/diagnostics, a local bridge server executing commands on the Mac, and a Cloudflare Worker serving as the AI agent using Anthropic's Claude. The system emphasizes privacy by keeping processing on user infrastructure, with secure Bearer token authentication between components. The architecture features a modular tool system where each capability (music, reminders, calendar, etc.) follows a consistent interface for easy extension. Communication occurs via HTTP/WebSocket: the UI sends requests to the Cloudflare Agent, which processes natural language with Claude and commands the local bridge. The bridge validates inputs, executes tools (using AppleScript/shell commands), and returns results. Real-time updates use WebSockets for scheduled tasks and notifications. Security is prioritized through command allowlisting, dangerous pattern blocking, and explicit macOS permission requirements. Key reusable components include secure command execution utilities and input validation patterns. The bridge server's validation and command modules isolate security concerns, while the tool interface pattern enables extensibility. The system is actively maintained with a pre-commit hook and low issue count, though distributed components risk drift. Dependencies show no known vulnerabilities (Dependabot Alerts: 0), but reliance on external services (Anthropic, Cloudflare) requires monitoring for API changes.