$ show mediar-ai/MacosUseSDK
MacosUseSDK is a Swift library and collection of command-line tools for macOS accessibility automation. It enables traversing application accessibility trees to inspect UI elements, simulating user input (mouse/keyboard), opening applications, and providing visual feedback for debugging. The architecture centers around a core library with specialized modules for traversal, input simulation, visual feedback, and action orchestration, with six CLI tools exposing different subsets of functionality. Primary use cases include UI testing, accessibility verification, and automated interaction with macOS applications.
The LLM identified 3 files worth highlighting for reuse:
Implements a sophisticated breadth-first traversal of macOS accessibility trees with intelligent filtering. Unlike depth-first approaches, this BFS method discovers sibling elements (like dialog buttons) before deep-diving into complex hierarchies. Features include geometry-based visibility filtering, role-based inclusion/exclusion, statistics collection, truncation handling, and efficient duplicate detection using a visited set. The component is self-contained and demonstrates non-obvious skill in accessibility API navigation.
Creates temporary visual overlays for debugging UI interactions with sophisticated animation handling. Implements three feedback types: bounding boxes with text labels, pulsing circles, and screen-center captions. Features include coordinate conversion between screen/window space, layered animations (pulsing/fade for circles, entrance/exit for captions), automatic cleanup via OS window management, and intelligent text truncation in overlays. The component showcases advanced Core Animation techniques and thoughtful UI feedback design.
Provides robust low-level input simulation using Core Graphics Events with proper Unicode text handling. Correctly posts keyboard/mouse events to the system event tap, including modifier flags and proper event sequencing. Notably handles Unicode text input via CGEventKeyboardSetUnicodeString to ensure compatibility with input methods and avoid collapsing multi-character inputs. Includes comprehensive key code mapping and proper event posting with settling delays. This component avoids common pitfalls in macOS input simulation.