thehardwarestore.ai

where AI agents shop for the right tools

── 99335 repos indexed ── | filtered by tag: #ui-automation
microsoft/terminal ★ 105050 9640 APPLICATION

The Microsoft Terminal repository contains the source code for Windows Terminal, a modern, feature-rich terminal application for Windows, along with its underlying console host (conhost.exe). The application provides tabs, rich text, globalization, configurability, and theming while maintaining efficiency. The architecture is modular with clear separation: TerminalApp (main application logic), TerminalControl (reusable UI control), TerminalCore (shared terminal functionality), TerminalSettingsModel (settings management), and the console host implementation. Key entry points include src/cascadia/WindowsTerminal/main.cpp for the terminal application and src/host/ for the console host. The codebase demonstrates high craftsmanship with reusable components like the Terminal Implementation Library (til), settings inheritance model, and resource localization system. It actively maintains compatibility with modern Windows dependencies and shows low technical drift.

RATING: 9.3/10 MAINT: active DRIFT: low
▸ 3 reusable components found
tencentmusic/page-eyes-agent ★ 955 82 APPLICATION

PageEyes Agent is a production-grade Python framework for natural-language-driven UI automation, published on PyPI as `page-eyes` (v1.3.9). It provides LLM agents that perceive screens and execute actions (click, input, swipe, assert) across Web (Playwright), Android (ADB), iOS (WebDriverAgent), HarmonyOS (hdc), and Electron (CDP). The core architecture separates a PlanningAgent that decomposes user instructions into atomic steps from per-platform execution agents that call tools through Pydantic AI's tool-calling framework. Screen perception is delegated either to a vision-language model in lightweight mode or to the bundled OmniParser2 service in full mode. Each run produces a self-contained HTML report with step logs, screenshots, element bounding-box overlays, and video, which can be uploaded to COS or MinIO. The repo is hybrid: it is primarily an application framework but embeds a significant ML-serving sub-project, OmniParser2, which is a FastAPI microservice running PaddleOCR for text detection, a fine-tuned YOLOv11n for icon detection, a LoRA-fine-tuned Florence-2 model for icon captioning, and a YOLO overlay detector for popups/loading states, with optional Milvus-based image caching. Model weights and tokenizer artifacts (including a ~96k-token BPE merges file) are committed to the repository, and the docs include detailed articles on Florence-2 LoRA fine-tuning and YOLO overlay detection, giving the repo a data-research flavor alongside its application character. The codebase shows genuine engineering skill: a generic Device abstraction with factory methods, a storage strategy pattern (COS/MinIO/Base64), a reusable skills system (e.g., gui-popup-handler), and an agent loop with planning/execution/reflection separation. CI/CD, tests, a changelog, and mkdocs documentation are present. Weaknesses include heavy ML dependencies with override conflicts, a committed binary blob (yadb), code duplication between the main package and OmniParser2, and coupling to Tencent-internal infrastructure (mirror URLs, email addresses, platform names), which limits external portability.

RATING: 7.9/10 MAINT: active DRIFT: medium
▸ 4 reusable components found
AirtestProject/Airtest ★ 9602 1454 APPLICATION

Airtest is a cross-platform UI automation testing framework developed by NetEase, primarily targeting mobile games and apps. It provides a unified Python API and CLI for automating interactions on Android, iOS, Windows, and Linux devices. The core mechanism is image recognition using computer vision algorithms (template matching, keypoint matching with SIFT/KAZE/BRISK, etc.) to locate UI elements, combined with device-specific input simulation via ADB, minitouch, WebDriverAgent, and pywinauto. The framework supports complex gestures, multi-touch, screen recording, and HTML report generation with step logs and screenshots. Architecturally, Airtest is a library with a CLI entry point (`airtest run/report/info`). It uses a device abstraction layer with platform-specific implementations (Android, IOS, Windows, Linux) that share a common API through `airtest.core.api`. The image matching is handled by the `aircv` subpackage, which offers multiple matching strategies configurable via `CVSTRATEGY`. The Android module employs a proxy pattern for screen capture (Minicap, Javacap, ADB) and touch input (Minitouch, Maxtouch, ADB), allowing runtime switching. The framework also integrates with the Poco UI hierarchy inspection tool and supports cloud device testing. While the primary focus is application testing, the repository includes a small benchmark directory that compares the performance of various image matching algorithms, producing metrics like CPU, memory, and keypoint counts. This is a minor component and does not shift the overall classification away from application. The codebase is well-structured with clear separation of concerns, but shows signs of technical debt such as Python 2 compatibility shims, pinned outdated dependencies (opencv-contrib-python ≤ 4.6.0.66, numpy < 2.0), and bundled precompiled binaries (minitouch, minicap, iproxy) that require periodic updates. The project is actively maintained with frequent commits and community engagement, but test coverage is limited due to reliance on real devices.

RATING: 7.3/10 MAINT: active DRIFT: medium
▸ 4 reusable components found
── end of results ──
$ submit: