$ show RunanywhereAI/on-device-browser-agent
Local Browser is a Chrome extension that enables on-device AI-powered web automation using WebLLM. The system implements a two-agent architecture (Planner + Navigator) where users enter natural language tasks in the popup UI, which are then executed through a loop of DOM observation, action planning, and browser interaction. All AI processing occurs locally on the device with no cloud APIs or API keys required. The extension features site-specific state machines for common tasks like Amazon shopping and YouTube navigation (minimizing LLM calls), obstacle detection for login requirements/CAPTCHAs/out-of-stock items, and robust action execution with retry logic and semantic selectors. Built with React, TypeScript, Vite, and CRXJS for MV3 Chrome extension architecture.
The LLM identified 4 files worth highlighting for reuse:
Implements deterministic verification of browser actions by comparing DOM state before/after execution using text hashing and pattern matching for success/error detection. Provides reliable verification without LLM inference, making it suitable for any web automation/testing framework. The component is isolated, extractable, and demonstrates non-obvious skill in efficient change detection.
Detects common web automation obstacles (login required, CAPTCHA, out of stock, errors) that require user intervention. Categorizes obstacles by recoverability and required user action, providing a clean interface for obstacle handling. The component is self-contained, easily extractable to other projects, and shows skill in classifying obstacles by their nature and resolution requirements.
Routes tasks to site-specific state machines (Amazon, YouTube) when possible, falling back to LLM-based agents for general cases. Provides a unified interface for the executor to use deterministic state machines without knowing implementation details. Demonstrates the valuable pattern of combining deterministic approaches with LLM fallbacks to minimize expensive inference calls while maintaining flexibility.
Executes browser actions in page context with robust error handling, retry logic, overlay dismissal, and semantic selector resolution (text:, input:, link:, button:). Implements sophisticated click handling that verifies element visibility and covers edge cases like forced navigation. The component is highly reusable across web automation tools and demonstrates expertise in building reliable DOM interaction primitives.