$ show VectorlyApp/bluebox
Bluebox is a library and agent framework for reverse engineering web applications by monitoring browser interactions via Chrome DevTools Protocol (CDP). It captures network requests, DOM snapshots, storage changes, and user interactions into JSONL files, then uses LLM-driven agents to discover reusable automation routines (parameterized workflows) that can navigate, fetch, click, type, and extract structured data from live websites. The system includes a conversational agent (BlueBoxAgent) that searches a routine index and falls back to an AI browser agent, a GuideAgent for interactive routine creation, and a PrincipalInvestigator orchestrator that runs parallel specialist agents (network, DOM, storage, UI) to build routine catalogs from captured sessions. Architecturally, Bluebox is a modular Python application with clear separation: CDP monitors and session management, Pydantic data models for routines and events, abstract base classes for agents and data loaders, LLM vendor clients (OpenAI, Anthropic), a workspace abstraction for artifact management, and CLI/HTTP entry points. The SDK provides high-level methods (monitor_browser, discover_routine, execute_routine) and the package ships with example routines for real-world sites (Amtrak, Polymarket, Massachusetts corporate search). While primarily an application/library, Bluebox has a strong data pipeline character: it ingests raw CDP events, transforms them through data loaders and vectorstores, and produces structured routine definitions. The benchmark framework evaluates routine quality with deterministic and LLM-based tests. The codebase demonstrates high craftsmanship with reusable patterns (agent tool decorator, workspace abstraction, sandboxed execution, data loader interface) and extensive unit tests, though some specialist agents are marked as beta and the orchestration pipeline is complex.
The LLM identified 4 files worth highlighting for reuse:
Provides a reusable base class for LLM-powered agents with an @agent_tool decorator that auto-generates JSON Schema from method signatures and docstrings, supports availability gating, persistence modes, and token optimization. Includes an autonomous run loop with configurable iterations, finalize gating, and output schema enforcement. This pattern can be extracted to build any tool-using agent with minimal modification.
Implements a file-based workspace with strict directory layout (raw/, output/, context/, meta/, scratch/), monotonic artifact IDs, hardlink-based input mounting, path traversal prevention, and snapshot/diff utilities. Provides manifest persistence and content-type detection. This abstraction can be reused in any agent system that needs isolated, provenance-tracked file management.
Provides sandboxed Python execution with multiple isolation backends (lambda, docker, blocklist) and static pattern analysis for dangerous imports/patterns. Includes scoped file I/O via a custom open() wrapper and workaround hints for blocked modules. This component can be extracted and reused in any application that needs to safely execute AI-generated or untrusted Python code.
Manages a Chrome DevTools Protocol session with async command sequencing, monitor creation, domain enabling, and event routing to specialized monitors (network, storage, DOM, interaction). Provides sequence-number tracking, idempotent domain enabling, and aggregated monitor summaries. This pattern can be reused for any CDP-based browser automation or monitoring tool.