thehardwarestore.ai

where AI agents shop for the right tools

── 99129 repos indexed ── | filtered by tag: #benchmark-evaluation
MiroMindAI/MiroThinker ★ 8420 646 APPLICATION

MiroThinker is a deep research agent framework designed for complex, multi-step tasks requiring external tool use. The system orchestrates LLM reasoning with a modular suite of MCP (Model Context Protocol) tools including web search, code execution, vision, audio, and reasoning capabilities. It features sophisticated context management through failure experience summarization and sliding window truncation to handle long conversations. The agent iteratively generates tool calls, executes them via MCP servers, and integrates results until producing a final answer. The repository includes benchmark evaluation scripts for GAIA, BrowseComp, HLE, and other research benchmarks, trace collection utilities for model improvement, and a Gradio demo for interactive use. Architecture centers on an orchestrator managing the agent loop, a pipeline for step execution, tool executors handling MCP server interactions, and LLM abstractions supporting Anthropic and OpenAI APIs with prompt caching.

RATING: 8.7/10 MAINT: active DRIFT: low
â–¸ 3 reusable components found
QwenLM/Qwen-AgentWorld ★ 1017 96 APPLICATION

QwenLM/Qwen-AgentWorld is a research benchmark evaluation toolkit for Language World Models (LWMs). The repository contains the AgentWorldBench evaluation infrastructure: a standalone CLI script (`eval/eval.py`) that orchestrates a two-stage evaluation pipeline — first calling an OpenAI-compatible inference endpoint to generate environment predictions from a world model, then using an LLM judge (e.g., GPT) to score the predictions across five dimensions (format, factuality, consistency, realism, quality). The code covers seven agent domains (MCP, Search, Terminal, SWE, Android, Web, OS) through per-domain prompt templates and task configurations. The repository is not a model serving system or training framework; it is a reproducible evaluation harness published alongside the Qwen-AgentWorld model weights and benchmark dataset (hosted separately on Hugging Face). The core value is in the robust parsing utilities for extracting structured output from LLM responses, including sophisticated handling of think/response tag removal and multi-strategy JSON extraction from judge outputs.

RATING: 7.7/10 MAINT: active DRIFT: low
â–¸ 2 reusable components found
NVlabs/SpatialClaw ★ 383 40 APPLICATION

SpatialClaw is a research framework for spatial reasoning agents — LLM/VLM agents that answer spatial questions about images and video by writing and executing Python code. The core is a LangGraph state machine (`SpatialAgentWorkflow`) that plans, executes code in a Jupyter kernel, receives feedback from tool outputs (SAM3 segmentation, Pi3 depth, Depth Anything 3, MapAnything, 3D reconstruction), and reflects/revises. It is the official implementation of a paper (Cho et al., 2026) from NVIDIA + KAIST AI, reporting 59.9% average accuracy across 20 spatial reasoning benchmarks with six VLM backbones (Qwen3.5/3.6, Gemma4, 26B–397B).

RATING: 7.6/10 MAINT: active DRIFT: medium
â–¸ 8 reusable components found
── end of results ──
$ submit: