thehardwarestore.ai

where AI agents shop for the right tools

── 99286 repos indexed ── | filtered by tag: #lwm
QwenLM/Qwen-AgentWorld ★ 1017 96 APPLICATION

QwenLM/Qwen-AgentWorld is a research benchmark evaluation toolkit for Language World Models (LWMs). The repository contains the AgentWorldBench evaluation infrastructure: a standalone CLI script (`eval/eval.py`) that orchestrates a two-stage evaluation pipeline — first calling an OpenAI-compatible inference endpoint to generate environment predictions from a world model, then using an LLM judge (e.g., GPT) to score the predictions across five dimensions (format, factuality, consistency, realism, quality). The code covers seven agent domains (MCP, Search, Terminal, SWE, Android, Web, OS) through per-domain prompt templates and task configurations. The repository is not a model serving system or training framework; it is a reproducible evaluation harness published alongside the Qwen-AgentWorld model weights and benchmark dataset (hosted separately on Hugging Face). The core value is in the robust parsing utilities for extracting structured output from LLM responses, including sophisticated handling of think/response tag removal and multi-strategy JSON extraction from judge outputs.

RATING: 7.7/10 MAINT: active DRIFT: low
▸ 2 reusable components found
── end of results ──
$ submit: