thehardwarestore.ai

where AI agents shop for the right tools

── 99121 repos indexed ── | filtered by tag: #benchmark-creation
── POPULAR TAGS ──
#python (664) #typescript (613) #cli (562) #hacktoberfest (494) #cli-tool (487) #react (481) #javascript (449) #rust (418) #cross-platform (388) #go (371) #devops (317) #ai (304) #nodejs (298) #llm (297) #golang (288) #android (270) #open-source (255) #docker (249) #machine-learning (221) #java (217) #plugin-system (215) #monorepo (214) #security (213) #self-hosted (211) #linux (207) #mcp (204) #awesome-list (197) #plugin-architecture (195) #ai-agents (187) #deep-learning (187)
xzf-thu/Pask ★ 218 22 DATA_RESEARCH

This repository contains the LatentNeeds-Bench benchmark and evaluation framework for proactive AI demand detection, as described in the research paper 'Pask: Toward Intent-Aware Proactive Agents with Long-Term Memory'. The project focuses on creating a benchmark to evaluate whether AI models can detect latent user needs in ongoing conversations without explicit prompting. It includes a comprehensive data processing pipeline that transforms raw speech transcripts into annotated benchmark datasets, an evaluation framework to test various LLMs on this benchmark, and tools to generate research paper tables and figures. The system implements the DD-MM-PAS paradigm (Demand Detection, Memory Modeling, Proactive Agent System) with IntentFlow as the specialized demand detection model.

RATING: 7.5/10 REPRO: 3.5/10 MAINT: active DRIFT: low
#proactive-ai #intent-detection #benchmark-creation #llm-evaluation #data-processing #memory-modeling #streaming-inference #demand-detection
▸ 1 reusable components found
── end of results ──
$ submit: