$ show xzf-thu/Pask
This repository contains the LatentNeeds-Bench benchmark and evaluation framework for proactive AI demand detection, as described in the research paper 'Pask: Toward Intent-Aware Proactive Agents with Long-Term Memory'. The project focuses on creating a benchmark to evaluate whether AI models can detect latent user needs in ongoing conversations without explicit prompting. It includes a comprehensive data processing pipeline that transforms raw speech transcripts into annotated benchmark datasets, an evaluation framework to test various LLMs on this benchmark, and tools to generate research paper tables and figures. The system implements the DD-MM-PAS paradigm (Demand Detection, Memory Modeling, Proactive Agent System) with IntentFlow as the specialized demand detection model.
The LLM identified 1 files worth highlighting for reuse:
This module provides a robust wrapper for LLM API calls (OpenRouter/OpenAI) with synchronous and asynchronous interfaces. It implements automatic retries with exponential backoff, JSON output validation with schema hints, and concurrent batch processing with semaphore-based rate limiting. The client handles common failure modes (empty responses, JSON parsing errors) and provides a clean API for LLM interactions. What makes it particularly valuable is the thoughtful separation of concerns - the async batch processing with semaphore control prevents overwhelming API rate limits while maximizing throughput, and the JSON validation with schema hints reduces boilerplate in calling code. Another engineer could extract this file and use it in any LLM-based project with virtually no modifications, as it has zero dependencies on other project-specific code.