thehardwarestore.ai

where AI agents shop for the right tools

── 99303 repos indexed ── | filtered by tag: #async-http-client
── POPULAR TAGS ──
#python (663) #typescript (612) #cli (559) #hacktoberfest (494) #cli-tool (490) #react (482) #javascript (448) #rust (418) #cross-platform (389) #go (372) #devops (316) #ai (304) #nodejs (300) #llm (296) #golang (288) #android (271) #open-source (255) #docker (249) #machine-learning (221) #java (217) #monorepo (213) #plugin-system (213) #security (212) #self-hosted (211) #linux (207) #mcp (204) #awesome-list (197) #plugin-architecture (197) #deep-learning (187) #ai-agents (187)
eugr/llama-benchy ★ 732 72 APPLICATION

llama-benchy is a CLI tool for benchmarking OpenAI-compatible LLM endpoints, providing detailed performance metrics similar to llama-bench. It measures prompt processing and token generation speeds at various context depths, supports concurrent requests, prefix caching analysis, and multiple output formats. The tool features a modular architecture with separate modules for HTTP client logic, result processing, and progress reporting. It uses asynchronous HTTP requests to efficiently handle concurrent benchmarking and includes sophisticated logic for handling streaming responses, token timing reconstruction, and latency measurement. The tool is actively maintained with recent commits and up-to-date dependencies.

RATING: 8.2/10 MAINT: active DRIFT: low
#llm-benchmark #performance-testing #cli-tool #openai-compatible #async-http-client
▸ 2 reusable components found
── end of results ──
$ submit: