$ show hanlinwenyuan/hlwy-ai-checker
This is a self-contained tool for detecting third-party AI API model substitution by analyzing the statistical fingerprint of random number generation. The backend is a Python proxy server (http.server) that forwards requests to OpenAI/Anthropic endpoints, handles GitHub baseline retrieval with caching, and provides an auto-update mechanism. The frontend is a single HTML file with embedded JavaScript (no bundler) that handles all user interaction: calibration of official models, testing of unknown channels, quick one-click verification against pre-downloaded baselines, and multi-channel comparison. The tool uses cosine similarity and Jensen-Shannon divergence to compare distributions, with special handling for model-specific parameters (max_tokens vs max_completion_tokens). The code is functional and well-organized for a single-developer tool, but lacks modularity (everything in one HTML file, backend in one script) and has no test suite. The baseline data files are precomputed artifacts, not part of a live data pipeline.
The LLM identified 3 files worth highlighting for reuse:
The _post_adaptive method in the ProxyHandler intelligently handles API incompatibility between models that accept different parameter names (max_tokens vs max_completion_tokens). When a 400 error indicates the parameter is unsupported, it automatically switches to the alternative and retries. This is a non-obvious solution to a common pain point when proxying requests across different AI model versions, and could be extracted as a reusable middleware pattern for any API proxy.
The error logging and early-fail-warning system in the frontend classifies API errors (network, auth, rate-limit, parse) with human-readable summaries and actionable suggestions. For example, it distinguishes between 'API Key invalid' (401) and 'model name misspelled' (400), and after 5 consecutive failures displays a large warning with specific troubleshooting steps. This pattern of structured error classification with user guidance could be reused in any application that interacts with external APIs.
The baseline retrieval system uses multiple fallback strategies: first tries raw.githubusercontent (free, unlimited), then GitHub Contents API (subject to rate limits), and finally falls back to a disk cache. It respects GitHub rate limits (60/hr anonymous, 5000/hr with token) and stores cache in ~/.hlwy-ai-checker/ so the tool works offline after initial fetch. This pattern of tiered data fetching with transparent fallback is applicable to any tool that consumes rate-limited external APIs.