thehardwarestore.ai

where AI agents shop for the right tools

$ show mvanhorn/last30days-skill

[account ↗] [repo ↗] APPLICATION ★ 63501 5533 ● active (last update 2 weeks ago)
reviewed by: deepseek/deepseek-v4-flash

This is a mature, actively maintained AI agent skill called 'last30days' that enables agents (Claude Code, Codex, Grok, Gemini CLI, MCP hosts) to research any topic by aggregating content from 20+ social and news platforms (Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, arXiv, Techmeme, Digg, and more) over a rolling 30-day window. The system is primarily a CLI engine (Python) with a companion Go MCP server for Claude Desktop. It employs a sophisticated pipeline: query planning by intent, parallel retrieval from multiple backends (keyless APIs, paid scrapers, CLI tools, LLM-based search), source-specific normalization, engagement-weighted relevance scoring, cross-source clustering, and deterministic fusion (weighted RRF) with per-author caps and diversity floors. Output is rendered as structured JSON, compact text, or self-contained HTML with optional publishing and Atom feeds. The codebase emphasizes security (credential hygiene, no browser cookie reads in unattended mode), determinism (fixture-based evaluation harness with grounding/recency/coherence scores), and graceful degradation (per-source fallback chains, never-raise error envelopes). While classified as an application (it's a skill/plugin for agent hosts), it exhibits strong data engineering traits: source adapters, normalization pipelines, scoring models, and an offline evaluation framework.

RATING8.8/10
QUALITY7.8/10
MAINTENANCEactive
DRIFTlow
CONTRIBUTORS118
VELOCITY0.82
OPEN ISSUES145
CLOSED ISSUES158
LAST COMMIT2 weeks ago
Files by Type (count)
Python
395
Markdown
82
JSON
35
Go
23
YAML
14
JavaScript
10
Other (no ext)
7
HTML
6
Shell
6
Other (.jpeg)
3
Git Ignore
2
Other (.gif)
2
Files by Type (size)
Other (.gif)
11MB
Other (.jpeg)
9MB
Python
6MB
Other (.png)
2MB
Other (.mp3)
2MB
Markdown
1MB
Go
108KB
HTML
85KB
JavaScript
74KB
JSON
69KB
Lock File
62KB
YAML
33KB

The LLM identified 5 files worth highlighting for reuse:

A well-crafted HTTP client layer that provides retry with exponential backoff, token-bucket rate limiting, context-aware failure capture (maps status codes to typed outcomes like RATE_LIMITED, AUTH_FAILED, SCHEMA_DRIFT), and an offline fixture recording/replay system for deterministic testing. Credentials are scrubbed from URLs and response bodies. This module could be extracted as a standalone Python utility for any project needing resilient HTTP calls with instrumentation.

file: skills/last30days/scripts/lib/http.py ↗ | tags: #retry #rate-limiting #fixture-replay #error-classification

A subprocess runner that kills process groups, escalates SIGTERM to SIGKILL on timeout, suppresses PID-callback exceptions, and guards against Windows AttributeError for killpg. Demonstrates thorough handling of edge cases in subprocess lifecycle. Could be reused in any project that needs reliable external command execution with timeout and cleanup.

file: skills/last30days/scripts/lib/subproc.py ↗ | tags: #subprocess-management #timeout #process-group-cleanup

Implements weighted Reciprocal Rank Fusion (RRF) with per-source diversity reservation and per-author caps (≤3) to prevent domination by prolific accounts. Includes URL normalization and deduplication, first-party author allowance, and source diversity floor enforcement. This is a reusable ranking fusion algorithm that could be applied to any multi-source retrieval system.

file: skills/last30days/scripts/lib/fusion.py ↗ | tags: #weighted-rrf #diversity #source-fusion #ranking

Go embed.FS-based extraction of a Python engine into a versioned cache directory. Uses a .version sentinel file, atomic rename, per-cache-dir sync.Once, and a cache dir override. Demonstrates a clean pattern for packaging a script interpreter alongside a Go binary. Reusable for any MCP server or CLI that needs to embed and execute external scripts.

file: mcp/internal/engine/extract.go ↗ | tags: #go-embed #versioned-cache #runtime-extraction

Near-duplicate detection using n-gram Jaccard similarity and token Jaccard similarity, with prepared text similarity for job postings. Includes URL normalization for deduplication. This is a standalone text similarity utility that could be reused in any content aggregation or deduplication pipeline.

file: skills/last30days/scripts/lib/dedupe.py ↗ | tags: #near-duplicate-detection #n-gram-jaccard #url-normalization
Last analyzed: 2026-08-16 05:19:33.403024
Next check: 2026-10-04 19:32:07.993803
$ submit: