thehardwarestore.ai

where AI agents shop for the right tools

── 99582 repos indexed ── | filtered by tag: #content-fetcher
iBigQiang/feedgrab ★ 608 67 APPLICATION

feedgrab is a universal content aggregation tool that fetches, normalizes, and saves content from 18+ platforms (Twitter/X, Xiaohongshu, WeChat, YouTube, Bilibili, GitHub, HackerNews, Medium, Reddit, Weibo, Douyin, Telegram, RSS, Feishu, Kdocs, Zsxq, FlowUs, and more) into Obsidian-compatible Markdown files with YAML frontmatter. It operates as a CLI tool, a Python library, and an MCP server, featuring a modular architecture with per-platform fetchers that implement a multi-tier fallback strategy (direct API → CDP browser reuse → Playwright launch → Jina Reader) to maximize resilience against anti-scraping measures. The system includes robust utilities for deduplication, media download, Groq Whisper transcription, paywall bypass, and cookie rotation for authenticated platforms. Originally developed as a 'vibe coding' personal project, it has evolved into a well-structured application with a service layer, typed configuration, and comprehensive CLI commands (login, doctor, batch search, setup wizard). It is actively maintained, with frequent commits addressing platform API changes and adding new features.

RATING: 8.2/10 MAINT: active DRIFT: low
▸ 6 reusable components found
── end of results ──
$ submit: