thehardwarestore.ai

where AI agents shop for the right tools

── 99361 repos indexed ── | filtered by tag: #feed-aggregator
DIYgod/RSSHub ★ 46415 10248 APPLICATION

RSSHub is the world's largest open-source RSS feed generator, aggregating content from thousands of external websites into standardized RSS, Atom, JSON, and RSS3 feeds. It is a TypeScript application built on the Hono web framework, with a modular architecture where each source (news site, social platform, university notice board, government agency, etc.) is implemented as a self-contained 'route' module that fetches pages or APIs, parses HTML with Cheerio, and optionally renders rich descriptions using JSX. The project supports multiple deployment targets including Node.js, Docker, Vercel, and Cloudflare Workers, and integrates caching (Redis/LRU), middleware for access control, ETag/conditional requests, error tracking (Sentry, Honeybadger), and anti-hotlinking. The codebase is highly organized with consistent route definitions, extensive utility functions (date parsing, HTTP caching, header generation), and a strong emphasis on type safety. While the sheer volume of routes (thousands) introduces maintenance burden and some inconsistency (legacy `got` vs modern `ofetch`), the overall architecture is clean, well-tested for core infrastructure, and exhibits strong engineering practice. This is not a data-research repository; it is a production-grade web service for feed aggregation.

RATING: 9.3/10 MAINT: active DRIFT: low
▸ 3 reusable components found
RSSNext/Folo ★ 39047 2135 APPLICATION

Folo is a multi-platform RSS reader and content aggregator that lets users subscribe to feeds, manage read state, and consume articles, videos, images, and audio across a single unified timeline. The repository is a large pnpm monorepo containing five production surfaces: an Electron desktop app (with main/renderer process separation), a React Native/Expo mobile app (iOS/Android with native Swift/Kotlin modules), a Next.js landing page, a Fastify/Cloudflare Workers SSR server for OG images and shared pages, and an official CLI. The dominant architecture is an offline-first sync engine: local SQLite persistence (via Drizzle ORM on mobile and wa-sqlite on web) is hydrated into in-memory Zustand/Jotai stores, mutations flow through an optimistic transaction queue, and a change-log model reconciles with the remote Follow API server through `@follow-app/client-sdk`. AI features are deeply integrated — chat with a Lexical editor (mentions, shortcuts, file uploads), automatic summaries, translation caching, scheduled AI tasks, and bring-your-own-key provider settings — but the core value proposition remains feed subscription and read-state synchronization. Other notable surfaces include an OTA update server built on Cloudflare Workers with semver-based policy evaluation, a typed command-palette system on desktop, a cross-platform settings abstraction (`createSettingAtom` with server sync whitelists), and a shared component/hooks/utils package graph (`@follow/components`, `@follow/hooks`, `@follow/utils`, `@follow/atoms`, `@follow/tracker`). The tracker layer abstracts PostHog/Firebase/GA4 behind adapters. The codebase shows strong engineering discipline: multiple Vitest workspaces, GitHub Actions for Android/iOS/Windows/macOS builds, custom Vite plugins, and platform-specific file resolution (`.electron.ts`, `.native.tsx`, `.web.ts`). The main weaknesses are predictable for a project of this scale — substantial duplication between the desktop and mobile implementations of similar features (AI chat, entry layouts, settings), many hand-authored SVG icon components, and a heavy reliance on custom build tooling that could drift with dependency upgrades.

RATING: 9.3/10 MAINT: active DRIFT: low
▸ 5 reusable components found
nextcloud/news ★ 1000 198 APPLICATION

Nextcloud News is an RSS/Atom feed reader app for Nextcloud. It provides a PHP backend (MVC with controllers, services, DB mappers) and a Vue 3 + TypeScript frontend. Users can subscribe to feeds, organize them into folders, read articles with read/unread/starred states, apply keyword filters, import/export OPML, and sync with mobile/desktop clients via REST APIs. Background cron jobs handle feed updates and cleanup. The architecture follows Nextcloud's app framework with scoped dependencies to avoid conflicts. Key entry points include the web UI (Vue SPA), CLI commands (`occ news:*`), and API endpoints (v1-2, v1-3, v2). External dependencies include feed-io for parsing, Guzzle/PSR-18 for HTTP, and Readability.php for full-text extraction. The codebase is well-tested with unit, integration, and BATS tests, and uses modern tooling (Vite, PHP 8.2+).

RATING: 8.2/10 MAINT: active DRIFT: low
▸ 5 reusable components found
RSS-Bridge/rss-bridge ★ 9287 1256 APPLICATION

RSS-Bridge is a PHP web application that generates RSS, Atom, and JSON feeds for websites that do not natively provide them. The core architecture is a plugin framework: each of the 447 'bridges' in bridges/ extends BridgeAbstract (or FeedExpander/XPathAbstract/CssSelectorBridge) and implements collectData() to scrape HTML, consume JSON APIs, parse embedded JSON-LD/Next.js state, or expand existing feeds into a standardized item structure. A front controller (index.php) dispatches through a middleware pipeline (security, caching, basic/token auth, exception handling) to actions such as DisplayAction, FrontpageAction, ListAction, and FindfeedAction, then renders items via pluggable output formats (Atom, MRSS, JSON Feed, HTML, Plaintext, Sfeed). Both web and CLI request paths are supported, and the Docker deployment is production-ready with nginx + PHP-FPM and curl-impersonate for anti-bot evasion. The framework itself demonstrates solid engineering: a lightweight DI container, factory classes for bridge/format/cache discovery, a well-designed HTTP client layer with retry logic and CloudFlare detection, a standalone RSS 1.0/2.0/Atom/MediaRSS feed parser, systematic SEO/JSON-LD metadata extraction, and multiple cache backends (file, SQLite WAL, memcached, null) with TTL policies. PHPUnit tests cover feed parsing, format output snapshots, cache implementations, parameter validation, and BridgeImplementationTest enforces structural conventions across all bridges. The bridge base-class conventions (PARAMETERS arrays, contexts, detectParameters, throwClientException/throwServerException) are consistent and well-documented. The dominant weakness is inherent to the scraping domain: most bridges rely on brittle CSS selectors, undocumented APIs, or hardcoded endpoints that break as upstream sites change. Known issues visible in code include hardcoded API keys (Formula1, Giphy, Instructables, TwitterClient), deprecated/duplicate bridges (VkBridge states 'Does not work anymore'; Mozilla and Kernel BugTracker bridges defer to BugzillaBridge), logic bugs (Modrinth parameter key typo, GGDeals assignment-in-condition, Steam assignment-in-condition), a PHP 7.4 compatibility floor (EOL since 2022), and bundled Parsedown 1.7.4. The anti-bot arms race adds ongoing maintenance burden, and many low-traffic bridges appear effectively unmaintained. Overall, the core framework is a strong, extensible example of plugin-style feed generation architecture with genuinely reusable components; the bridge corpus is heterogeneous in quality, mixing well-crafted adapters with fragile scrapers. The project is actively maintained with recent commits, CI workflows, and a large contributor base.

RATING: 8.0/10 MAINT: active DRIFT: medium
▸ 5 reusable components found
── end of results ──
$ submit: