RSSHub is the world's largest open-source RSS feed generator, aggregating content from thousands of external websites into standardized RSS, Atom, JSON, and RSS3 feeds. It is a TypeScript application built on the Hono web framework, with a modular architecture where each source (news site, social platform, university notice board, government agency, etc.) is implemented as a self-contained 'route' module that fetches pages or APIs, parses HTML with Cheerio, and optionally renders rich descriptions using JSX. The project supports multiple deployment targets including Node.js, Docker, Vercel, and Cloudflare Workers, and integrates caching (Redis/LRU), middleware for access control, ETag/conditional requests, error tracking (Sentry, Honeybadger), and anti-hotlinking. The codebase is highly organized with consistent route definitions, extensive utility functions (date parsing, HTTP caching, header generation), and a strong emphasis on type safety. While the sheer volume of routes (thousands) introduces maintenance burden and some inconsistency (legacy `got` vs modern `ofetch`), the overall architecture is clean, well-tested for core infrastructure, and exhibits strong engineering practice. This is not a data-research repository; it is a production-grade web service for feed aggregation.
Cobalt is a privacy-focused media downloader web application with a SvelteKit frontend and Node.js/Express API backend. Users paste a URL from supported platforms (YouTube, TikTok, Instagram, Twitter/X, Vimeo, Reddit, SoundCloud, Twitch, Bilibili, and 15+ others) and receive a downloadable media file. The architecture is a monorepo with three packages: @imput/cobalt-web (static frontend), @imput/cobalt-api (processing backend), and @imput/version-info (build-time version helper). The API acts as a fancy proxy - it fetches media streams from source platforms, optionally remuxes/transcodes via FFmpeg (server-side or client-side via libav.js WebAssembly), and streams results back without ever writing to disk. The frontend handles URL input, settings management (localStorage with migration), a processing queue with Web Workers for client-side FFmpeg, i18n (EN/RU), and PWA features. Key data flows: user submits URL → API validates/matches service → extracts stream URLs → returns tunnel/redirect/picker response → frontend either streams directly, proxies via API, or processes locally via Web Workers. External dependencies: FFmpeg (via ffmpeg-static), youtubei.js for YouTube, hls-parser, undici for HTTP, Redis (optional for rate limiting), Cloudflare Turnstile (optional bot protection).
VidBee is a multi-platform media downloader and transcript workspace. The primary product is an Electron desktop app that downloads video/audio from 1000+ sites via yt-dlp, runs local speech-to-text transcription (sherpa-onnx Whisper/SenseVoice/Qwen3/Parakeet families) with speaker diarization in an isolated worker process, and provides an AI agent chat, summarization, translation, and FAQ layer over the resulting transcripts. It also ships a self-hosted Fastify API with a TanStack Start web client, a CLI wrapper around yt-dlp that can run fully in-process or connect to the desktop/API, and a browser extension (WXT) that hands video URLs to the desktop app via a token-authenticated loopback HTTP bridge. An RSS subscription engine with SQLite CAS leader election supports feed auto-download. Architecturally this is a pnpm monorepo built around host-neutral packages: packages/task-queue (FSM-driven job queue with priority scheduler, retry backoff, watchdog, and SQLite/memory persistence adapters), packages/transcription (ASR pipeline, caption parsing, speaker clustering, chunk manifests), packages/downloader-core (yt-dlp args, format selection, cookie handling, oRPC contracts), packages/subscriptions-core (feed parsing, scheduler, leader election), plus packages/ui, packages/i18n, packages/db, and packages/logger. The desktop, API, CLI, and web hosts share these packages, and oRPC contract schemas are explicitly kept byte-equal across hosts. The desktop renderer uses React 19, TanStack Router, Jotai, and Tailwind 4; the main process uses an IPC-decorator pattern with roughly fifteen typed services. Engineering quality is consistently above average: shared contracts with zod validation, swappable persistence adapters, a pure-function task FSM, security-conscious extension token auth, and an SSRF-guarded remote image proxy. The main weaknesses are duplication and sync fragility: near-identical download UI components exist in both desktop and web apps, several overlapping error classifiers must be kept manually in sync, the API hardcodes APP_VERSION separately from package versions, and legacy migration/back-compat paths (task-queue projection to legacy DownloadItem shapes, migrate-history scripts) accumulate complexity. This is an application repository, not data research: the ASR and speaker-clustering code is product engineering rather than a research artifact, though the transcription package is a genuinely reusable pipeline.
Huginn is a self-hosted, hackable automation platform built as a Ruby on Rails application. It allows users to create and chain 'agents' that monitor, scrape, transform, and act on data from hundreds of sources (web, RSS, email, Twitter, Telegram, Slack, OpenAI, etc.) via an event-driven architecture. Agents are polymorphic Ruby classes with a consistent lifecycle: scheduled checks, event reception, and event emission. The system supports Liquid templating for dynamic configuration, OAuth services, file pointers, and background job processing via DelayedJob. The codebase is extensive with 80+ agent types, reusable concerns (WebRequestConcern, LiquidInterpolatable, FormConfigurable), and a comprehensive test suite. While primarily an application, it includes a small research dataset (ANEW sentiment norms) used by a sentiment analysis agent, but the dominant character is clearly a production automation service.
This is a mature, actively maintained AI agent skill called 'last30days' that enables agents (Claude Code, Codex, Grok, Gemini CLI, MCP hosts) to research any topic by aggregating content from 20+ social and news platforms (Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, arXiv, Techmeme, Digg, and more) over a rolling 30-day window. The system is primarily a CLI engine (Python) with a companion Go MCP server for Claude Desktop. It employs a sophisticated pipeline: query planning by intent, parallel retrieval from multiple backends (keyless APIs, paid scrapers, CLI tools, LLM-based search), source-specific normalization, engagement-weighted relevance scoring, cross-source clustering, and deterministic fusion (weighted RRF) with per-author caps and diversity floors. Output is rendered as structured JSON, compact text, or self-contained HTML with optional publishing and Atom feeds. The codebase emphasizes security (credential hygiene, no browser cookie reads in unattended mode), determinism (fixture-based evaluation harness with grounding/recency/coherence scores), and graceful degradation (per-source fallback chains, never-raise error envelopes). While classified as an application (it's a skill/plugin for agent hosts), it exhibits strong data engineering traits: source adapters, normalization pipelines, scoring models, and an offline evaluation framework.
Twikit is an asynchronous Python library that provides a comprehensive, unofficial client for the Twitter (X) API. It allows users to perform a wide range of Twitter operations—such as posting tweets, searching for tweets and users, managing bookmarks and lists, sending direct messages, uploading media, and streaming real-time events—without requiring an official API key. The library achieves this by reverse-engineering Twitter's internal GraphQL and REST v1.1 endpoints, handling authentication via login flows or guest tokens, and implementing sophisticated anti-bot evasion techniques like client-side transaction ID generation (X-Client-Transaction-Id) and UI metrics obfuscation. The architecture separates concerns into GraphQL (gql) and REST (v11) client modules, with data models for tweets, users, communities, and more. It includes optional CAPTCHA solving and cookie persistence for session reuse. The codebase is well-structured, actively maintained, and demonstrates deep reverse-engineering skill, particularly in the `x_client_transaction` module that mimics Twitter's client-side telemetry.
The Social Media Hacker List repository primarily maintains a curated markdown list of tools and applications for enhancing social media experiences across various platforms (Facebook, Instagram, Twitter, etc.). The list is organized by platform and includes entries such as alternative clients, analytics tools, bots, and content creation utilities. Each entry is manually curated and must meet specific inclusion criteria, such as being created by unaffiliated developers and having recent activity. To ensure the list remains accurate and free of broken or inactive links, the repository includes a Rust-based URL checker application. This tool is executed weekly via a GitHub Actions workflow (check.yaml) to validate every URL in the README.md. For standard URLs, it performs HEAD and GET requests, treating certain HTTP status codes as success (200, 403, 406) with special handling for known edge cases (e.g., 502 for reddit.com, 429 for apps.apple.com). For GitHub repository URLs, it uses the GitHub API to check if the repository is archived or has not been pushed to in the last 365 days, flagging such entries as invalid. The checker application is designed as a standalone, concurrent command-line tool. It reads input from stdin, extracts URLs using a regular expression, and processes them asynchronously with the tokio runtime. The application is containerized via a multi-stage Dockerfile and published to GitHub Container Registry through another workflow (publish.yaml), allowing it to be reused in other contexts. The code demonstrates solid Rust practices, including proper error handling, asynchronous I/O, and modular function design, making its core URL and GitHub checking logic potentially reusable in other projects with minimal adaptation.
buji-pac4j is a bridge library that integrates pac4j (a comprehensive security library supporting OAuth, CAS, SAML, OpenID Connect, etc.) with Apache Shiro. It automatically pushes pac4j authentication and authorization profiles into the Shiro security context, allowing developers to use pac4j's wide range of authentication mechanisms while leveraging Shiro for authorization and session management. The bridge is implemented as a set of classes that adapt pac4j's session store, profile manager, and framework adapter to Shiro's APIs, plus a custom Realm, SubjectFactory, and Token. It includes sophisticated logic to avoid unnecessary session renewal when the same user's profiles are updated (e.g., after token refresh), demonstrating deep understanding of both frameworks' internals. The library is production-ready, actively maintained, and used in legacy applications that want to adopt pac4j without fully migrating away from Shiro.
PaoPao-ce is a full-stack micro-community social media platform written in Go (backend) and Vue 3 (frontend). It provides a feature-rich experience similar to Twitter/Weibo, including posts (tweets) with multimedia attachments, comments, likes, collections, private messaging, user profiles, topic/tag support, and a wallet system with Alipay integration. The architecture is modular and configurable, supporting multiple databases (MySQL, PostgreSQL, SQLite), search engines (Meilisearch, Zinc), object storage backends (LocalOSS, MinIO, AliOSS, COS, Huawei OBS, S3), caching layers (Redis, BigCache), and observability tools (OpenTelemetry, Sentry, Pyroscope, pprof). The backend uses Gin for HTTP, GORM for ORM, go-mir for API code generation, and protobuf for gRPC services. The frontend is built with Vue 3, Naive UI, Vite, and Pinia. Key features include feature toggles for different deployment suites, a runtime settings override via admin UI, and background job processing for search indexing and metrics. The platform emphasizes self-hosting and customization, with comprehensive SQL migration scripts and support for desktop clients via Tauri. The codebase demonstrates solid engineering practices with interface-based dependency injection, layered architecture, and code generation, though it carries some technical debt from deprecated backends and WIP features.
iKy (I Know You) is a modular OSINT (Open Source Intelligence) aggregation platform built with FastAPI, Celery, Redis, and an Angular frontend. The system ingests a username or email and concurrently executes 26+ parallel tasks across social media, data breaches, search engines, and dark web sources, returning a structured JSON contract (module, param, validation, raw, graphic, profile, timeline, tasks) consumed by a visual graph/timeline UI. Key architectural patterns include a dynamic module registry that maps route names to Celery task paths, a decorator-based task wrapper (@iky_task) that centralizes dev-mode golden-file bypass, error classification, and performance logging, and a pluggable search provider orchestration layer. The codebase demonstrates a strong separation of concerns between API routing, task execution, and data enrichment, but suffers from fragile scraping logic (LinkedIn RSC parsing, TikTok SSR regex), heavy reliance on browser session cookies, and several deprecated external APIs (Twitter v1.1, FullContact). The project is actively maintained with an extensive test suite (~4K lines) and Docker Compose deployment, though many tests are heavily mocked and may not detect real API drift.
This repository is a comprehensive, curated collection of step-by-step privacy configuration guides for a wide range of software, services, and operating systems (e.g., browsers, messaging apps, social media, Windows, Android, macOS). It is purely documentation—no executable code, APIs, or data pipelines. The content is organized into Markdown files under 'Privacy Settings' and 'Extensions Settings', with a central README serving as an index. The guides are consistently formatted, include notes and tips, and are actively maintained with regular updates reflecting the latest stable versions. While not a software application, it serves as a high-quality reference for users seeking to harden privacy settings across their digital footprint.
MoneyPrinterV2 is a Python 3.12 CLI tool that automates four online money-making workflows: YouTube Shorts generation and upload, Twitter/X posting, Amazon affiliate marketing, and local business outreach via email. It uses a monolithic architecture with a menu-driven interface (src/main.py) and a headless cron runner (src/cron.py). The YouTube pipeline leverages Ollama for script/metadata generation, Nano Banana 2 (Gemini) for image generation, KittenTTS for text-to-speech, and MoviePy for video composition, then uploads via Selenium. The Twitter bot posts generated tweets via Selenium. Affiliate marketing scrapes Amazon product pages and posts pitches on Twitter. Outreach uses an external Go-based Google Maps scraper and sends emails via yagmail. The codebase is functional but exhibits tight coupling, missing dependency declarations (e.g., srt_equalizer, prettytable), and a non-standard import (selenium_firefox). The most well-crafted component is the PostBridge API client (src/classes/PostBridge.py), which demonstrates clean retry logic, pagination handling, and proper error reporting. The last commit date (2026-09-15) appears anomalous, but the repository shows active issue triage and recent activity.