This is a mature, actively maintained AI agent skill called 'last30days' that enables agents (Claude Code, Codex, Grok, Gemini CLI, MCP hosts) to research any topic by aggregating content from 20+ social and news platforms (Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, arXiv, Techmeme, Digg, and more) over a rolling 30-day window. The system is primarily a CLI engine (Python) with a companion Go MCP server for Claude Desktop. It employs a sophisticated pipeline: query planning by intent, parallel retrieval from multiple backends (keyless APIs, paid scrapers, CLI tools, LLM-based search), source-specific normalization, engagement-weighted relevance scoring, cross-source clustering, and deterministic fusion (weighted RRF) with per-author caps and diversity floors. Output is rendered as structured JSON, compact text, or self-contained HTML with optional publishing and Atom feeds. The codebase emphasizes security (credential hygiene, no browser cookie reads in unattended mode), determinism (fixture-based evaluation harness with grounding/recency/coherence scores), and graceful degradation (per-source fallback chains, never-raise error envelopes). While classified as an application (it's a skill/plugin for agent hosts), it exhibits strong data engineering traits: source adapters, normalization pipelines, scoring models, and an offline evaluation framework.
This repository is the `lingam` Python package, a comprehensive, research-oriented library for causal discovery from observational data using LiNGAM (Linear Non-Gaussian Acyclic Model) and its numerous extensions. It implements over 20 algorithms, including DirectLiNGAM, ICALiNGAM, VAR-LiNGAM, RCD, CAM-UV, RESIT, LiM, mLiNGAM, GroupLiNGAM, MultiGroup variants, and more. The library provides a consistent API where each algorithm exposes `fit()` on a data matrix, followed by properties like `causal_order_` and `adjacency_matrix_`, and all support bootstrap-based stability analysis via a shared `BootstrapMixin`. Key contributions include handling of latent confounders, time series, missing data, mixed data types, and multiple datasets, alongside utilities for causal effect estimation, visualization, model evaluation, and synthetic data generation. The code is well-structured, uses standard scientific Python dependencies (numpy, scipy, scikit-learn, graphviz), and includes a thorough test suite and extensive Jupyter notebook tutorials.
Local Deep Research is a self-hosted, privacy-focused AI research assistant that performs deep, agentic research using multiple LLMs and web searches. It provides a full-stack web application (FastAPI + JavaScript), a programmatic API, and a Model Context Protocol (MCP) server. The system ingests user queries, decomposes them, executes parallel searches across web and academic sources (SearXNG, Google, arXiv, PubMed, Semantic Scholar, etc.), fetches and extracts content, evaluates evidence, and synthesizes reports with citations. It features per-user encrypted SQLCipher databases, a settings system with environment variable overrides, a RAG library with FAISS vector store, journal quality scoring, news subscriptions, and extensive security controls (egress policy, SSRF protection, credential scrubbing). The codebase is heavily tested with unit, integration, UI, accessibility, and security tests. It is a hybrid of a web application and a research data pipeline, but the dominant character is a production-grade application.
Phuzz is a research-oriented web application fuzzing framework that orchestrates multiple fuzzers (Burp Suite, Wapiti, WFuzz, ZAP) alongside its own custom fuzzer to detect vulnerabilities in web applications. The repository includes a custom Python-based fuzzer with mutation and scoring modules, a crawler for discovering endpoints, a HAR generator for request capture, and Docker-based orchestration for running fuzzers against vulnerable test applications (bWAPP, DVWA, WackoPicko, XVWA, WordPress plugins). It also contains extensive experimental data and analysis scripts from studies on known and zero-day vulnerabilities, particularly targeting WordPress plugins. The architecture is modular, with separate components for crawling, fuzzing, and result analysis, all containerized via Docker Compose. While primarily a research tool, the core fuzzer and instrumentation components are well-structured and could be extracted for reuse in other security testing pipelines.