This repository is a curated collection of over 3400 SVG icons for popular brands, maintained by the Simple Icons project. It is primarily a data repository, with the core assets being the SVG files in `icons/` and the structured metadata in `data/simple-icons.json`. The JSON schema defines brand title, hex color, source URL, guidelines, license, and aliases (including duplicates, localizations, and old names). The repository includes a comprehensive toolchain for managing and validating the icon set: scripts for adding/removing icons, formatting data, and running lint checks (custom linter `ourlint.js`, SVG linting via `svglint`, and JSON schema validation). A JavaScript SDK (`sdk.mjs`) provides utilities for accessing icon data, slug generation, and color normalization. The project is published as an npm package and also served via a CDN. While the primary purpose is data curation, the repository exhibits strong software engineering practices with automated CI/CD (GitHub Actions for release, label sync, issue management), pre-commit hooks, and extensive tests that validate every icon's properties. It is a hybrid of a data collection and a well-engineered toolset for maintaining that collection.
XIU2/TrackersListCollection is a curated, daily-updated collection of BitTorrent tracker URLs. It provides several pre-filtered lists (best, all, HTTP-only, no-HTTP) in both plain text and Aria2-compatible formats, served via CDN. The core of the repository is a GitHub Actions workflow (`update-trackers.yml`) that automatically fetches trackers from multiple public sources, normalizes and deduplicates them, applies a blacklist, validates DNS resolution, and performs TCP connectivity checks on HTTP/HTTPS trackers. The resulting lists are committed back to the repository and mirrored to Bitbucket. While the repo is essentially a static data repository with no application logic, the CI pipeline demonstrates a clean, modular ETL pattern using bash and standard Unix tools. The primary value is the curated tracker data itself, which is widely used in BitTorrent clients to improve download speeds.
Jitsu is an open-source event data platform (alternative to Segment) that ingests events from websites, apps, and servers, transforms them via user-defined functions (UDFs) and built-in destination functions, and delivers them to data warehouses (ClickHouse, BigQuery, Snowflake, Redshift, Postgres, S3, GCS) and SaaS tools (Amplitude, PostHog, HubSpot, etc.). The architecture is multi-service: a Go-based bulker handles ingestion and warehouse delivery, a TypeScript-based rotor executes event transformations and profile building, a Next.js console provides the management UI and API, and a CLI tool supports project management and function deployment. The data flow is: HTTP ingest → Kafka → rotor (functions) → bulker (SQL/file destinations) → ClickHouse/PostgreSQL for metrics and logs. External dependencies include Kafka, PostgreSQL, ClickHouse, Redis, MongoDB, and Google Cloud Storage. The console supports multi-tenant workspaces, authentication (Firebase, OIDC, NextAuth), billing, audit logging, rate limiting, and an MCP server for AI tool integration. The codebase is a monorepo with pnpm workspaces and Go modules, featuring extensive test suites (unit, integration with testcontainers, e2e) and CI/CD pipelines. It demonstrates strong engineering practices with reusable libraries (juava, jitsubase), custom high-performance JSON parsers, and well-abstracted repository and configuration patterns. However, there is some technical drift: duplicate JSON libraries (jsoniter and jsonorder), commented-out code, and ESLint disable comments in admin routes. Overall, it is a mature, actively maintained platform suitable for production use.
MediaCrawler is a multi-platform social media data collection tool targeting Chinese platforms: Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Baidu Tieba, and Zhihu. It uses Playwright browser automation to log in (QR code, phone, cookie) and then scrapes posts, comments, and media files based on search keywords, specific post IDs, or creator IDs. The architecture is a monolith with modular platform-specific crawlers, a shared abstract base layer, and a FastAPI backend that provides a WebUI for visual control. Data can be stored in multiple formats (CSV, JSON, JSONL, SQLite, MySQL, PostgreSQL, MongoDB, Excel). Additional features include a robust media downloader with DASH support, proxy rotation, word cloud generation, and privacy-focused anonymization. The codebase demonstrates solid engineering with design patterns (factory, abstract base, mixin), good test coverage, and a clean separation of concerns, though there is some duplication across platform implementations.
This is a CLI-based web scraper for Sina Weibo (微博) user data and posts. It scrapes user profiles, weibo contents, images, and videos from the mobile version of Weibo (weibo.cn) using a combination of synchronous (requests, lxml) and asynchronous (aiohttp) HTTP requests. The architecture is a pipeline: parsers extract structured data from HTML, downloaders asynchronously fetch media files, and writers persist results to multiple formats (CSV, TXT, JSON, MySQL, MongoDB, SQLite, Kafka, HTTP POST). The spider is driven by a config file (config.json) that controls user id lists, date ranges, filtering, and download options. It supports retry logic, random delays, and global wait periods to avoid rate limiting. External dependencies include requests, lxml, aiohttp, browser_cookie3, and absl-py; optional dependencies add database and message queue support. The codebase is modular with clear separation of concerns, includes unit tests with mocked HTML fixtures, and demonstrates solid engineering practices for a data collection tool.
This repository is a curated dataset of NFC tag dumps (`.nfc` files) from Toniebox audio figures, organized by language (English, French, German) and character series. The primary purpose is to enable emulation of Tonie figures on Flipper Zero devices using the official firmware's SLIX-L chip support. The dataset includes over 850 dumps, each containing the full memory image of an ISO15693 SLIX tag (UID, data content, passwords). Supporting scripts provide validation (`validate_files.sh`), automated directory listing generation (`build_directories.sh`), and a GUI tool (`figures_name_tool.pyw`) that uses upstream metadata from `tonies-json` to help contributors correctly place new dumps. The repository is part of the Toniebox reverse-engineering ecosystem and relies on community contributions. While the codebase is minimal, the data is well-structured and validated through CI pipelines, making it a reliable resource for Flipper Zero users.
Flent is a mature, actively maintained network testing tool designed primarily for bufferbloat detection. It wraps multiple simultaneous network performance tools (netperf, iperf, ping, fping, irtt) into a single coordinated test, aggregates time-series data, and produces publication-quality plots via matplotlib. The architecture is a Python CLI application with an optional Qt GUI, built around a modular system of test definitions (Python .conf files), pluggable runners, aggregators, combiners, and formatters. It supports batch execution for reproducible experiments, extensive metadata collection (kernel parameters, qdisc stats, WiFi stats), and multiple output formats (CSV, JSON, plots). While primarily an application, it has strong data collection and analysis characteristics, making it a hybrid tool for network performance research. The codebase is well-structured with clear separation of concerns, though it carries some legacy Python 2 compatibility shims and platform-specific Linux dependencies.
This repository is a data collection and presentation project for single-board computer (SBC) benchmarks maintained by Jeff Geerling. It consists of a set of automated benchmarking scripts written in Python using the pyinfra orchestration tool, which can be run against any SSH-accessible SBC to gather standardized performance metrics including Geekbench, HPL (LINPACK), power consumption, memory bandwidth, disk I/O, GPU benchmarks, and AI inference performance. A secondary set of Ansible playbooks provides a simpler but less comprehensive alternative. All collected results are manually curated into a YAML file (web/data/results.yml) and rendered into a static Hugo website with a sortable comparison table. The repository serves as a public reference for SBC performance comparisons. While primarily a data research project focused on benchmarking hardware, it also provides reusable infrastructure for running benchmarks on any Linux or macOS system.
EasySpider is a visual, no-code web scraping tool that combines an Electron desktop application, a Chrome browser extension, and a Python execution engine. Users design scraping workflows via a flowchart-based UI (Vue.js) where they define steps like opening pages, clicking elements, extracting data, looping, and conditional branching. The task definitions are saved as JSON graphs. The Python backend (ExecuteStage) loads these graphs and drives Chrome using Selenium with an undetected-chromedriver fork to evade bot detection. Extracted data can be written to CSV, Excel, JSON, or MySQL. The tool also includes OCR support (ddddocr), custom JavaScript/Python execution, and Cloudflare bypass capabilities. The architecture is a hybrid desktop app with a local HTTP server (Express) that spawns executor processes and communicates via WebSocket. The Chrome extension assists in element selection and XPath generation. While the tool is functional and popular (44k+ stars), the codebase exhibits significant duplication (e.g., identical task files across platform directories, duplicated Bootstrap and content scripts), hardcoded paths, monolithic executor scripts, and mixed-language strings. Several experimental modules (LLM, federated learning) are present but not integrated. The project is actively maintained with recent commits and packaging scripts, but technical debt and drift (Bootstrap 4, Vue 2, pinned Python dependencies) are moderate.
This repository is a static website ranking Chinese GitHub developers by follower count, built with Astro and Tailwind CSS. It features two ranking tabs: 'Developers' and 'Markdown Engineers', based on a curated list of users known for documentation/content creation. The site fetches data daily via GitHub Actions using a Node.js script that queries the GitHub API. While primarily a web application, it has a significant data-engineering component in the form of the data collection script, which includes rate limiting, pagination, caching, and deduplication. The frontend implements client-side search functionality and lazy-loaded avatars for performance. The UI is designed with a VSCode-inspired dark theme. The codebase is well-structured, with clear separation of data fetching, utilities, and presentation components. However, the ranking logic and the classification of 'Markdown Engineers' are based on a hardcoded list, which is opinionated and not algorithmic. Overall, the project is a functional and visually polished static ranking site with a robust data pipeline.
This repository is a curated collection of all-cause mortality data from 128 countries and territories, spanning 2015–2024, with additional local and preliminary mortality datasets. It is primarily a data asset rather than a software project: the core deliverable is the `world_mortality.csv` file, which aggregates weekly, monthly, or quarterly death counts from official sources such as the Human Mortality Database (STMF), Eurostat, and various national statistical offices. The READMEs provide exhaustive documentation of data sources, transformation notes (e.g., corrections for delayed registration, unknown-month distribution), and methodological caveats. No executable scripts (e.g., R or Python) are present in the provided file tree, meaning the ETL pipeline is not included in this repository. The data supports excess mortality analysis, notably the companion repository `dkobak/excess-mortality`. The project is actively maintained with recent commits and community engagement, but the lack of dependency management, environment definitions, and automated reproducibility infrastructure limits its reusability for other analysts.
A monolithic Python application for crawling public data from the Chinese social network Weibo (Sina Weibo). It retrieves user profiles, weibo posts (including text, images, videos, live photos), comments, and reposts via the Weibo mobile API. Data can be exported to CSV, JSON, SQLite, MySQL, MongoDB, Markdown, or forwarded to a custom HTTP endpoint. It provides a CLI entry point for scheduled crawling (using the `schedule` library) and a Flask-based REST API for remote trigger and data retrieval. The codebase is heavily centralized—the core `Weibo` class (~3000 lines) handles fetching, parsing, downloading media, writing to multiple outputs, and anti‑ban throttling. A lightweight LLM analysis module can optionally enrich each weibo with sentiment, summary, and anomaly detection. The repository is community‑maintained with 4.6k stars, 402 open issues, and frequent commits as of mid‑2026. It is primarily an application but has a strong data‑collection character; the data outputs are suitable for subsequent research or archiving.
SQLWATCH is a SQL Server performance monitoring solution designed for self-monitoring of database instances. The system collects performance metrics, wait statistics, disk utilization, index usage, and query performance data through Extended Events, DMVs, and performance counters, storing the data in a structured SQL Server schema. It features a dual-component architecture: a monitoring agent deployed on each target SQL Server instance (using SQL Agent jobs or Service Broker for scheduling) and an optional central repository importer (SqlWatchImport) for aggregating data from multiple instances. The solution includes alerting via threshold-based checks, automated actions, and integrates with Power BI and Grafana for visualization. Although the code demonstrates solid engineering patterns like snapshot-based collection, delta calculations, and metadata-driven configuration, the README explicitly states the open-source version is no longer actively developed, with active development moved to sqlwatch.cloud.
This repository functions as a static dataset collection designed for prompt engineering and Large Language Model (LLM) evaluation. It contains no executable application logic, servers, or CLI tools. The primary data assets are structured JSON files (`prompts-zh.json`, `prompts-zh-TW.json`) and a fragmented corpus of over 600 Markdown files (`question/*.md`) containing tabular prompt data. The data lineage is direct: raw text prompts are curated into these files without ETL pipelines or transformation scripts visible in the repository. The repository serves as a benchmark and training corpus for Chinese-speaking AI users. It addresses the problem of prompt standardization and provides a wide range of task categories (classification, generation, translation, coding) for testing model capabilities. However, the data engineering structure is suboptimal; the use of 600+ individual Markdown files for a single dataset introduces unnecessary fragmentation and maintenance overhead compared to a consolidated CSV or JSON format. Maintenance activity is indicated by the provided commit timestamp (2026-04-28), suggesting the repository is actively updated. However, the content exhibits significant technical drift. The prompts heavily reference GPT-3.5 and specific time-sensitive data (stock prices, news events from 2022-2023), which reduces the long-term utility of the dataset without manual curation. The data is self-contained and requires no external dependencies to consume, making it highly reproducible for immediate use.
This repository is a manually curated collection of IPTV channel playlists (M3U format) providing live streaming URLs for Chinese television networks, including CCTV, provincial satellites, Hong Kong/Macau/Taiwan channels, digital pay channels, and themed content loops (movies, TV series, Spring Festival Galas). The playlists are organized by category and include metadata such as tvg-id, tvg-name, tvg-logo, and group-title for compatibility with IPTV player software. The repository contains multiple dated backup files (IPv4 and IPv6), reflecting active maintenance through periodic updates. However, it is purely a data collection—no application code, scripts, or automated validation exists. The streams are sourced from various third-party CDNs, proxy servers, and ISP-specific endpoints, many of which rely on time-limited tokens and private infrastructure, leading to high link fragility and reproducibility challenges.