$ show ViggoZ/producthunt-daily-hot
This repository is a data collection and publishing pipeline for Product Hunt daily launches. It ingests data from the Product Hunt API (via an external consumer app identified by the UTM campaign 'decohack') and produces one markdown file per day, with each file containing structured entries for products: title, tagline, description, keywords, vote counts, comment counts, featured status, and links. The markdown files are the primary data artifact and are consistently formatted, making them parseable for trend analysis, competitive research, or AI training corpora. Supporting Python scripts handle image repair (with exponential backoff for rate limits), WordPress publishing via REST API, and markdown generation. GitHub Actions workflows automate generation, image fixing, and publishing. The repository is best classified as a data pipeline rather than an application: it exposes no API endpoints, UI, or library interface, and its value lies in the accumulated dataset and the automation around it. However, it is a hybrid in that it contains application-like scripts and CI/CD workflows. The data lineage is straightforward: Product Hunt API responses are transformed into daily markdown files, then optionally enriched with images and published to WordPress. The dataset in the analyzed chunk is incomplete (only a subset of dates is present), and there is no schema enforcement, validation, or test suite. The pipeline depends on external services (Product Hunt API, OpenAI, WordPress) that may change or rate-limit, and the hardcoded OpenAI model (gpt-4o-mini) is a drift risk. The markdown format is human-readable but less machine-friendly than JSON/CSV, and no README or data dictionary documents the schema or provenance. Although the last commit is recent, the data files in the analyzed chunk stop at 2025-09-28, suggesting the ingestion pipeline may not be running reliably.
The LLM identified 4 files worth highlighting for reuse:
Representative daily Product Hunt data file with a consistent, parseable markdown schema. Each product entry includes title, tagline, description, keywords, vote/comment counts, featured status, and links. This is a reusable data asset for trend analysis, competitive research, or as a training corpus. The uniform structure makes it easy to write parsers or import into a database, though there is no schema enforcement or validation.
Repairs missing images in existing markdown files by fetching from an API or scraping Open Graph tags. Implements exponential backoff for rate limits, a resilient pattern reusable in any scraper or enrichment pipeline. The script is somewhat tightly coupled to the markdown schema and duplicates image-fetching logic from product_hunt_list_to_md.py, but the retry/backoff approach is a solid pattern.
Core ETL script that fetches Product Hunt data and generates the daily markdown files. It defines the consistent schema used across the dataset and handles API interaction, image fetching, and markdown serialization. While tied to Product Hunt's API, the pattern of transforming a paginated API response into a structured, human-readable document is reusable for similar scraping pipelines.
Publishes generated markdown files as WordPress posts via the WordPress REST API. Demonstrates a simple content-publishing pipeline pattern that could be adapted for any markdown-to-CMS workflow. It requires WordPress credentials and is tied to the repo's file naming convention, but the REST API interaction is a clean reference for similar automation.