$ show ofou/graham-essays
This repository provides a Python CLI tool and Makefile pipeline to scrape all essays from Paul Graham's website (paulgraham.com), convert them to Markdown, and produce an EPUB ebook. The core logic resides in a single script (`graham.py`) that uses `requests`, `BeautifulSoup`, `html2text`, and `htmldate` to download, parse, and date-stamp each essay. A `Makefile` orchestrates the full build: creating a virtual environment, fetching essays, merging them with `pandoc`, and generating the final EPUB with a custom CSS and an iBooks dark-mode patch. The CI workflow (GitHub Actions) runs daily, detects new essays by comparing article counts, and automatically creates a release with the updated EPUB and metadata CSV. The architecture is straightforward—no web server, database, or tests—but the pipeline is robust and well-documented for its single purpose. While the scraper is tightly coupled to Paul Graham's site structure, several components (the iBooks patcher, the footnote conversion function, and the EPUB stylesheet) are cleanly isolated and reusable in other EPUB generation projects.
The LLM identified 3 files worth highlighting for reuse:
A self-contained script that patches the `META-INF/com.apple.ibooks.display-options.xml` inside an EPUB to set `specified-fonts=false`. This ensures Apple Books respects the reader's theme (including dark mode) rather than forcing fixed fonts. The script handles ZIP file manipulation with proper compression preservation and is reusable for any EPUB that needs this iBooks compatibility fix.
The `convert_to_pandoc_footnotes` function within `graham.py` converts HTML-style footnotes (e.g., `[1]` followed by definitions) into pandoc-style footnotes (`[^prefix1]:`). It supports optional essay ID prefixes to avoid collisions when merging multiple documents. The function uses regex to identify the notes section, extract definitions, replace inline references, and append the definitions. It is self-contained and could be extracted for use in any HTML-to-Markdown pipeline that needs footnote conversion.
A well-crafted EPUB stylesheet that avoids fixed text or background colors, instead using `color-scheme: light dark` and `currentColor` for borders and horizontal rules. This allows Apple Books and other readers to apply their own themes, including dark mode, without rendering text unreadable. The CSS includes a comprehensive reset, proper typography for Georgia serif, and responsive image handling. It is directly reusable in any EPUB project that prioritizes reader theme compatibility.