$ show dgtlmoon/changedetection.io
changedetection.io is a self-hosted web application for monitoring web pages for content changes. It periodically fetches URLs, computes diffs (text, JSON, image, or stock/price), and sends notifications via 100+ channels (Apprise). The system is built on Flask with Socket.IO for real-time UI updates, a pluggable processor architecture (text_json_diff, restock_diff, image_ssim_diff), and an LLM integration layer (via litellm) for intelligent change summarization and intent-based filtering. It supports multiple fetcher backends (requests, Playwright, Selenium, Puppeteer), browser automation steps, RSS feeds, and a REST API with OpenAPI spec. Architecturally, it is a monolith with background workers: a priority queue (RecheckPriorityQueue) dispatches watch rechecks to async workers, which fetch content, run processors, and trigger notifications. The datastore is file-based JSON with atomic writes and schema migrations. The frontend uses jQuery, SCSS, and custom JavaScript for diff visualization, visual selectors, and real-time queue monitoring. The plugin system (pluggy) allows extending fetchers, processors, conditions, and UI elements. Key entry points include the CLI (changedetection.py), the Flask app factory (flask_app.py), the worker pool (worker_pool.py), and the notification service (notification_service.py). External dependencies include Playwright, Apprise, litellm, lxml, and various image processing libraries (OpenCV, libvips). The codebase is well-tested with over 120 test files covering security, integration, and unit tests.
The LLM identified 4 files worth highlighting for reuse:
Implements a natural text tokenizer for diff engines, splitting text into words and whitespace tokens. It is a self-contained strategy class that can be extracted and reused in any diff or text comparison project. Demonstrates clean separation of concerns and extensibility via the tokenizer interface.
Provides BM25-based trimming of text to fit LLM context windows. It scores lines by relevance and keeps only the most important ones. This is a reusable utility for any LLM application that needs to reduce prompt size while preserving key information. The implementation is self-contained and parameterized.
Implements multi-layer URL validation to prevent Server-Side Request Forgery (SSRF). It checks scheme, blocks private IPs, detects parser differential attacks, and validates against allowed patterns. This is a security-critical component that can be extracted and reused in any Python web application that fetches external URLs.
A custom Apprise plugin for Discord that sends rich embeds with color-coded diffs and formatted content. It demonstrates how to extend the Apprise notification library with a custom handler. The plugin is self-contained and can be reused in any Apprise-based notification system.