$ show karlicoss/promnesia
Promnesia is a browser extension (Chrome/Firefox) paired with a Python backend that enriches browsing history with contextual metadata from personal data sources. The backend indexes visits from browser history, bookmarks, chat logs (Telegram, Signal, etc.), social media (Reddit, Twitter, Hacker News), note-taking apps (Hypothesis, Pocket, Instapaper), and plaintext files into a SQLite database. A FastAPI server exposes endpoints (`/visits`, `/search`, `/visited`) that the extension queries to annotate visited links and display a sidebar with visit context, timestamps, and highlights. The system uses URL normalization (canonical URLs) to deduplicate visits across sources and supports user-defined filtering and hooks. The architecture is split into a Python CLI (`promnesia index` / `promnesia serve`), a TypeScript browser extension built with Rollup, and Docker containers for automated indexing. While the core is a personal data application, it also serves as a data pipeline for aggregating and querying personal activity, giving it a hybrid character.
The LLM identified 4 files worth highlighting for reuse:
Implements domain-specific URL canonicalization (e.g., YouTube, Reddit, GitHub, Facebook) to deduplicate visits from different sources. This module is self-contained with no internal dependencies, can be extracted for any project needing robust URL normalization, and demonstrates non-obvious handling of site-specific quirks (e.g., mobile subdomains, archive.org redirections).
Defines core data types (`Visit`, `DbVisit`, `Loc`, `Source`, `Res`) and utility functions (`extract_urls`, `traverse`, `from_epoch`) used throughout the codebase. These are clean, well-typed, and could be reused as a foundation for any personal data aggregation system. The `Res[T] = T | Exception` pattern for error handling is a simple but effective reusable pattern.
Provides a `LazyLogger` with lazy initialization, colored console output, and a debug collapse handler for noisy logs. This is a self-contained utility that can be dropped into any Python project to improve logging ergonomics without external dependencies.
Frontend URL normalization logic that mirrors the backend `cannon.py`. This module normalizes URLs in the browser extension (e.g., stripping tracking parameters, handling mobile subdomains). It is isolated, well-tested, and can be reused in other TypeScript projects needing client-side URL deduplication.