OpenDataLoader-PDF is a production-grade, multi-language PDF-to-structured-content extraction tool designed for AI/LLM pipelines, RAG systems, and accessibility workflows. It converts PDFs into JSON, Markdown, HTML, plain text, and tagged PDF (PDF/UA) with high accuracy (0.907 overall benchmark). The core engine is written in Java (veraPDF-based) and provides a rich CLI, while Python and Node.js SDKs bundle the Java JAR and expose idiomatic APIs. A hybrid mode routes pages to an external AI backend (Docling) for OCR, formula extraction, and picture descriptions, with a triage processor deciding per-page routing based on content signals. The project also includes an MCP server for AI agent integration, a LangChain loader, and an agent skill bundle for correct usage.
This monorepo provides a suite of libraries and CLI tools for generating changelogs from git metadata according to the Conventional Commits specification. The core pipeline consists of: a git client to fetch commits, a parser to extract structured data from commit messages, a filter to remove reverted commits, and a writer to render the changelog in Markdown. Additional packages handle version bump recommendations, preset loading (Angular, Conventional Commits), and template rendering. The system is designed for both programmatic use via builder-style APIs (e.g., `ConventionalChangelog`, `Bumper`) and direct CLI invocation. It supports multiple git hosting platforms (GitHub, GitLab, Bitbucket) and is configured through presets that define commit type effects, reference formatting, and section ordering. The architecture is a monorepo with 11+ independently published packages, each with its own tests, TypeScript definitions, and CLI entry points. The codebase is actively maintained, with frequent releases, breaking changes managed through major version bumps, and a migration from Handlebars to render functions. It requires Node.js >= 22 and is fully ESM. The project also includes a documentation website (Astro/Starlight), CI/CD via GitHub Actions, and an AI agent skill for commit message guidance. While primarily a developer tool, the repository demonstrates strong software engineering practices: clean separation of concerns, stream-based processing for large commit histories, a preset plugin pattern, and comprehensive test coverage including real git repository interactions. The lower-level packages (parser, filter, writer, git client) are designed to be reusable in other contexts, and the template utilities provide a foundation for custom changelog formatting.
MarkdownDB is a JavaScript library and CLI tool that parses markdown files to extract structured data (frontmatter, tags, links, tasks) and stores it in a queryable SQLite database (or JSON files). It provides a lightweight Node.js API for querying the indexed data and includes a command-line tool for indexing folders and executing scripts. The architecture follows a modular design with clear separation between markdown parsing, processing, storage, and querying layers. The system uses remark-family tools for markdown parsing, Knex.js for database operations, and Zod for schema validation. Key entry points are the CLI (`src/bin/index.js`) and Node.js API (`src/index.ts`).
Logseq is a privacy-first, open-source knowledge management platform built as a monorepo containing an Electron desktop app, Capacitor-based mobile apps (iOS/Android), a web app, and a Node.js db-sync server. The codebase is predominantly ClojureScript (CLJS) with React/Rum for the UI, DataScript for the in-memory graph database, and SQLite (via better-sqlite3 and sqlite-wasm) for persistence. It uses shadow-cljs for CLJS compilation, webpack for JS bundling, and Tailwind CSS for styling. A separate OCaml-based CLI (logseq-cli) built with Melange provides graph inspection, sync, and CRUD operations against the same db-worker-node backend. The architecture separates concerns into deps/ subprojects (db, outliner, graph-parser, db-sync, common, publishing, shui) that are compiled as separate shadow-cljs targets and shared across Electron, browser, mobile, and CLI runtimes. The system has two graph types: file-based graphs (Markdown/Org) and DB graphs (SQLite-backed with DataScript in-memory querying). DB graphs enable real-time collaboration (RTC) via a Cloudflare Worker-based sync server (deps/db-sync) using WebSockets and D1 SQLite. The db-worker-node is a long-lived Node.js process that manages SQLite connections and sync state, communicating with the UI via a thread-api IPC mechanism. The outliner module handles block tree operations with structural invariants, while the graph-parser handles Markdown/Org parsing via mldoc. The publishing module generates static HTML sites from graphs. Key architectural patterns include: missionary for reactive async flows, promesa for promise composition, malli for schema validation, datascript for Datalog queries, and a component-based UI with Rum/React. The codebase has extensive test coverage via cljs.test, Playwright E2E tests (clj-e2e), and Python-based CLI E2E tests (cli-e2e). The repository is actively maintained with recent commits, 43k+ stars, and active issue triage, though the large monorepo complexity and multi-language surface (CLJS, OCaml, Swift, Kotlin, Python, JS) create maintenance challenges.
Defuddle is a content extraction library that converts web pages into clean, readable Markdown with structured metadata (title, author, site, published date). It is the engine behind the Obsidian Web Clipper browser extension and also runs as a standalone CLI tool, an npm package, and a hosted API service on Cloudflare Workers. The architecture follows a modular pipeline: fetch HTML, detect site-specific extractors, score and select main content, remove clutter (nav, ads, sidebars), standardize complex elements (footnotes, math, code blocks, images), and convert to Markdown via Turndown. The library is designed as a more forgiving alternative to Mozilla's Readability, with support for mobile-style heuristics, schema.org metadata extraction, and extensive site-specific extractors for platforms like YouTube, Reddit, Twitter/X, ChatGPT, GitHub, and Gmail. The codebase is written in TypeScript and produces three bundles: core (browser, no extra dependencies), full (adds MathML-to-LaTeX conversion), and node (accepts any DOM implementation). The CLI uses Commander and supports file, URL, and stdin input with options for Markdown, JSON, frontmatter, and debug output. The hosted API runs on Cloudflare Workers with Stripe-based billing, rate limiting, and Durable Objects for stateful API key management. The test suite is exceptionally comprehensive, with hundreds of HTML fixtures and expected Markdown outputs covering edge cases from dozens of real-world websites, CMS patterns, and syntax highlighters. Key external dependencies include Turndown (Markdown conversion), linkedom (DOM parsing in Node.js), mathml-to-latex and Temml (math rendering), and various third-party APIs for site-specific extractors (YouTube InnerTube, FxTwitter, Reddit Atom feeds, Bilibili). The project maintains strict security practices: no innerHTML assignments outside a single utility function, URL sanitization, and XSS prevention through template-based HTML parsing. The build system uses webpack with aliases for math module switching and enforces bundle size budgets via CI scripts.
Tandoor is a self-hosted, multi-tenant recipe management web application. The system is a monolithic Django application (cookbook app) exposing a Django REST Framework API, paired with a Vue 3 + Vuetify single-page frontend that consumes an auto-generated TypeScript OpenAPI client. Core production domains, verified in code, are: structured recipes (steps, ingredients, scaling), a tree-based hierarchical food database with nutritional properties and USDA FDC IDs, meal planning with iCal export, shopping lists with supermarket categories and offline sync queues, inventory/pantry booking, cookbooks, share links, and view/cook logging. Tenancy is enforced through Space/Household/UserSpace models with django-scopes middleware and granular guest/user/admin permission classes rather than through database row-level security. The largest subsystem is recipe import/export: an abstract Integration base class with 20+ adapters for other recipe managers (Paprika, Mealie, Cooklang, Gourmet, ChefTap, etc.), URL scraping via recipe-scrapers/microdata/pytubefix, cloud storage sync providers (Dropbox, Nextcloud, WebDAV, local), a bulk open-data importer for USDA-derived reference foods/units/conversions, and an embedded PDF.js viewer for external recipe files. A connector/event layer dispatches async updates to external services such as Home Assistant, a Telegram bot webhook writes shopping list entries, and recent AI features (litellm-based, per-space credit tracked) handle image/PDF recipe import, step sorting, and nutrition extraction. Background work is done via threads rather than a task queue. Craftsmanship is above average: clean abstract patterns (Integration, Provider, Connector ABCs), chainable queryset managers, mixin-based permission/merge/tree logic, a strong permission-matrix test suite, and genuinely reusable helper algorithms (BFS unit conversion, ingredient parsing, Cooklang parsing). However, the codebase carries notable debt: 240+ migrations with heavy schema churn, a ~3000-line monolithic api.py view file, committed auto-generated OpenAPI client code that can drift, vendored PDF.js binaries, some tests that suppress tenant scoping via scopes_disabled, and copy-paste duplication inside the open-data importer. Overall it is a well-maintained, actively developed application with solid architecture for its scale.
nb is a command-line note-taking, bookmarking, and archiving tool with Git-backed versioning, encryption, full-text search, tagging, pinning, and a built-in web server for browsing. It is implemented primarily as a single portable Bash script, with optional Go, Ksh, and Zsh wrappers. The architecture is a monolith CLI with a plugin system for extending subcommands and themes. Key entry points are the `nb` script and its aliases `bin/notes` and `bin/bookmark`. Data flows involve creating/editing notes in `NB_DIR`, where each notebook is a Git repository; operations create automatic commits, and an `.index` file per folder maintains ordering. The tool supports multiple notebooks, nested folders, encryption (OpenSSL/GPG), bookmark import from browsers, and remote sync via Git. The extensive test suite (2200+ BATS tests) covers all major features and edge cases.
Eleventy Notes is a highly opinionated static site generator template built on top of Eleventy (11ty). It is designed for publishing personal notes, documentation, wikis, or blogs with a focus on rich interlinking, client-side search, and customization. The system is a static site generator that processes Markdown files into a fully featured website. It includes a modular architecture with core infrastructure (layout, header, sidebar), and modules for wikilinks, callouts (admonitions), tags, custom properties, dynamic content queries, table of contents, search (using Pagefind), bookmarks, theming, translations, and page navigation. The build process uses Parcel for CSS/JS bundling and Eleventy for HTML generation. The application is designed to be configured via a single JavaScript file (`app.mjs`) and a SCSS file (`app.scss`). The code is well-structured, uses ES modules, and demonstrates advanced patterns such as custom markdown-it plugins, a generic query engine, and a tree generator. It is a template intended for end users to fork and customize, but the underlying codebase itself exhibits high craftsmanship and could serve as a reference for building Eleventy-based projects.
SiYuan is a privacy-first, local-first personal knowledge management system with a block-based WYSIWYG editor, AI assistant integration, cloud sync, and a plugin ecosystem. It is architected as a Go kernel (Gin HTTP API + WebSocket) paired with a TypeScript/Electron frontend, supporting desktop (Windows, macOS, Linux), mobile (Android, iOS, HarmonyOS), and browser/Docker deployment. The kernel handles all data operations: block tree persistence as JSON `.sy` files, SQLite indexing with FTS5 and SQLCipher encryption, attribute-view databases (tables, galleries, kanban), backlink resolution, full-text search, and sync via multiple providers (SiYuan cloud, S3, WebDAV, LAN). The frontend provides a rich editing experience with Markdown, LaTeX, Mermaid, mind maps, PDF annotation, flashcards (FSRS), and an agent chat interface. External dependencies include the Lute Markdown parser, dejavu sync library, Pandoc for document conversion, and PDF.js for PDF rendering. The system exposes a comprehensive REST API with contract-based validation and auto-generated TypeScript types, enabling third-party integrations and a community marketplace for plugins, themes, and templates.
presenterm is a Rust-based CLI application that renders Markdown presentations directly in the terminal. It parses Markdown into a structured intermediate representation via comrak, then builds slides as sequences of render operations. The rendering engine supports rich text, images, code highlighting through bat/syntect, column layouts, themes, slide transitions, footers, speaker notes, and interactive keyboard-driven navigation. It also includes PDF/HTML export backends and hot-reload for live presentation development.
This is a Go library for rendering richly formatted tables in a variety of output formats, including ASCII, Unicode, Markdown, HTML, ANSI-colored, and SVG. It provides both a batch mode (`Render`) and a streaming mode (`Start`, `Append`, `Close`) for incremental table generation. The library supports advanced features such as horizontal, vertical, and hierarchical cell merging, per-column alignment, padding, word wrapping, and custom cell formatters. It includes a CLI tool `csv2table` that converts CSV input to tables with the full feature set. The codebase is modular, with separate packages for display width calculation (East Asian language support), word wrapping (minimal raggedness algorithm), an LRU cache, and a set of pluggable renderers. Configuration is done via functional options and a fluent `ConfigBuilder` pattern. Although comprehensive, the API is in a state of transition: many legacy methods (e.g., `SetBorder`, `SetAutoFormatHeaders`) are deprecated, and the migration guide warns that the current version (v1) is still evolving. The architecture is sound, with clear separations between data processing (`tablewriter` core), presentation (`tw` types and `renderer` implementations), and utility packages (`twwidth`, `twwarp`, `twcache`). The library is well-tested with over 50 test files and benchmarks, and demonstrates strong engineering practices.
This is a PHP library that converts HTML strings to Markdown. It parses HTML into a DOMDocument, then recursively traverses the DOM tree, converting each node to its Markdown equivalent using a registry of converter classes. The architecture is clean: an `HtmlConverter` orchestrates the process, an `Environment` manages configuration and converter registration, and individual converter classes handle specific HTML elements (headers, links, lists, tables, etc.). The library supports extensive configuration options including stripping tags, preserving comments, customizing emphasis styles, and enabling table conversion. It provides both a programmatic API and a CLI binary. The code is well-structured with strict typing, interfaces, and comprehensive test coverage. It demonstrates solid software engineering practices and is actively maintained.
omni-article-markdown is a CLI tool and library that converts web articles from various sources into clean Markdown format. It employs a modular pipeline architecture: Reader fetches HTML (using requests or Playwright for JavaScript-heavy sites), Extractor isolates article content by removing boilerplate and ads, and Parser converts the cleaned HTML to Markdown with support for code blocks, tables, math equations, and GitHub Gists. The system uses a plugin-based architecture where site-specific readers and extractors are automatically discovered and loaded, allowing easy extension for new websites. Key features include SSL verification control, cookie persistence for sites requiring authentication (like Zhihu), and flexible output options (stdout or file saving).
render-markdown.nvim is a Neovim plugin that transforms raw Markdown files into a richer, more readable in-editor experience. Instead of opening an external preview window, it hooks into Neovim's tree‑sitter parser to understand the Markdown AST and overlays virtual text, highlights, signs, and concealments directly onto the buffer. It handles headings, code blocks, tables, checkboxes, callouts, LaTeX math, wikilinks, HTML tags, and YAML frontmatter, with full support for custom icons and per‑element styling. The plugin is pure Lua and requires Neovim ≥ 0.9.0 (recommended ≥ 0.10.0) and a Nerd Font for optimal icon display. The architecture is modular and layered. The main entry point `setup()` in `init.lua` merges user configuration with default presets and per‑buffer overrides. The `core/` directory manages the rendering lifecycle: `manager.lua` attaches to buffers and schedules renders, `handlers.lua` dispatches per‑language tree‑sitter parser requests, and `ui.lua` applies the resulting extmarks to visible windows. Each Markdown element (heading, code block, table, etc.) has its own renderer under `render/markdown/`, all inheriting from a common base class in `render/base.lua`. Utility modules in `lib/` provide reusable primitives for interval arithmetic, extmark lifecycle, virtual text construction, line manipulation, and configuration schema validation. The plugin also integrates with completion engines (nvim‑cmp, blink.cmp, coq_nvim) through a shared completion source in `integ/source.lua`. Code quality is high: the codebase includes an extensive test suite (unit, integration, and visual regression tests), benchmarks for performance on large files, a health check for dependencies, and a LuaRocks CI pipeline. The documentation includes a custom handler interface for user extensions, detailed troubleshooting guides, and auto‑generated type documentation. Despite a large configuration surface, schema validation and preset overrides keep the system flexible yet robust. The main risks are dependency on tree‑sitter grammar internals and Neovim API version changes, but the project is actively maintained and adapts quickly to ecosystem shifts.
Madopic is a client-side web application that converts Markdown content into images (PNG, PDF, HTML) with support for mathematical formulas, diagrams, charts, and custom styling. It operates as a single-page application with a split-screen interface: users write Markdown in the left editor and see a live preview on the right. The application processes Markdown through a pipeline that includes syntax highlighting, formula rendering (via KaTeX/mhchem), diagram rendering (via Mermaid), chart rendering (via ECharts), and custom card syntax, all before updating the preview. The application emphasizes a seamless user experience with real-time previews, customizable themes (background, font size, layout), and multiple export options. It implements sophisticated features like offline image persistence using IndexedDB with an in-memory cache, a protected token system to prevent interference between different Markdown extensions (such as preventing math formula replacements inside code blocks), and exact export cloning to ensure that the exported image matches the preview precisely, including handling of specific aspect ratios for platforms like Xiaohongshu and WeChat Moments. Built with vanilla HTML, CSS, and JavaScript, Madopic relies on several CDN-hosted libraries for its core functionality but maintains a lightweight, dependency-conscious architecture. The application is designed as a progressive web app (PWA) and includes features like local draft autosave, keyboard shortcuts, and responsive design to cater to a wide range of use cases from academic to social media content creation.
Dillinger is a cloud-enabled, online Markdown editor with live preview, export to Markdown, HTML, and PDF, and synchronization with cloud storage services including GitHub, Dropbox, Google Drive, OneDrive, and Bitbucket. Built with Next.js 14 (App Router), the application uses Monaco Editor for editing, Zustand for client-side state management with localStorage persistence, and markdown-it with a rich plugin chain for rendering. It features a modular architecture with API routes for OAuth flows and file operations per cloud provider, a versioned public REST API (v1) for rendering and export, and separate packages for a CLI tool and an MCP server for LLM integration. The codebase is well-organized with clear component separation (editor, preview, sidebar, modals, UI primitives), custom hooks per integration, and comprehensive unit and E2E test coverage. Auxiliary agent/skill files and design system CSVs exist in the repository but are development tools, not part of the application runtime.
Goldmark is a CommonMark-compliant Markdown parser and HTML renderer written in Go, designed for high performance and extensibility. The v2 branch introduces a complete redesign with Go generics, a cleaner separation of parser and renderer, and more semantic AST nodes. The core library exposes a `parser` package with `New()` and `Parse()` to produce an AST, and a `renderer/html` package with `New()` and `Render()` to output HTML. Extensions (tables, footnotes, strikethrough, task lists, typographer, linkify, definition lists) are implemented via a well-defined extension pattern using `parser.Extension` and `html.Extension` interfaces, making it straightforward to add custom syntax. The architecture is a library rather than a standalone application; it has no CLI or server, but is used by many static site generators and other tools. The codebase demonstrates strong engineering practices: extensive test coverage (including CommonMark spec tests and fuzzing), benchmarking against the C reference implementation, and a modular design that abstracts parsing, rendering, and text handling. The v2 version is still maturing, with some breaking changes from v1 that may affect third-party extensions, but the core is robust and well-documented.
Qiita CLI is a command-line tool and preview server for managing Qiita blog posts locally. It allows users to create, edit, preview (with hot-reload), and publish articles and slides (Marp-based) to the Qiita platform via its REST API. The architecture separates CLI commands (init, login, new, preview, publish, pull) from a file-system-backed repository layer that synchronizes local markdown files with Qiita's remote state using a `.remote` mirror directory. A React-based web frontend provides a sidebar for navigating articles/slides and an embedded preview of rendered Markdown (including MathJax and embedded scripts). The server also supports Marp slide preview with shadow DOM isolation for CSS scoping. The codebase is well-structured, uses TypeScript, has comprehensive unit tests, and follows modern Node.js patterns (async/await, error handling, dependency injection). It is actively maintained by Qiita Inc. and targets Node.js 22+.
Firecrawl is a production-grade web scraping, crawling, and structured data extraction platform designed to feed AI agents and LLMs with clean, markdown-formatted web data. The system exposes a multi-version REST API (v0, v1, v2) with endpoints for scraping single URLs, crawling entire sites, mapping site structures, searching the web, extracting structured JSON, monitoring page changes, and running autonomous AI agents. It supports multiple output formats including markdown, HTML, screenshots, PDF parsing, product/menu extraction, and branding detection. The architecture is a microservices-style monolith with a Node.js/Express API server at its core, backed by Redis for job queues and caching, PostgreSQL for persistent state, ClickHouse for analytics, Bigtable for job access logs, and GCS for blob storage. Background workers handle crawl orchestration, PDF processing, webhook delivery, and billing via RabbitMQ and FoundationDB queues. The platform integrates with external services for threat protection (Google Web Risk, Zscaler), billing (Autumn), search (SearXNG, DuckDuckGo, X Search), and LLM-based extraction (OpenAI, Anthropic). Nine official SDKs (Python, JavaScript/TypeScript, Go, Java, .NET, Ruby, PHP, Rust, Elixir) wrap the API with typed models, error handling, and async support, making it accessible across ecosystems. The codebase is extensive and well-tested, with over 2,000 files spanning controllers, scrapers, transformers, services, and utilities, but shows signs of technical drift through deprecated v0/v1 endpoints, multiple parallel implementations of similar logic (e.g., extract pipelines), and configuration sprawl across environment variables.
Slidev is a developer-oriented slide presentation tool that turns Markdown files into interactive, web-based slide decks. The system is built as a monorepo with several packages: a Node.js CLI and Vite-based dev server (`packages/slidev`), a Vue 3 client application (`packages/client`), a markdown parser (`packages/parser`), shared type definitions (`packages/types`), a VS Code extension (`packages/vscode`), and scaffolding tools (`create-app`, `create-theme`). The core workflow involves parsing Markdown slides with frontmatter, rendering them as Vue components with support for code highlighting (Shiki), diagrams (Mermaid, PlantUML, KaTeX), interactive editors (Monaco), animations (v-click, v-mark), and drawing tools. Export capabilities include PDF, PPTX (both raster and editable), PNG, and Markdown via Playwright. The system also includes a Model Context Protocol (MCP) server for AI agent integration, enabling programmatic slide editing. The architecture is modular: the parser handles slide splitting and feature detection, the client provides a rich presentation UI with presenter mode, and the CLI orchestrates build, serve, and export commands. The VS Code extension adds language support, a preview panel, and AI tools. Overall, Slidev is a mature, feature-rich application for creating technical presentations with minimal friction.