$ show Crosstalk-Solutions/project-nomad
Project NOMAD is an offline-first personal server platform that turns a single machine (Debian-based) into a self-contained knowledge and AI appliance. It orchestrates Docker containers for services like Kiwix (offline Wikipedia/books), Ollama (local LLMs), Qdrant (vector database for RAG), and many other curated apps (Calibre-Web, Jellyfin, Vaultwarden, etc.) via a Node.js/TypeScript backend (AdonisJS) with a React frontend (Inertia.js). The system manages content downloads (ZIM files, maps, FDA drug labels), provides an AI chat assistant with RAG (Retrieval Augmented Generation), offers hardware benchmarking (NOMAD Score), and includes a full auto-update mechanism for core, apps, and content. Key entry points include the installation script (`install_nomad.sh`), the HTTP server (`admin/bin/server.ts`), and an extensive REST API defined in `admin/start/routes.ts`. Background processing is handled by BullMQ job queues (downloads, embeddings, drug data ingestion, benchmarks). External dependencies include MySQL, Redis, Docker (DooD), Ollama, Qdrant, and openFDA for drug labels. The platform is designed for privacy (no telemetry) and resilience (offline operation, content caching, health-gated rollbacks).
The LLM identified 6 files worth highlighting for reuse:
A collection of pure, framework-agnostic functions for parsing FDA drug label JSON (openFDA format). Includes `mapDrugLabelRecord` which normalizes fields, joins text sections, and clamps column widths. `parseLabelSection` extracts structured data from plain text. These functions are isolated (no imports from the rest of the codebase) and reusable in any Node.js project that needs to transform FDA label data into a relational model.
Implements a resumable HTTP download function (`doResumableDownload`) that supports partial content requests, progress callbacks, stall detection (aborts if no bytes received within a configurable window), and atomic file writes (`.tmp` then rename). Includes mime-type validation. This is a self-contained utility that could be extracted into any Node.js project needing robust, resumable file downloads.
Contains reusable VineJS validation rules (`assertNotPrivateUrl` and `assertNotCloudMetadataUrl`) that block SSRF attacks by rejecting private/loopback/link-local IPs while allowing RFC1918 LAN addresses. Explicitly blocks the cloud metadata endpoint (169.254.169.254). These validators are used across multiple download endpoints and can be reused in any AdonisJS (or similar) project requiring URL input validation.
A BullMQ job handler that streams a large (~1.7GB compressed) JSON file from openFDA, processes records in batches with back-pressure (per-batch timeout, stall watchdog), and upserts into MySQL using `onConflict.merge` for idempotency. Handles continuation jobs for large datasets. Demonstrates memory-safe streaming ETL with error recovery, and is a reusable pattern for any Node.js project that must ingest large flat files into a database.
A central service (~1500 lines) that manages the full lifecycle of Docker containers (install, update, uninstall, force-reinstall) with health‑gated rollbacks (rename old container, start new, wait for health, then remove old or roll back). Auto‑detects GPU type (NVIDIA/AMD) via Docker info, marker files, and lspci; resolves host storage root from mount info to relocate all app binds; generates self‑signed TLS certs for Vaultwarden/MeshCore; and humanizes Docker errors. Though large, it demonstrates a robust, reusable container orchestration pattern that is not tied to a specific cloud provider.
A generic, reusable React table component supporting loading states, expandable rows, sorting, pagination, and dark mode styling. Accepts column definitions and row data as props, with no dependency on backend APIs or state management libraries beyond React. Shows solid component design and could be extracted into any React project needing a flexible table.