tqdm is a fast, extensible progress bar library for Python and CLI. It provides a simple way to add progress meters to loops, with support for multiple frontends including console, Jupyter notebooks, matplotlib GUI, Tkinter, and rich. The core implementation in `tqdm/std.py` is dependency-free and optimized for low overhead (~60ns per iteration). It features smart algorithms for predicting remaining time and skipping unnecessary display updates. The library integrates with pandas, keras, dask, asyncio, and concurrent.futures. The CLI mode allows piping data through tqdm for progress monitoring. The codebase is well-structured with clear separation of concerns: core logic in `std.py`, frontends in separate modules (`notebook.py`, `gui.py`, `tk.py`, `rich.py`), and integrations in `contrib/`. Extensive test suite covers performance, correctness, and edge cases. The project is actively maintained with regular releases and CI.
This repository produces and maintains the official Jupyter Docker Stacks: a family of layered Docker images that ship JupyterLab, Jupyter Notebook, JupyterHub single-user servers, and a broad set of scientific computing kernels and libraries. The actual deliverable is infrastructure-as-code: Dockerfiles, entrypoint/startup scripts, a Python tagging system, CI workflows, and a container-based test suite. Although the images serve data-science workflows (pandas, Spark, PyTorch, R, Julia, etc.), the repository itself is an application/devops project for building, versioning, and publishing reproducible computing environments, not an analytics or ETL codebase. This hybrid nature shows in bundled research-oriented packages and notebook-based tests, but the core engineering artifact is container image definitions and release automation.
Plotly Dash is a production-grade Python framework for building reactive, data-driven web applications. It provides a declarative callback system (server-side, clientside, background, streaming), a rich component library (core components, data table, HTML wrappers), and a React/Redux frontend renderer. The architecture supports multiple backends (Flask, Quart, FastAPI), real-time communication via WebSocket, cross-process shared storage (local, diskcache, Redis), and an MCP (Model Context Protocol) server for AI/LLM integration. The monorepo includes component generation tooling, extensive testing infrastructure (unit, integration, visual, WebSocket, background callback), and performance benchmarks. While primarily an application framework, its data table, filtering syntax tree, and shared storage subsystems demonstrate strong data engineering patterns.
JupyterHub is a production-grade multi-user server for Jupyter notebooks. It provides authentication, spawning of single-user notebook servers, and proxying. Built on Tornado, it uses SQLAlchemy for persistence, supports pluggable Authenticators and Spawners, and includes an OAuth2 provider, RBAC scopes, a REST API, and a web UI. The system is designed for deployment in education, research, and enterprise settings, managing user sessions, isolating notebook environments, and routing traffic through a configurable HTTP proxy. The architecture follows a hub-and-spoke model: the Hub (Tornado process) handles authentication, authorization, and state; the ConfigurableHTTPProxy routes requests to per-user single-user servers; and Spawners manage server lifecycle. Key entry points include the `jupyterhub` CLI command, the `jupyterhub_config.py` configuration file, and the REST API at `/hub/api`. External dependencies include Tornado, SQLAlchemy, Alembic, oauthlib, prometheus_client, and aiohttp. The codebase is well-structured with clear abstractions for authentication, spawning, proxying, and authorization, making it a strong example of a complex web application framework.
Vaex is a high-performance Python library for lazy out-of-core DataFrames designed to handle datasets larger than memory. It achieves efficiency through memory mapping, zero-copy policies, lazy evaluation, and virtual columns that avoid materializing intermediate results. The system supports HDF5, Apache Arrow, Parquet, CSV, and FITS formats, with accelerated operations via C++ extensions for critical paths like hash-based aggregations and string processing. The architecture is modular, consisting of a core package (vaex-core) and specialized extensions: vaex-viz (visualization), vaex-hdf5 (HDF5 enhancements), vaex-astro (astronomy tools), vaex-ml (machine learning integrations), vaex-jupyter (notebook support), vaex-server (remote access via WebSocket/REST), and vaex-graphql (GraphQL API). Key entry points include the `vaex` CLI tool, `vaex.open()` for data loading, and programmatic APIs for DataFrame operations, expressions, and visualizations. Data flows prioritize out-of-core processing: data remains on disk until needed, computations are lazily executed and cached via fingerprinting, and results are streamed to minimize memory footprint. The codebase demonstrates strong engineering practices with a plugin system for extensibility (expression/dataframe accessors), centralized settings configuration, and a task-based execution engine supporting parallelism and async operations. While primarily a general-purpose data analysis library, it is widely used in data research due to its efficiency with large datasets and domain-specific integrations (e.g., astronomy via vaex-astro). Recent commits indicate active maintenance, low technical drift through adoption of modern dependencies (Apache Arrow, FastAPI), and a comprehensive test suite covering core functionality and extensions.
Jupyverse is a modular Jupyter server implementation built on the FPS (FastAPI Plugins System) framework. It provides a customizable backend for Jupyter interfaces (JupyterLab, Notebook) through interchangeable plugins handling authentication, file contents, kernel management, terminals, real-time collaboration, and more. The system uses FastAPI for HTTP/WebSocket endpoints and anyio for async operations, with a core API defining interfaces that plugins implement. Entry points include a CLI command and composed plugin modules. Key data flows involve routing requests to appropriate plugins (e.g., contents API for file operations, kernel plugin for code execution) and leveraging WebSockets for real-time features like collaboration and terminal communication. External dependencies include FPS, Jupyter libraries, Yjs for CRDTs, and various auth providers.
ITables is a Python library that transforms Pandas, Polars, and other DataFrame libraries into interactive, sortable, and filterable HTML tables using the DataTables.js library. It integrates seamlessly with Jupyter Notebook/Lab, VS Code, Colab, Quarto, and web frameworks like Dash, Streamlit, Shiny, and Panel. The architecture is a thin Python wrapper that generates HTML/JavaScript snippets, with a bundled JavaScript package (dt_for_itables) containing DataTables and its extensions. Key entry points include `itables.show()`, `itables.init_notebook_mode()`, and framework-specific components (e.g., `itables.dash.ITable`). Data flows from DataFrame to JSON serialization (with special handling for floats, categories, and large integers) to HTML template injection. External dependencies include DataTables.js (via CDN or offline bundle), optional Narwhals for multi-backend support, and framework-specific packages (Dash, Streamlit, etc.). The library also provides fallback static HTML and Markdown table rendering for environments without JavaScript.
Calysto Scheme is a sophisticated Scheme interpreter implemented in Python that integrates with Jupyter notebooks. It features a multi-stage compilation pipeline (CPS → DS → RM → Python) with JIT compilation, continuation support (call/cc), and seamless Python interoperability. The system includes advanced optimizations like persistent vectors for O(1) lexical address lookup, epoch-based cache invalidation, and escape analysis for JIT inlining. It's used in educational contexts and provides debugging tools like a stepper-debugger, making it both a practical development tool and a platform for teaching programming language concepts.
HugoBlox Kit is a modular static site framework built on Hugo, providing a block-based page builder for academic, portfolio, documentation, and marketing websites. It uses Hugo's template system with Preact for interactive components, Tailwind CSS v4 for styling, and supports various third-party services (analytics, comments, search). The architecture is a monorepo with core modules (blox, analytics, slides, integrations) and multiple starter templates. Key entry points include the block system (blox/), layouts, shortcodes, and partials. Data flows from Markdown content and YAML configuration through Hugo's build pipeline, with client-side hydration via Preact. External dependencies include Hugo Extended, Node.js, pnpm, and various CDN-hosted libraries (Reveal.js, Mermaid, KaTeX, etc.). The framework is heavily oriented toward academic and research use cases, with features like BibTeX import, publication pages, and author profiles, but also serves general-purpose sites like SaaS landing pages and documentation.
This repository is a Jupyter Book for the University of Tennessee's GEOG-312 course on geospatial programming. It contains lecture notebooks covering Python fundamentals and geospatial libraries (GeoPandas, Rasterio, Xarray, Leafmap, Geemap, etc.), along with lab exercises. It is primarily an educational resource, not a production application or data pipeline. The content is well-structured with paired Markdown and Jupyter notebook files using jupytext, and includes CI/CD for building the book. While not a reusable software library, the pedagogical patterns and notebook organization are of high quality.
nbconvert is a mature, actively maintained CLI tool and Python library for converting Jupyter Notebooks (.ipynb) into a wide variety of static formats, including HTML, LaTeX, PDF, Markdown, reStructuredText, AsciiDoc, slides (Reveal.js), executable scripts, and even back to notebook format. The architecture is a pipeline: a notebook is first processed by a chain of configurable preprocessors (e.g., for executing code, removing tags, extracting attachments, converting SVG to PDF), then rendered through Jinja2 templates into the target format, and finally written to disk or stdout via a writer, with optional post-processing (e.g., serving slides). The system is highly extensible via Python entry points for custom exporters, preprocessors, and templates, and all components are configurable through traitlets, enabling both programmatic and command-line use. The codebase is well-structured with clear separation of concerns: exporters (base, template-based, and format-specific), preprocessors (over a dozen), filters (for Jinja2 templates), writers, and postprocessors. The template system supports inheritance and custom syntax for LaTeX. Notable implementation highlights include a custom Markdown parser with LaTeX math support (handling both Mistune v2 and v3), ANSI-to-HTML/LaTeX conversion, SVG-to-PDF conversion with security hardening, and a context manager for directory changes. The test suite is extensive, covering exporters, filters, preprocessors, and the CLI application, with CI testing across multiple Python versions and platforms. While the core functionality is robust, the codebase shows signs of technical drift: some modules are deprecated (e.g., lexers.py), external tool dependencies (pandoc, Inkscape, Playwright) introduce version-specific behavior, and the template system has accumulated complexity with multiple inheritance chains and legacy compatibility paths. The project is actively maintained with recent security fixes and releases, but the breadth of supported formats and external dependencies creates ongoing maintenance overhead.
This repository contains a Jupyter notebook that processes Strava data exports to generate interactive running heatmaps. The core functionality involves parsing GPS and sensor data from .fit.gz files, filtering activities by type and date, detecting a home location, rasterizing GPS tracks into multiple value grids (frequency, pace, heart rate, gradient, elevation change), applying Gaussian blurring and normalization, and rendering the results as overlay layers on a Leaflet basemap via Folium. The output is a single HTML file with six toggleable layers showing different aspects of running patterns. The code demonstrates solid data processing techniques including efficient caching, coordinate transformations, and custom visualization logic, though it remains tightly coupled as a monolithic notebook.
This repository is a monorepo implementing a Language Server Protocol (LSP) integration for JupyterLab, consisting of a Python backend (`jupyter-lsp`) and a TypeScript frontend (`jupyterlab-lsp`). The backend manages multiple language server processes (Python, R, Julia, bash, SQL, etc.) via WebSocket proxying, file system shadowing for disk-based servers, and configurable spec discovery. The frontend provides LSP features (autocompletion, diagnostics, hover, jump-to-definition, rename, signature help, syntax highlighting) as JupyterLab plugins, with custom CodeMirror 6 extensions, a diagnostics panel, and a status bar. The architecture includes virtual documents for notebook cells with IPython magic handling, foreign code extraction (e.g., `%%R`, `%%sql`), and a settings schema system that dynamically populates language server configuration UIs. The codebase is well-tested with Python unit tests, TypeScript unit tests, and Robot Framework acceptance tests running in CI. While the core functionality is robust, there is moderate technical drift due to reliance on undocumented JupyterLab internal APIs, fragile regex-based magic parsing, and some deprecated language server specifications.
This repository is a curated list of awesome Jupyter projects, libraries, and resources. It serves as a reference for developers and data scientists seeking tools to enhance their Jupyter experience, including runtimes, visualization libraries, collaboration tools, and more. The list is organized into thematic categories (e.g., Runtimes/Frontends, Collaboration/Education, Visualization) for easy navigation and discovery. The repository includes maintenance workflows to ensure the quality and consistency of the list. A GitHub Actions workflow runs awesome-lint and remark-lint to validate the format of the list and keep it alphabetized. Another workflow tracks GitHub traffic statistics. The project follows standard contribution guidelines and is licensed under Creative Commons Attribution-ShareAlike 4.0. While the repository itself does not contain executable code or data, it provides significant value as a discovery tool for the Jupyter ecosystem. The maintained workflows demonstrate good practices for keeping awesome lists clean and consistent, which can be reused in similar projects. The content is regularly updated, as evidenced by recent commits, indicating active maintenance.
This repository is an educational tutorial for Apache Spark using Scala, designed to teach distributed data processing concepts through practical examples. It functions as a data research and engineering learning resource, featuring Jupyter notebooks and Scala source code that demonstrate RDD operations, SparkSQL, and streaming pipelines. The project includes bundled datasets such as the King James Version Bible and the Enron Spam/Ham email corpus, along with golden output files for test verification. While it serves as a comprehensive guide for Spark development, it primarily focuses on teaching data transformation patterns rather than deploying a production service. The codebase demonstrates solid engineering practices for an educational context, including unit tests, custom utility classes for CLI argument parsing, and structured data schemas. However, the technical stack is significantly outdated, relying on Spark 2.3.0 and Scala 2.11.8, which are no longer supported. The examples frequently use deprecated APIs, such as the RDD-based DStream for streaming, which has been superseded by Structured Streaming in modern Spark versions. This creates a high technical drift, making the code less suitable for new projects without significant refactoring. Despite the outdated dependencies, the repository remains a valuable resource for understanding Spark fundamentals. The inclusion of golden files and test utilities provides a reproducible framework for validating data processing logic. The tutorial structure progresses logically from basic word counts to complex SQL and streaming operations, making it effective for learning. Users should be aware that while the patterns are sound, the specific API implementations will require updates to align with current Spark LTS versions.
This repository provides the `lidar` Python package, a specialized library for hydrological depression analysis of high-resolution topographic data, particularly LiDAR-derived DEMs. Its core functionality includes DEM smoothing (mean, median, Gaussian filters), sink extraction, and a novel level-set algorithm for delineating nested depression hierarchy, yielding topological and geometric properties (size, volume, depth, perimeter). The package is designed as a reusable library with a CLI (`lidar` command), a PySimpleGUI-based graphical interface, and integration with ArcGIS via a dedicated toolbox. It also includes Jupyter notebooks for tutorial and research workflows, demonstrating the complete analysis pipeline. The project is research-grade, backed by peer-reviewed publications, and follows a modular architecture with separate modules for filtering, filling, slicing, and mounts. However, some dependencies (GDAL, arcpy) and Python version constraints introduce moderate technical drift risk.
bertviz is a Python library for interactive visualization of attention in Transformer-based NLP models. It provides three complementary views: Head View (per-head attention patterns), Model View (attention flow between tokens across layers), and Neuron View (query/key interactions within individual attention heads). The library is designed for Jupyter notebooks, rendering D3.js-based interactive HTML directly in cell output. It accepts attention tensors from most HuggingFace models via output_attentions=True, and includes a vendored, modified copy of older HuggingFace transformers code to expose query/key vectors for the Neuron View on BERT, GPT-2, RoBERTa, XLNet, and similar architectures.
This repository serves as a comprehensive guide and utility for managing Visual Studio Code extensions. It provides a curated list of extensions categorized by functionality (themes, productivity, data science, etc.) with descriptions and screenshots. The repository includes a PowerShell script for bulk installing extensions and a set of tools (Python and Bash) for comparing extension lists between different environments. Additionally, it contains a standalone decision tree implementation in Python, which, while not directly related to the main purpose, demonstrates solid algorithmic implementation. The main value lies in the extension recommendations and the practical scripts for managing VS Code setups, making it a useful reference for developers seeking to enhance their coding environment.
jupyter-manim is a Jupyter/IPython extension that provides a `%%manim` cell magic for rendering Manim animations and displaying the resulting video or GIF inline in notebooks. It integrates with the manimlib library (specifically the Cairo backend) to execute scene definitions defined in notebook cells, handles exporting notebook variables to the Manim script via pickling, extracts imports from the notebook's global scope, and captures Manim's output to locate and display the generated media file. The architecture centers around the `ManimMagics` class in `jupyter_manim/__init__.py`, which orchestrates the execution flow: executing cell code for early feedback, preparing a temporary script with exported variables and imports, invoking manimlib.main(), and rendering the output using IPython's display system. While the project is functional and serves its niche purpose well, it shows signs of technical drift through inconsistent versioning between setup.py and __init__.py, lack of testing on modern Python versions (only 3.6 and 3.7 in CI), and an aging Travis CI configuration. The codebase contains several reusable utilities such as StringIOWithCallback for stdout capture, is_pickable for safe serialization checks, find_ipython_frame for accessing the user namespace, and video/gif helpers for media display, which demonstrate solid craftsmanship and could be extracted for use in other Jupyter extensions.