drawio-skill is a comprehensive toolkit for generating, manipulating, validating, and publishing draw.io architecture diagrams from natural language, codebases, infrastructure-as-code, API specifications, and other structured sources. It provides a unified CLI (`diagramctl.py`) and an optional MCP server (`diagramctl_mcp.py`) that expose workflows for building, syncing, viewing, testing, reviewing, querying, what-if analysis, story generation, and publishing diagrams. The system uses a formal Diagram Intermediate Representation (IR) to separate semantic meaning from geometry, enabling incremental reconciliation, multi-view projection, architecture contract enforcement, and accessible HTML story output. It includes 15+ importers for languages (Python, JS, Go, Rust), IaC (Terraform, Kubernetes, Docker Compose), schemas (SQL, OpenAPI, AsyncAPI, Protobuf, GraphQL), and live infrastructure (Docker, Terraform state). The project is packaged as an AI agent skill compatible with Claude Code, Cursor, Copilot, and other Agent Skills hosts, and ships with GitHub Actions for CI integration. While primarily an application, it also bundles curated icon indices and style presets that could be considered data assets.
Petastorm is a library for loading Apache Parquet datasets into machine learning training frameworks, primarily TensorFlow and PyTorch. It provides a reader API that supports parallel I/O, row-group shuffling, predicate filtering, and on-the-fly transformations. The library also includes a Spark Dataset Converter to materialize Spark DataFrames as cached Parquet for training. Architecture: modular with worker pools (process/thread), filesystem abstraction (local, HDFS, S3, GCS), schema system (Unischema with codecs), and shuffling buffers. It features CLI tools for benchmarking, metadata generation, and dataset copying. The codebase demonstrates strong engineering practices with clear separation of concerns, but shows signs of aging: reliance on TensorFlow 1.x, Python 2 compatibility, and pinned dependencies that may cause drift. The project has a large test suite with legacy dataset fixtures for backward compatibility, but maintenance has slowed significantly with the last release (v0.13.1) and minimal recent commits.