Crawlee is a comprehensive web scraping and crawling library for Node.js/TypeScript that provides a unified interface for both HTTP-based crawling (using got-scraping with Cheerio/JSDOM/Linkedom) and browser-based crawling (using Playwright/Puppeteer). It features a modular architecture with separate packages for core functionality, specific crawlers, storage abstractions, browser pooling, and utilities. The library handles request queuing, automatic concurrency scaling based on system resources, session management with cookie persistence and fingerprinting, proxy rotation, and robust error handling. It includes built-in mechanisms to avoid bot detection and provides storage abstractions for datasets (structured data), key-value stores (binary data), and request queues (URL frontier). Designed for reliability and scalability, Crawlee supports both local execution and deployment on the Apify platform, with extensive documentation, examples, and CLI tools for project scaffolding.
This repository provides Docker images and Kubernetes Helm charts for deploying Selenium Grid, the industry-standard browser automation framework. It covers all major browser backends (Chrome, Firefox, Edge, Chromium) and supports multiple deployment modes: standalone single-container, hub-and-node, fully distributed grid, and dynamic grid with Docker-in-Docker or Kubernetes backends. The architecture follows a microservice pattern where each Grid component (Router, Distributor, Session Queue, Nodes, etc.) runs in its own container, orchestrated by supervisord. Beyond core Grid images, the repo includes a comprehensive monitoring stack with a Go-based Prometheus exporter that scrapes the Grid GraphQL endpoint, pre-built Grafana dashboards, and a KEDA external scaler for Kubernetes native autoscaling. Video recording and upload services are provided as sidecar containers, with both event-driven (ZeroMQ) and polling-based implementations. The CI/CD pipeline is extensive, with GitHub Actions workflows for building multi-architecture images (amd64/arm64), running integration tests, scanning for vulnerabilities, and publishing to Docker Hub and GHCR. The codebase is well-tested with unit tests for Go components, Python integration tests for autoscaling, and Java/Playwright browser tests. Despite its complexity, the repository demonstrates solid engineering practices with clear separation of concerns, reusable shell libraries, and automated dependency management via Renovate and Dependabot.
AIBrix is a Kubernetes-native control plane and intelligent gateway for serving production LLM workloads. The repository implements two intertwined planes: a data plane composed of Envoy ext_proc gateway plugins (cmd/plugins, pkg/plugins/gateway) that intercept OpenAI-compatible inference requests and route them to vLLM, SGLang, or TensorRT-LLM replicas using algorithms such as least-load, prefix-cache-aware routing, SLO queueing, token-load tracking, and prefill/decode disaggregation; and a control plane of Kubernetes controllers (cmd/controllers) that reconcile custom resources for autoscaling (PodAutoscaler), LoRA adapters, model warmup/claims, KV caches, RayCluster fleets, and StormService/RoleSet orchestration. Supporting runtime code includes a Python metadata and batch-job service, a distributed KV-cache package, a GPU optimizer, and an OpenAI-compatible frontend.
Custom Pod Autoscaler is a Kubernetes operator that provides custom autoscaling functionality. It runs as a pod in a Kubernetes cluster, periodically gathering metrics (via custom scripts or Kubernetes metrics server), evaluating those metrics to determine desired replica count, and scaling target resources accordingly. The application features a REST API for manual interaction, supports hooks for custom logic at various points in the process, and is designed to work with any language or framework through shell commands or HTTP requests. The architecture follows clean separation of concerns with distinct packages for configuration loading, execution abstraction, metric gathering, evaluation, and scaling logic.
This is the Kubernetes Autoscaler monorepo, containing multiple production-grade autoscaling components for Kubernetes: Cluster Autoscaler (horizontal node scaling), Vertical Pod Autoscaler (vertical pod resource scaling), Addon Resizer (resource adjustment for addons), and Balancer (workload distribution). The architecture is a collection of independent Go applications and libraries, each with its own entry points (main.go) and Kubernetes API extensions. The codebase is heavily multi-cloud, with vendored SDKs for over 15 cloud providers (AWS, GCE, Azure, OCI, Huawei, Alicloud, etc.) and extensive generated client code for custom CRDs. The core autoscaling logic is well-engineered, but the repository is bloated with auto-generated SDKs and boilerplate, leading to moderate technical drift risk. It is actively maintained with frequent commits and CI pipelines.
This repository implements a threshold-based vertical autoscaling system for Proxmox VE virtual machines. It runs as a systemd service that periodically polls VM CPU and RAM usage via SSH commands (qm, pvesh) and adjusts CPU cores and RAM by 1 core or 512 MB when configurable high/low thresholds are crossed. The system includes host resource safety limits, per-VM threshold overrides, independent cooldowns for CPU and RAM, and optional billing tracking with CSV export and webhook notifications. It also provides a lightweight Prometheus metrics endpoint built on stdlib, supports Gotify and SMTP notifications, and offers a dry-run mode for safe evaluation. The architecture is a single-threaded polling loop with in-memory state, designed for simplicity and reliability rather than horizontal scaling or predictive analytics. The codebase is well-tested with over 290 unit tests and end-to-end tests against real Proxmox nodes, and includes comprehensive documentation via a VitePress site. Key reusable components include a generic SSH client with retry/backoff and host key policies, a thread-safe Prometheus metrics registry without external dependencies, and a configuration validation module that reports all errors at once.