$ show commaai/openpilot
openpilot is a full-stack autonomous driver assistance system that runs on embedded Linux hardware (comma four/three) and controls 300+ car models. It is a safety-critical, real-time system written primarily in Python and C/C++, with a custom messaging layer (cereal/msgq) and a Cap'n Proto-based schema for all inter-process communication. The architecture is a collection of independent processes (controlsd, plannerd, radard, modeld, locationd, calibrationd, dmonitoringd, etc.) communicating via a pub/sub message queue, managed by a central process manager. The system ingests CAN bus data, camera streams (road, wide-road, driver), IMU/GPS sensor data, and runs neural network models (ONNX/tinygrad) for perception and planning. The codebase demonstrates strong engineering discipline: a comprehensive process replay regression testing framework that replays logged routes through individual processes and compares outputs bit-for-bit; a safety model implemented in the panda firmware (separate submodule) with ISO 26262 guidelines; real-time scheduling and CPU affinity management across processes; and a sophisticated build system using SCons with Cython compilation. The UI is built on raylib (with a Qt-based Cabana tool for CAN analysis), and there's a MetaDrive simulator bridge for development. Key architectural patterns include: a generic car interface abstraction (opendbc) that allows supporting 300+ car models through fingerprinting and parameter configuration; a model-driven control pipeline where neural network outputs (desired curvature, lead vehicle trajectories) feed into lateral controllers (PID, torque-based, curvature-based, angle-based) and longitudinal MPC planners; an EKF-based sensor fusion system (locationd) combining GPS, IMU, and vehicle odometry; and a driver monitoring system using a separate neural network for distraction detection. The system uses a parameter store (params) backed by a filesystem for persistent configuration. The repository includes extensive tooling: replay infrastructure for regression testing, PlotJuggler/Cabana integration for data visualization, automated car porting tools with fuzzy fingerprinting, release automation with stripped builds, and a comprehensive CI/CD pipeline with GitHub Actions. The codebase is actively maintained with daily commits, a structured release process (release-mici/release-tizi branches), and active community contribution through Discord and GitHub issues.
The LLM identified 8 files worth highlighting for reuse:
A sophisticated process replay regression testing framework that replays recorded driving routes through individual processes and compares outputs against reference logs. Supports whitelisting/blacklisting processes and cars, ignoring specific fields, and updating references. This is a genuinely reusable pattern for testing deterministic real-time systems - any system that processes time-series data could adapt this approach. The architecture allows replaying processes in isolation with mocked inputs, making it valuable for CI/CD pipelines.
Implements a torque-based lateral controller with a novel 'live torque' estimation approach that learns the vehicle's steering torque characteristics in real-time. Uses a steering torque estimator (torqued) that adapts to the specific vehicle's EPS system. The controller includes saturation detection, integrator windup protection, and a steering delay compensation buffer. This demonstrates advanced vehicle dynamics control with online parameter estimation - reusable for any steer-by-wire system.
A comprehensive coordinate transformation library handling conversions between geodetic, ECEF, NED, device, calibrated, car, view, and camera reference frames. Uses numpy vectorization with a clever `numpy_wrap` decorator to handle both single values and arrays. Includes quaternion, rotation matrix, and Euler angle conversions with proper conventions. This is a self-contained, well-documented module that could be extracted for any robotics or computer vision project needing coordinate frame transformations.
A robust log reader for Cap'n Proto serialized messages with support for multiple compression formats (bz2, zstd), remote URLs with caching, segment range selectors, and automatic format detection. Implements a clean iterator protocol and handles corrupted logs gracefully. The segment range parsing (e.g., 'route/1:5/q') is a nice DSL for selecting log subsets. Reusable for any Cap'n Proto-based data pipeline.
Implements an Extended Kalman Filter for fusing GPS, IMU, wheel odometry, and camera odometry into a unified pose estimate. Uses rednose for automatic filter generation and includes careful handling of coordinate frames, time delays, and sensor dropout. The EKF structure with separate prediction and update steps, plus the live calibration handling, demonstrates production-grade state estimation. The pattern of using code generation (rednose) for the Kalman filter math is notable.
A streaming Zstd file writer with an input cache that batches writes for efficiency. Handles proper stream finalization, error checking, and resource cleanup. The pattern of caching input data until reaching the compression stream's input size threshold is a useful optimization for high-throughput logging. Self-contained and reusable for any C++ application needing compressed file output.
A feature-rich PID controller with gain scheduling (interpolating PID gains based on speed), feedforward, integrator windup protection that respects output limits, and derivative term support. The gain scheduling via interpolation tables is particularly useful for nonlinear systems like vehicles where controller dynamics change with operating conditions. Clean, well-tested implementation reusable for any control application.
Handles migration of Cap'n Proto messages between schema versions, ensuring backward compatibility as the messaging spec evolves. This is critical for a system with logged data spanning years. The migration framework allows renaming fields, adding new fields with defaults, and restructuring messages while maintaining the ability to read old logs. A reusable pattern for any long-lived system using serialized data.