bae is a PyTorch library for exact second-order optimization, specializing in Bundle Adjustment (BA) and Pose Graph Optimization (PGO) for robotics and 3D vision. It provides a sparse autograd system that automatically computes sparse block Jacobians by tracing operations on tracked parameters, enabling efficient Levenberg-Marquardt and Schur complement optimizers. The library includes CUDA-accelerated sparse matrix operations via cuSPARSE and Warp, ambient gradient implementations for SE(3) operations that are torch.compile compatible, and utilities for loading BAL and PGO datasets. It is designed as a research library with clean abstractions for defining residual graphs and optimizing over Lie groups. The architecture separates autograd tracing (bae/autograd), optimization algorithms (bae/optim), sparse matrix utilities (bae/sparse), and supporting utilities (bae/utils). The sparse autograd system uses a custom Tensor subclass (TrackingTensor) that records index, map, and cat operations via __torch_function__, then computes sparse BSR Jacobians via a backward traversal. The LM and Schur optimizers support trust region and adaptive strategies, with PCG as the default linear solver. The library also provides monkeypatches for PyPose LieTensor to enable torch.compile full-graph capture, and ambient (plain PyTorch) implementations of SE(3) operations for gradient compatibility.