$ show JuliaSmoothOptimizers/LinearOperators.jl
LinearOperators.jl is a Julia library that provides a framework for representing and manipulating linear operatorsβobjects that behave like matrices but are defined by their effect on vectors. It enables lazy composition, transposition, conjugation, and arithmetic (addition, multiplication, scaling) of operators, deferring costly computations until application to a vector. The library includes a rich set of built-in operators: identity, zero, ones, diagonal, restriction/extension, Householder, Hermitian, inverse (via Cholesky, LDL, or general solve), and limited-memory quasi-Newton approximations (L-BFGS and L-SR1) commonly used in optimization. It also provides utility functions for norm estimation, Hermiticity checks, and solving shifted linear systems. The architecture is modular, with a core abstract type hierarchy, separate files for operations, adjoint/transpose wrappers, concatenation, Kronecker product, and timed operators. Extensions support GPU backends (CUDA, AMDGPU, Metal) and automatic differentiation (ChainRulesCore). The code is well-tested, with comprehensive unit tests covering correctness, allocation-free paths, and edge cases. It is primarily used within the JuliaSmoothOptimizers ecosystem for smooth optimization.
The LLM identified 5 files worth highlighting for reuse:
Implements limited-memory BFGS (L-BFGS) operator in both forward and inverse forms, with optional damping and scaling. Uses a two-loop recursion for efficient multiplication, tracks an upper bound on the operator norm, and provides push!/reset! for incremental updates. The implementation is allocation-free after warm-up and demonstrates careful memory management and numerical robustness.
Implements a shifted linear operator representing H + ΟI, where H is any linear operator and Ο is a scalar (possibly complex). Correctly propagates symmetry and hermiticity based on H and Ο, and provides efficient forward, transpose, and adjoint products. The design cleanly separates data (the shift and underlying operator) from product closures, making it easy to reuse or modify.
Provides four diagonal quasi-Newton Hessian approximations: DiagonalPSB, DiagonalAndrei, SpectralGradient, and DiagonalBFGS. Each satisfies the weak secant equation and is updated in-place with zero allocations. The implementations are self-contained, demonstrate efficient element-wise operations, and serve as reusable patterns for diagonal curvature approximations.
Implements the Kronecker tensor product as a linear operator, supporting combinations of operators and matrices. Uses reshape and matrix multiplication to compute (A β B) * vec(X) = vec(B X Aα΅). The implementation is concise, handles type promotion, and correctly computes transpose and adjoint products.
Defines lazy wrapper types (AdjointLinearOperator, TransposeLinearOperator, ConjugateLinearOperator) that defer the actual transposition/conjugation until multiplication. Correctly propagates size, symmetry, hermiticity, and product counters, and infers missing product functions from available ones. The pattern is a clean example of the decorator pattern in Julia.