ojAlgo is a pure Java numerical computing library providing high-performance linear algebra (dense and sparse matrices, decompositions like LU, QR, SVD, Cholesky, eigenvalue), optimization (linear programming via simplex, convex QP via active-set/ADMM, integer programming with branch-and-cut), neural networks, statistical distributions, random processes, time series analysis, clustering, financial portfolio models, and special functions. It has zero external dependencies and is modularized via JPMS. The architecture is a classic library: users create matrix objects or optimization models, invoke solvers or decompositions, and retrieve results. Key entry points include `MatrixStore`, `ExpressionsBasedModel`, `ArtificialNeuralNetwork`, and various decomposition factories. The codebase is well-tested with extensive unit and regression tests, and demonstrates strong engineering practices such as interface-driven design, factory patterns, and careful numerical stability. While primarily a software library, it also contains data analysis and research-oriented components (clustering, financial series, random processes), making it a hybrid but dominantly an application library.
This repository provides a PyTorch implementation of Preconditioned Stochastic Gradient Descent (PSGD), a general-purpose second-order optimizer. It implements various preconditioner structures including Kronecker product, low-rank approximation, dense matrix, affine group, and X-matrix preconditioners. The code offers both Newton-type and whitening preconditioners with multiple update rules for the preconditioner matrices. Key features include support for different geometries (Q0.5EQ1.5, EQ, QEP, etc.), momentum handling, gradient clipping, and numerical stability mechanisms. The library includes optimizers suitable for various deep learning applications as demonstrated in the provided examples (vision transformers, GPT2, LSTMs, CNNs, etc.).
Uno is a modular C++ solver for nonlinearly constrained optimization (NLP, QP, LP). It implements a unification framework that combines interchangeable algorithmic ingredients—constraint relaxation strategies, inequality handling methods, Hessian models (exact, quasi-Newton, identity, zero), inertia correction, subproblem solvers (QP/LP), and globalization mechanisms (line search, trust region, filter, funnel). The solver supports both sequential quadratic programming (SQP) and interior-point methods, and can be configured via presets that mimic established solvers like filterSQP and IPOPT. The architecture is a library with a main orchestration loop in `Uno.cpp`. The user defines the optimization problem through callback functions (objective, gradient, Jacobian, Hessian) via a C API, which is then wrapped by interfaces for AMPL, Fortran, Julia, and Python. Internally, the solver builds a regularized KKT system, delegates subproblem solves to external libraries (HiGHS, BQPD, HSL MA27/57/86, MUMPS, SPRAL), and applies globalization to ensure convergence. The codebase is heavily templated and uses expression templates for linear algebra, factory patterns for component selection, and runtime dynamic loading for optional dependencies. Key entry points include `uno_optimize()` (C API), `UnoSolverWrapper::optimize()` (Python), and the AMPL executable `uno_ampl`. The solver is used in research and production environments via JuMP.jl, CasADi, pyOptSparse, and OpenMDAO. External dependencies include BLAS/LAPACK, HSL (proprietary), MUMPS, SPRAL, HiGHS, BQPD, and AMPL ASL. The project is actively maintained with extensive CI, cross-platform builds, and a growing ecosystem of language bindings.
CppNumericalSolvers is a header-only C++17 library for numerical optimization that provides a suite of modern solvers for both unconstrained and constrained problems. The library features a sophisticated expression template system for building complex objective functions, implementations of popular algorithms like L-BFGS, BFGS, and Newton's method, and advanced constrained optimization via the Augmented Lagrangian method. It uses Eigen3 as its sole dependency for linear algebra operations and is designed for easy integration into other projects. The architecture is modular with separate components for function interfaces, solvers, line searches, and utilities, all leveraging C++ templates for flexibility and performance.
ExprTk is a header-only C++ library for runtime mathematical expression parsing and evaluation. It provides a complete system for defining mathematical expressions as strings, compiling them into an internal AST representation, and evaluating them with variable values. The library supports scalar, vector, and string types, arithmetic operations, trigonometric functions, logarithms, control structures (if, for, while, switch), user-defined functions, and assignment operations within expressions. It employs sophisticated parsing techniques including constant folding, strength reduction optimizations, and runtime safety checks for vector bounds and loop execution time. The architecture consists of three main components: symbol table (for variables/constants/functions), expression (holding the parsed AST), and parser (compiling expression strings).