QQN: Quadratic Quasi-Newton Optimization Methods
Comprehensive research on the Quadratic-Quasi-Newton (QQN) algorithm, a novel optimization method that combines gradient descent and quasi-Newton directions through quadratic interpolation. QQN began as an experimental optimizer inside the MindsEye Java deep learning framework, was formally published alongside a purpose-built Rust benchmarking framework, and is now available as reference implementations for three mainstream Python deep learning frameworks (JAX, PyTorch, and TensorFlow). QQN achieves statistically significant dominance across 62 benchmark problems, winning 72.6% of test cases with 50-80% fewer function evaluations than traditional methods.
Key Research Findings
Originated as an experimental orienter in the MindsEye Java framework, then formalized and published with an independent Rust reference implementation
QQN variants won 45 out of 62 benchmark problems (72.6% win rate) with statistical significance (p < 0.001)
QQN-StrongWolfe achieved 100% success rate on challenging Rosenbrock problems vs 0% for most competitors
QQN-GoldenSection demonstrated perfect success on multimodal Rastrigin problems across all dimensions
Theoretical guarantees include global convergence and local superlinear convergence properties
Rust benchmarking framework enables reproducible evaluation with automated statistical analysis and multi-format reporting
Ported to JAX, PyTorch, and TensorFlow as drop-in optimizers, making the method directly usable in mainstream deep learning workflows
Technical Details & Impact
Technologies Used
Research Status
Last updated: 2025
Research Impact
Establishes new standards for optimization algorithm evaluation through a comprehensive Rust benchmarking framework, and delivers a robust general-purpose optimizer — outperforming L-BFGS, Adam, and gradient descent — that is directly adoptable via JAX, PyTorch, and TensorFlow implementations
This research is part of ongoing open-source development. Contributions, discussions, and collaborations are welcome.