Automatic Differentiation Tools in Optimization Software
Jorge J. Mor\'e
TL;DR
This paper explores the integration of automatic differentiation tools in optimization software, focusing on large-scale problems and efficient computation of gradients and Hessians for partially separable functions.
Contribution
It demonstrates how automatic differentiation can be effectively used to compute derivatives with guaranteed bounds in time and memory for large-scale optimization.
Findings
Automatic differentiation enables efficient gradient and Hessian computation.
Guaranteed bounds in time and memory are achievable for large-scale problems.
Focus on partially separable functions enhances optimization performance.
Abstract
We discuss the role of automatic differentiation tools in optimization software. We emphasize issues that are important to large-scale optimization and that have proved useful in the installation of nonlinear solvers in the NEOS Server. Our discussion centers on the computation of the gradient and Hessian matrix for partially separable functions and shows that the gradient and Hessian matrix can be computed with guaranteed bounds in time and memory requirements
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Optimization Algorithms Research · Matrix Theory and Algorithms · Sparse and Compressive Sensing Techniques