A Model-Based Derivative-Free Optimization Algorithm for Partially Separable Problems

TL;DR

UPOQA is a novel derivative-free optimization algorithm designed for partially separable problems, utilizing quadratic interpolation and structured trust regions to improve efficiency and performance in high-precision scenarios.

Contribution

The paper introduces UPOQA, a new derivative-free optimization method that exploits partial separability with innovative models and trust-region strategies, enhancing efficiency over existing methods.

Findings

UPOQA reduces function evaluations significantly on benchmark problems.

The speed-up is more pronounced in high-precision optimization scenarios.

Applications to quantum variational problems demonstrate practical utility.

Abstract

We propose UPOQA, a derivative-free optimization algorithm for partially separable unconstrained problems, leveraging quadratic interpolation and a structured trust-region framework. By decomposing the objective into element functions, UPOQA constructs underdetermined element models and solves subproblems efficiently via a modified projected gradient method. Innovations include an approximate projection operator for structured trust regions, improved management of elemental radii and models, a starting point search mechanism, and support for hybrid black-white-box optimization, etc. Numerical experiments on 85 CUTEst problems demonstrate that \texttt{UPOQA} can significantly reduce the number of function evaluations. To quantify the impact of exploiting partial separability, we introduce the speed-up profile to further evaluate the acceleration effect. Results show that the speed-up of UPOQA over baselines is less significant in low-precision scenarios but becomes more pronounced in high-precision scenarios. Applications to quantum variational problems further validate its practical utility.