Relationships between full-space and subspace quadratic interpolation models and simplex derivatives

TL;DR

This paper explores the theoretical relationships between full-space and subspace quadratic models and simplex derivatives, providing formulas and insights crucial for high-dimensional derivative-free optimization.

Contribution

It derives explicit conversion formulas between full-space and subspace models, enhancing understanding of subspace approximation techniques in derivative-free optimization.

Findings

Models coincide on the affine subspace

Models align along orthogonal complement directions

Provides a theoretical framework for subspace approximation

Abstract

Quadratic interpolation models and simplex derivatives are fundamental tools in numerical optimization, particularly in derivative-free optimization. When constructed in suitably chosen affine subspaces, these tools have been shown to be especially effective for high-dimensional derivative-free optimization problems, where full-space model construction is often impractical. In this paper, we analyze the relationships between full-space and subspace formulations of these tools. In particular, we derive explicit conversion formulas between full-space and subspace models, including minimum-norm models, minimum Frobenius norm models, least Frobenius norm updating models, as well as models constructed via generalized simplex gradients and Hessians. We show that the full-space and subspace models coincide on the affine subspace and, in general, along directions in the orthogonal complement. Overall, our results provide a theoretical framework for understanding subspace approximation techniques and offer insight into the design and analysis of derivative-free optimization methods.