Sobolev Seminorm of Quadratic Functions with Applications to Derivative-Free Optimization

TL;DR

This paper analyzes the Sobolev seminorm of quadratic functions to improve understanding and performance of derivative-free optimization methods, particularly in least-norm interpolation and Broyden updates.

Contribution

It provides explicit formulas for the Sobolev seminorm of quadratic functions and applies these insights to enhance derivative-free optimization techniques.

Findings

Explicit formula for Sobolev seminorm in terms of Hessian and gradient

Improved performance of Broyden update using the new theory

Proposed a new method for comparing derivative-free solvers

Abstract

This paper studies the $H^1$ Sobolev seminorm of quadratic functions. The research is motivated by the least-norm interpolation that is widely used in derivative-free optimization. We express the $H^1$ seminorm of a quadratic function explicitly in terms of the Hessian and the gradient when the underlying domain is a ball. The seminorm gives new insights into least-norm interpolation. It clarifies the analytical and geometrical meaning of the objective function in least-norm interpolation. We employ the seminorm to study the extended symmetric Broyden update proposed by Powell. Numerical results show that the new thoery helps improve the performance of the update. Apart from the theoretical results, we propose a new method of comparing derivative-free solvers, which is more convincing than merely counting the numbers of function evaluations.