A trust-region framework for optimization using Hermite kernel surrogate models

TL;DR

This paper introduces a trust-region optimization method that uses Hermite kernel surrogate models with gradient information, improving efficiency for expensive objective functions in medium to high dimensions.

Contribution

It presents a novel trust-region framework employing Hermite kernel surrogates with convergence guarantees, suitable for high-dimensional problems with costly evaluations.

Findings

Demonstrates convergence to stationary points.

Shows efficiency gains over direct optimization.

Validates effectiveness through numerical experiments.

Abstract

In this work, we present a trust-region optimization framework that employs Hermite kernel surrogate models. The method targets optimization problems with computationally demanding objective functions, for which direct optimization is often impractical due to expensive function evaluations. To address these challenges, we leverage a trust-region strategy, where the objective function is approximated by an efficient surrogate model within a local neighborhood of the current iterate. In particular, we construct the surrogate using Hermite kernel interpolation and define the trust-region based on bounds for the interpolation error. As mesh-free techniques, kernel-based methods are naturally suited for medium- to high-dimensional problems. Furthermore, the Hermite formulation incorporates gradient information, enabling precise gradient estimates that are crucial for many optimization algorithms. We prove that the proposed algorithm converges to a stationary point, and we demonstrate its effectiveness through numerical experiments, which illustrate the convergence behavior as well as the efficiency gains compared to direct optimization.