Information Preserving Line Search via Bayesian Optimization

TL;DR

This paper introduces a Bayesian optimization-based line search method that preserves and utilizes more information during optimization, leading to guaranteed convergence and improved performance over traditional methods.

Contribution

It presents a novel line search approach using Bayesian optimization that retains valuable information typically discarded, enhancing optimization efficiency and reliability.

Findings

Guaranteed convergence of the proposed method

Superior empirical performance on benchmark problems

Effective utilization of discarded information in line search

Abstract

Line search is a fundamental part of iterative optimization methods for unconstrained and bound-constrained optimization problems to determine suitable step lengths that provide sufficient improvement in each iteration. Traditional line search methods are based on iterative interval refinement, where valuable information about function value and gradient is discarded in each iteration. We propose a line search method via Bayesian optimization, preserving and utilizing otherwise discarded information to improve step-length choices. Our approach is guaranteed to converge and shows superior performance compared to state-of-the-art methods based on empirical tests on the challenging unconstrained and bound-constrained optimization problems from the CUTEst test set.