Optimization by moving ridge functions: Derivative-free optimization for computationally intensive functions

TL;DR

This paper introduces OMoRF, a derivative-free optimization algorithm that combines trust region methods with output-based dimension reduction, enabling efficient optimization of high-dimensional functions without known low-dimensional structure.

Contribution

The paper presents OMoRF, a novel algorithm that accelerates convergence in high-dimensional derivative-free optimization by dynamically updating a low-dimensional subspace during the trust region process.

Findings

OMoRF performs favorably compared to other derivative-free methods.

It effectively optimizes high-dimensional functions without known low-dimensional structure.

The algorithm requires low computational resources, enabling rapid progress.

Abstract

A novel derivative-free algorithm, optimization by moving ridge functions (OMoRF), for unconstrained and bound-constrained optimization is presented. This algorithm couples trust region methodologies with output-based dimension reduction to accelerate convergence of model-based optimization strategies. The dimension-reducing subspace is updated as the trust region moves through the function domain, allowing OMoRF to be applied to functions with no known global low-dimensional structure. Furthermore, its low computational requirement allows it to make rapid progress when optimizing high-dimensional functions. Its performance is examined on a set of test problems of moderate to high dimension and a high-dimensional design optimization problem. The results show that OMoRF compares favourably to other common derivative-free optimization methods, even for functions in which no underlying global low-dimensional structure is known.