Direct search based on probabilistic descent in reduced spaces

TL;DR

This paper introduces a probabilistic direct-search algorithm using random subspaces for optimization, providing new complexity guarantees and demonstrating improved performance through numerical experiments.

Contribution

It extends existing direct-search methods by incorporating random subspaces and relaxed direction bounds, leading to enhanced complexity bounds and broader applicability.

Findings

Randomized subspace strategies improve complexity bounds.

Numerical results show benefits of subspace randomization in moderate dimensions.

The approach generalizes previous deterministic methods with probabilistic analysis.

Abstract

In this paper, we study a generic direct-search algorithm in which the polling directions are defined using random subspaces. Complexity guarantees for such an approach are derived thanks to probabilistic properties related to both the subspaces and the directions used within these subspaces. Our analysis crucially extends previous deterministic and probabilistic arguments by relaxing the need for directions to be deterministically bounded in norm. As a result, our approach encompasses a wide range of new optimal polling strategies that can be characterized using our subspace and direction properties. By leveraging results on random subspace embeddings and sketching matrices, we show that better complexity bounds are obtained for randomized instances of our framework. A numerical investigation confirms the benefit of randomization, particularly when done in subspaces, when solving problems of moderately large dimension.