Direct search based on probabilistic descent in reduced spaces
Lindon Roberts, Cl\'ement W. Royer

TL;DR
This paper introduces a probabilistic direct-search algorithm using random subspaces for optimization, providing new complexity guarantees and demonstrating improved performance through numerical experiments.
Contribution
It extends existing direct-search methods by incorporating random subspaces and relaxed direction bounds, leading to enhanced complexity bounds and broader applicability.
Findings
Randomized subspace strategies improve complexity bounds.
Numerical results show benefits of subspace randomization in moderate dimensions.
The approach generalizes previous deterministic methods with probabilistic analysis.
Abstract
In this paper, we study a generic direct-search algorithm in which the polling directions are defined using random subspaces. Complexity guarantees for such an approach are derived thanks to probabilistic properties related to both the subspaces and the directions used within these subspaces. Our analysis crucially extends previous deterministic and probabilistic arguments by relaxing the need for directions to be deterministically bounded in norm. As a result, our approach encompasses a wide range of new optimal polling strategies that can be characterized using our subspace and direction properties. By leveraging results on random subspace embeddings and sketching matrices, we show that better complexity bounds are obtained for randomized instances of our framework. A numerical investigation confirms the benefit of randomization, particularly when done in subspaces, when solving…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Robotics and Sensor-Based Localization · Diffusion and Search Dynamics
