Rover Descent: Learning to optimize by learning to navigate on prototypical loss surfaces

TL;DR

This paper introduces Rover Descent, a novel learning-based optimization method that navigates loss surfaces using only zero-order information, achieving state-of-the-art results on complex functions without prior gradient knowledge.

Contribution

It presents a new approach to learning optimization policies through navigation on prototypical loss surfaces, effective even in high dimensions with limited local information.

Findings

Achieves state-of-the-art convergence on complex 2D functions like Rosenbrock.

Successfully generalizes to unseen optimization problems without gradient access.

Extends to high-dimensional landscapes with promising preliminary results.

Abstract

Learning to optimize - the idea that we can learn from data algorithms that optimize a numerical criterion - has recently been at the heart of a growing number of research efforts. One of the most challenging issues within this approach is to learn a policy that is able to optimize over classes of functions that are fairly different from the ones that it was trained on. We propose a novel way of framing learning to optimize as a problem of learning a good navigation policy on a partially observable loss surface. To this end, we develop Rover Descent, a solution that allows us to learn a fairly broad optimization policy from training on a small set of prototypical two-dimensional surfaces that encompasses the classically hard cases such as valleys, plateaus, cliffs and saddles and by using strictly zero-order information. We show that, without having access to gradient or curvature information, we achieve state-of-the-art convergence speed on optimization problems not presented at training time such as the Rosenbrock function and other hard cases in two dimensions. We extend our framework to optimize over high dimensional landscapes, while still handling only two-dimensional local landscape information and show good preliminary results.