Learning to Optimize

TL;DR

This paper introduces a reinforcement learning-based method to automatically discover optimization algorithms, outperforming traditional hand-designed algorithms in convergence speed and final objective value.

Contribution

It presents the first approach to automatically learn optimization algorithms using reinforcement learning, reducing manual effort in algorithm design.

Findings

Learned algorithms outperform hand-engineered ones in convergence speed

The method demonstrates improved final objective values

Reinforcement learning effectively discovers novel optimization strategies

Abstract

Algorithm design is a laborious process and often requires many iterations of ideation and validation. In this paper, we explore automating algorithm design and present a method to learn an optimization algorithm, which we believe to be the first method that can automatically discover a better algorithm. We approach this problem from a reinforcement learning perspective and represent any particular optimization algorithm as a policy. We learn an optimization algorithm using guided policy search and demonstrate that the resulting algorithm outperforms existing hand-engineered algorithms in terms of convergence speed and/or the final objective value.