Towards Robust Learning to Optimize with Theoretical Guarantees

TL;DR

This paper provides the first theoretical analysis of learning to optimize (L2O), establishing conditions for robustness and demonstrating how input features affect OOD performance, along with a new model that outperforms existing methods.

Contribution

It offers the first theoretical guarantees for robust L2O models and introduces a novel gradient-based feature construction and history modeling approach.

Findings

Proves a sufficient condition for robustness of L2O over all In-Distribution instances.

Shows that OOD convergence rate deteriorates based on input features.

Demonstrates up to 10× convergence speedup over state-of-the-art methods.

Abstract

Learning to optimize (L2O) is an emerging technique to solve mathematical optimization problems with learning-based methods. Although with great success in many real-world scenarios such as wireless communications, computer networks, and electronic design, existing L2O works lack theoretical demonstration of their performance and robustness in out-of-distribution (OOD) scenarios. We address this gap by providing comprehensive proofs. First, we prove a sufficient condition for a robust L2O model with homogeneous convergence rates over all In-Distribution (InD) instances. We assume an L2O model achieves robustness for an InD scenario. Based on our proposed methodology of aligning OOD problems to InD problems, we also demonstrate that the L2O model's convergence rate in OOD scenarios will deteriorate by an equation of the L2O model's input features. Moreover, we propose an L2O model with a concise gradient-only feature construction and a novel gradient-based history modeling method. Numerical simulation demonstrates that our proposed model outperforms the state-of-the-art baseline in both InD and OOD scenarios and achieves up to 10 $\times$ convergence speedup. The code of our method can be found from https://github.com/NetX-lab/GoMathL2O-Official.