Neural Fixed-Point Acceleration for Convex Optimization

TL;DR

This paper introduces a neural network-based method to accelerate fixed-point iterations in convex optimization, improving efficiency and stability for real-time applications by learning from problem distributions.

Contribution

It proposes a novel neural fixed-point acceleration framework that combines meta-learning with classical methods, applicable to any CVXPY-expressible optimization problem.

Findings

Achieves faster convergence in convex cone programming problems.

Overcomes challenges of unrolled optimization and acceleration instabilities.

Demonstrates applicability to a wide range of convex optimization problems.

Abstract

Fixed-point iterations are at the heart of numerical computing and are often a computational bottleneck in real-time applications that typically need a fast solution of moderate accuracy. We present neural fixed-point acceleration which combines ideas from meta-learning and classical acceleration methods to automatically learn to accelerate fixed-point problems that are drawn from a distribution. We apply our framework to SCS, the state-of-the-art solver for convex cone programming, and design models and loss functions to overcome the challenges of learning over unrolled optimization and acceleration instabilities. Our work brings neural acceleration into any optimization problem expressible with CVXPY. The source code behind this paper is available at https://github.com/facebookresearch/neural-scs