Revisiting Spectral Bundle Methods: Primal-dual (Sub)linear Convergence Rates

TL;DR

This paper analyzes the spectral bundle method for large-scale SDPs, showing it achieves sublinear convergence generally and linear convergence under certain structural conditions, with practical speedups demonstrated in large problems.

Contribution

It establishes sublinear and linear convergence rates for the spectral bundle method under new conditions, and demonstrates its efficiency with matrix sketching on large SDPs.

Findings

Achieves sublinear convergence under strong duality.

Attains linear convergence when strict complementarity and rank conditions hold.

Solves large SDPs with billions of variables in minutes using sketching.

Abstract

The spectral bundle method proposed by Helmberg and Rendl is well established for solving large-scale semidefinite programs (SDP) thanks to its low per iteration computational complexity and strong practical performance. In this paper, we revisit this classic method showing it achieves sublinear convergence rates in terms of both primal and dual SDPs under merely strong duality, complementing previous guarantees on primal-dual convergence. Moreover, we show the method speeds up to linear convergence if (1) structurally, the SDP admits strict complementarity, and (2) algorithmically, the bundle method captures the rank of the optimal solutions. Such complementary and low rank structure is prevalent in many modern and classical applications. The linear convergent result is established via an eigenvalue approximation lemma which might be of independent interest. Numerically, we confirm our theoretical findings that the spectral bundle method, for modern and classical applications, speeds up under these conditions. Finally, we show that the spectral bundle method combined with a recent matrix sketching technique is able to solve an SDP with billions of decision variables in a matter of minutes.