A Riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraints

TL;DR

This paper introduces a Riemannian low-rank optimization algorithm for semidefinite problems with block-diagonal constraints, demonstrating significant improvements in speed and accuracy over existing methods on relevant estimation and combinatorial problems.

Contribution

The paper presents a novel Riemannian low-rank method that efficiently solves semidefinite programs with block-diagonal constraints, leveraging geometric insights for better performance.

Findings

The staircase method outperforms state-of-the-art software in speed and accuracy.

The approach scales well to large problem instances.

Second-order critical points correspond to KKT points under certain conditions.

Abstract

We propose a new algorithm to solve optimization problems of the form $\min f(X)$ for a smooth function $f$ under the constraints that $X$ is positive semidefinite and the diagonal blocks of $X$ are small identity matrices. Such problems often arise as the result of relaxing a rank constraint (lifting). In particular, many estimation tasks involving phases, rotations, orthonormal bases or permutations fit in this framework, and so do certain relaxations of combinatorial problems such as Max-Cut. The proposed algorithm exploits the facts that (1) such formulations admit low-rank solutions, and (2) their rank-restricted versions are smooth optimization problems on a Riemannian manifold. Combining insights from both the Riemannian and the convex geometries of the problem, we characterize when second-order critical points of the smooth problem reveal KKT points of the semidefinite problem. We compare against state of the art, mature software and find that, on certain interesting problem instances, what we call the staircase method is orders of magnitude faster, is more accurate and scales better. Code is available.