A Semidefinite Relaxation for Sums of Heterogeneous Quadratic Forms on the Stiefel Manifold

TL;DR

This paper introduces a semidefinite relaxation approach for optimizing sums of heterogeneous quadratic forms on the Stiefel manifold, providing theoretical guarantees and practical validation for global optimality in signal processing applications.

Contribution

The work presents a novel SDP relaxation for a nonconvex problem on the Stiefel manifold, including a dual certificate for global optimality and conditions for relaxation tightness.

Findings

Global optimality certificate via dual certificate.

Relaxation reduces to an assignment LP for jointly diagonalizable problems.

Numerical validation confirms theoretical guarantees.

Abstract

We study the maximization of sums of heterogeneous quadratic forms over the Stiefel manifold, a nonconvex problem that arises in several modern signal processing and machine learning applications such as heteroscedastic probabilistic principal component analysis (HPPCA). In this work, we derive a novel semidefinite program (SDP) relaxation of the original problem and study a few of its theoretical properties. We prove a global optimality certificate for the original nonconvex problem via a dual certificate, which leads to a simple feasibility problem to certify global optimality of a candidate solution on the Stiefel manifold. In addition, our relaxation reduces to an assignment linear program for jointly diagonalizable problems and is therefore known to be tight in that case. We generalize this result to show that it is also tight for close-to jointly diagonalizable problems, and we show that the HPPCA problem has this characteristic. Numerical results validate our global optimality certificate and sufficient conditions for when the SDP is tight in various problem settings.