General Constrained Matrix Optimization

TL;DR

This paper introduces a comprehensive matrix optimization model with general constraints, providing a novel solution approach that outperforms classical methods in solving complex problems like QCQPs.

Contribution

It develops the first matrix optimization framework with broad constraints and a feasible, convergent first-order algorithm, enabling new problem formulations and improved solution capabilities.

Findings

The model covers a wide range of problems including SDP, matrix completion, and QCQPs.

The proposed algorithm converges to approximate KKT points in $ ext{O}(1/ extepsilon^2)$ iterations.

Our method solves over ten times more QCQPs than classical approaches.

Abstract

This paper presents and analyzes the first matrix optimization model which allows general coordinate and spectral constraints. The breadth of problems our model covers is exemplified by a lengthy list of examples from the literature, including semidefinite programming, matrix completion, and quadratically constrained quadratic programs (QCQPs), and we demonstrate our model enables completely novel formulations of numerous problems. Our solution methodology leverages matrix factorization and constrained manifold optimization to develop an equivalent reformulation of our general matrix optimization model for which we design a feasible, first-order algorithm. We prove our algorithm converges to $(\epsilon,\epsilon)$-approximate first-order KKT points of our reformulation in $\mathcal{O}(1/\epsilon^2)$ iterations. The method we developed applies to a special class of constrained manifold optimization problems and is one of the first which generates a sequence of feasible points which converges to a KKT point. We validate our model and method through numerical experimentation. Our first experiment presents a generalized version of semidefinite programming which allows novel eigenvalue constraints, and our second numerical experiment compares our method to the classical semidefinite relaxation approach for solving QCQPs. For the QCQP numerical experiments, we demonstrate our method is able to dominate the classical state-of-the-art approach, solving more than ten times as many problems compared to the standard solution procedure.