Quadratic Programming Over Ellipsoids (with Applications to Constrained Linear Regression and Tensor Decomposition)

TL;DR

This paper introduces a new algorithm for quadratic programming over ellipsoids, splitting the problem into simpler sub-problems and applying an augmented-Lagrangian method, with applications to linear regression and tensor decomposition.

Contribution

It presents a novel splitting approach and an augmented-Lagrangian algorithm for quadratic programming over ellipsoids, including a tighter bound for the secular equation and a PSD matrix correction method.

Findings

Efficient solution via splitting into sphere QP and projection

Tighter bounds for secular equation minimizer

Improved convergence with low condition number PSD matrices

Abstract

A novel algorithm to solve the quadratic programming problem over ellipsoids is proposed. This is achieved by splitting the problem into two optimisation sub-problems, quadratic programming over a sphere and orthogonal projection. Next, an augmented-Lagrangian algorithm is developed for this multiple constraint optimisation. Benefit from the fact that the QP over a single sphere can be solved in a closed form by solving a secular equation, we derive a tighter bound of the minimiser of the secular equation. We also propose to generate a new psd matrix with a low condition number from the matrices in the quadratic constraints. This correction method improves convergence of the proposed augmented-Lagrangian algorithm. Finally, applications of the quadratically constrained QP to bounded linear regression and tensor decompositions are presented.