Cubic-Regularized Newton for Spectral Constrained Matrix Optimization and its Application to Fairness

TL;DR

This paper introduces a cubic-regularized Newton method for spectral constrained matrix optimization, providing new theoretical insights and demonstrating its effectiveness in fair covariance matrix estimation.

Contribution

It develops a second-order chain rule for matrix functions, offers a convergence analysis for the method, and applies it to fair covariance matrix estimation.

Findings

Effective in estimating fair and robust covariance matrices.

Provides theoretical convergence guarantees for the proposed method.

Demonstrates advantages over existing approaches in experiments.

Abstract

Matrix functions are utilized to rewrite smooth spectral constrained matrix optimization problems as smooth unconstrained problems over the set of symmetric matrices which are then solved via the cubic-regularized Newton method. A second-order chain rule identity for matrix functions is proven to compute the higher-order derivatives to implement cubic-regularized Newton, and a new convergence analysis is provided for cubic-regularized Newton for matrix vector spaces. We demonstrate the applicability of our approach by conducting numerical experiments on both synthetic and real datasets. In our experiments, we formulate a new model for estimating fair and robust covariance matrices in the spirit of the Tyler's M-estimator (TME) model and demonstrate its advantage.