Regression on fixed-rank positive semidefinite matrices: a Riemannian approach

TL;DR

This paper introduces a Riemannian gradient descent approach for learning fixed-rank positive semidefinite matrices in regression tasks, enabling scalable, invariant algorithms that do not restrict the matrix's range space.

Contribution

It develops a novel Riemannian optimization method for fixed-rank PSD matrices that improves scalability and invariance over previous methods.

Findings

Algorithms achieve linear complexity in problem size

Effective in learning distance functions with positive semidefinite matrices

Demonstrates good performance on benchmark datasets

Abstract

The paper addresses the problem of learning a regression model parameterized by a fixed-rank positive semidefinite matrix. The focus is on the nonlinear nature of the search space and on scalability to high-dimensional problems. The mathematical developments rely on the theory of gradient descent algorithms adapted to the Riemannian geometry that underlies the set of fixed-rank positive semidefinite matrices. In contrast with previous contributions in the literature, no restrictions are imposed on the range space of the learned matrix. The resulting algorithms maintain a linear complexity in the problem size and enjoy important invariance properties. We apply the proposed algorithms to the problem of learning a distance function parameterized by a positive semidefinite matrix. Good performance is observed on classical benchmarks.