Weighted Low-rank Approximation via Stochastic Gradient Descent on Manifolds

TL;DR

This paper introduces a stochastic gradient descent method on manifolds for weighted low-rank approximation, providing convergence guarantees and demonstrating superior performance on real-world Netflix data compared to Euclidean approaches.

Contribution

The paper develops a convergence theorem for manifold-based stochastic gradient descent and applies it to weighted low-rank approximation, outperforming Euclidean methods on practical data.

Findings

Algorithm outperforms Euclidean stochastic gradient descent on Netflix data

Convergence theorem established for manifold-based stochastic gradient descent

Accelerated line search on manifolds improves optimization efficiency

Abstract

We solve a regularized weighted low-rank approximation problem by a stochastic gradient descent on a manifold. To guarantee the convergence of our stochastic gradient descent, we establish a convergence theorem on manifolds for retraction-based stochastic gradient descents admitting confinements. On sample data from the Netflix Prize training dataset, our algorithm outperforms the existing stochastic gradient descent on Euclidean spaces. We also compare the accelerated line search on this manifold to the existing accelerated line search on Euclidean spaces.