Learning of Generalized Low-Rank Models: A Greedy Approach

TL;DR

This paper introduces a flexible greedy algorithm for generalized low-rank models that outperforms existing methods in speed while maintaining or improving prediction accuracy across various convex and nonsmooth objectives.

Contribution

It develops a new greedy algorithm applicable to a wide range of low-rank modeling problems, extending beyond matrix completion with square loss.

Findings

Faster convergence than existing algorithms

Comparable or better prediction performance

Applicable to smooth and nonsmooth, convex and strongly convex objectives

Abstract

Learning of low-rank matrices is fundamental to many machine learning applications. A state-of-the-art algorithm is the rank-one matrix pursuit (R1MP). However, it can only be used in matrix completion problems with the square loss. In this paper, we develop a more flexible greedy algorithm for generalized low-rank models whose optimization objective can be smooth or nonsmooth, general convex or strongly convex. The proposed algorithm has low per-iteration time complexity and fast convergence rate. Experimental results show that it is much faster than the state-of-the-art, with comparable or even better prediction performance.