Learning Robust Low-Rank Representations

TL;DR

This paper introduces a neural network-based framework for fast, robust low-rank representation learning that approximates robust PCA efficiently and can be extended to handle complex data transformations, with applications in multimedia processing.

Contribution

The paper develops a trainable encoder architecture that approximates robust PCA using structured non-convex optimization, enabling real-time processing and improved performance in practical tasks.

Findings

Significant speedup over exact RPCA solvers with minimal accuracy loss

Outperforms traditional RPCA in music source separation tasks

Achieves faster-than-real-time processing on mobile hardware

Abstract

In this paper we present a comprehensive framework for learning robust low-rank representations by combining and extending recent ideas for learning fast sparse coding regressors with structured non-convex optimization techniques. This approach connects robust principal component analysis (RPCA) with dictionary learning techniques and allows its approximation via trainable encoders. We propose an efficient feed-forward architecture derived from an optimization algorithm designed to exactly solve robust low dimensional projections. This architecture, in combination with different training objective functions, allows the regressors to be used as online approximants of the exact offline RPCA problem or as RPCA-based neural networks. Simple modifications of these encoders can handle challenging extensions, such as the inclusion of geometric data transformations. We present several examples with real data from image, audio, and video processing. When used to approximate RPCA, our basic implementation shows several orders of magnitude speedup compared to the exact solvers with almost no performance degradation. We show the strength of the inclusion of learning to the RPCA approach on a music source separation application, where the encoders outperform the exact RPCA algorithms, which are already reported to produce state-of-the-art results on a benchmark database. Our preliminary implementation on an iPad shows faster-than-real-time performance with minimal latency.