Regularisation for PCA- and SVD-type matrix factorisations

TL;DR

This paper explores regularisation techniques for PCA and SVD to improve robustness against noise, revealing that different formulations of the regularisation problem yield qualitatively different solutions.

Contribution

It introduces new formulations of regularisation for PCA and SVD, highlighting their impact on the solutions obtained.

Findings

Different regularisation formulations lead to distinct solutions.

Regularisation can improve noise robustness in PCA/SVD.

Theoretical analysis of regularisation effects on matrix decompositions.

Abstract

Singular Value Decomposition (SVD) and its close relative, Principal Component Analysis (PCA), are well-known linear matrix decomposition techniques that are widely used in applications such as dimension reduction and clustering. However, an important limitation of SVD/PCA is its sensitivity to noise in the input data. In this paper, we take another look at the problem of regularisation and show that different formulations of the minimisation problem lead to qualitatively different solutions.