On Robust Probabilistic Principal Component Analysis using Multivariate $t$-Distributions

TL;DR

This paper clarifies the theoretical foundations of robust probabilistic PCA using multivariate t-distributions, correcting previous misrepresentations, and introduces a new MCEM algorithm for implementation.

Contribution

It establishes equivalent relationships between t-PPCA and hierarchical models, correcting literature inaccuracies, and proposes a novel MCEM algorithm for robust PPCA.

Findings

Theoretical clarification of t-PPCA hierarchical models

Simulation results demonstrating robustness improvements

Introduction of a new MCEM algorithm for model implementation

Abstract

Probabilistic principal component analysis (PPCA) is a probabilistic reformulation of principal component analysis (PCA), under the framework of a Gaussian latent variable model. To improve the robustness of PPCA, it has been proposed to change the underlying Gaussian distributions to multivariate $t$-distributions. Based on the representation of $t$-distribution as a scale mixture of Gaussian distributions, a hierarchical model is used for implementation. However, in the existing literature, the hierarchical model implemented does not yield the equivalent interpretation. In this paper, we present two sets of equivalent relationships between the high-level multivariate $t$-PPCA framework and the hierarchical model used for implementation. In doing so, we clarify a current misrepresentation in the literature, by specifying the correct correspondence. In addition, we discuss the performance of different multivariate $t$ robust PPCA methods both in theory and simulation studies, and propose a novel Monte Carlo expectation-maximization (MCEM) algorithm to implement one general type of such models.