MML Probabilistic Principal Component Analysis

TL;DR

This paper introduces a Bayesian minimum message length method for automatically determining the number of principal components in PCA, improving residual variance estimation and extending to mixture models.

Contribution

It presents a novel Bayesian approach for selecting PCA components and enhances residual variance estimation, with extensions to mixture models.

Findings

Improved residual variance estimation over maximum likelihood

Automatic component selection based on Bayesian MML

Extension to finite mixture models of PCA

Abstract

Principal component analysis (PCA) is perhaps the most widely used method for data dimensionality reduction. A key question in PCA is deciding how many factors to retain. This manuscript describes a new approach to automatically selecting the number of principal components based on the Bayesian minimum message length method of inductive inference. We derive a new estimate of the isotropic residual variance and demonstrate that it improves on the usual maximum likelihood approach. We also discuss extending this approach to finite mixture models of principal component analyzers.