Density Estimation and Classification via Bayesian Nonparametric Learning of Affine Subspaces

TL;DR

This paper introduces a Bayesian nonparametric model that learns the low-dimensional submanifold structure of high-dimensional data, enabling efficient density estimation and flexible classification with interpretable parameters.

Contribution

It proposes a novel Bayesian nonparametric framework for jointly learning the submanifold dimension and performing density estimation and classification in high-dimensional settings.

Findings

Effective dimension reduction in high-dimensional data

Improved density estimation accuracy

Interpretable model parameters

Abstract

It is now practically the norm for data to be very high dimensional in areas such as genetics, machine vision, image analysis and many others. When analyzing such data, parametric models are often too inflexible while nonparametric procedures tend to be non-robust because of insufficient data on these high dimensional spaces. It is often the case with high-dimensional data that most of the variability tends to be along a few directions, or more generally along a much smaller dimensional submanifold of the data space. In this article, we propose a class of models that flexibly learn about this submanifold and its dimension which simultaneously performs dimension reduction. As a result, density estimation is carried out efficiently. When performing classification with a large predictor space, our approach allows the category probabilities to vary nonparametrically with a few features expressed as linear combinations of the predictors. As opposed to many black-box methods for dimensionality reduction, the proposed model is appealing in having clearly interpretable and identifiable parameters. Gibbs sampling methods are developed for posterior computation, and the methods are illustrated in simulated and real data applications.