Learning Subspaces of Different Dimension

TL;DR

This paper presents a Bayesian framework for inferring mixtures of subspaces of varying dimensions, with a novel prior specification, efficient sampling, and applications to topic modeling, backed by theoretical guarantees.

Contribution

It introduces a new Bayesian mixture model for subspaces of different dimensions, including a prior on Grassmann manifolds and an efficient sampling algorithm.

Findings

Successful application to real and simulated data

Proven posterior consistency

Extended model for topic modeling

Abstract

We introduce a Bayesian model for inferring mixtures of subspaces of different dimensions. The key challenge in such a mixture model is specification of prior distributions over subspaces of different dimensions. We address this challenge by embedding subspaces or Grassmann manifolds into a sphere of relatively low dimension and specifying priors on the sphere. We provide an efficient sampling algorithm for the posterior distribution of the model parameters. We illustrate that a simple extension of our mixture of subspaces model can be applied to topic modeling. We also prove posterior consistency for the mixture of subspaces model. The utility of our approach is demonstrated with applications to real and simulated data.