Adaptive Mixtures of Factor Analyzers

TL;DR

This paper introduces a robust, adaptive mixture of factor analyzers model that performs simultaneous clustering and nonlinear dimensionality reduction, with automatic model complexity selection tailored to data.

Contribution

It presents a novel model selection algorithm for mixtures of factor analyzers that adapts the number of factors per component, improving clustering and manifold learning.

Findings

Effective in clustering and manifold learning

Outperforms related model selection algorithms

Fast and robust approach

Abstract

A mixture of factor analyzers is a semi-parametric density estimator that generalizes the well-known mixtures of Gaussians model by allowing each Gaussian in the mixture to be represented in a different lower-dimensional manifold. This paper presents a robust and parsimonious model selection algorithm for training a mixture of factor analyzers, carrying out simultaneous clustering and locally linear, globally nonlinear dimensionality reduction. Permitting different number of factors per mixture component, the algorithm adapts the model complexity to the data complexity. We compare the proposed algorithm with related automatic model selection algorithms on a number of benchmarks. The results indicate the effectiveness of this fast and robust approach in clustering, manifold learning and class-conditional modeling.