Efficient Parametric Projection Pursuit Density Estimation

TL;DR

This paper introduces the UPoE model, a tractable parametric approach for projection pursuit density estimation using low-dimensional experts, with efficient learning algorithms and connections to ICA.

Contribution

It presents the UPoE model, a novel tractable parametric projection pursuit density estimator, and derives efficient sequential learning algorithms.

Findings

UPoE is fully tractable and interpretable.

ML learning rules align with under-complete ICA.

Efficient sequential learning algorithm developed.

Abstract

Product models of low dimensional experts are a powerful way to avoid the curse of dimensionality. We present the ``under-complete product of experts' (UPoE), where each expert models a one dimensional projection of the data. The UPoE is fully tractable and may be interpreted as a parametric probabilistic model for projection pursuit. Its ML learning rules are identical to the approximate learning rules proposed before for under-complete ICA. We also derive an efficient sequential learning algorithm and discuss its relationship to projection pursuit density estimation and feature induction algorithms for additive random field models.