PMOG: The projected mixture of Gaussians model with application to blind source separation

TL;DR

This paper introduces the PMOG model, extending Gaussian mixtures to projected data, and applies it to blind source separation, offering a flexible approach to modeling non-Gaussian sources without heavy assumptions.

Contribution

The paper develops an EM algorithm for the PMOG model and demonstrates its application to blind source separation, improving flexibility over traditional methods.

Findings

PMOG effectively models non-Gaussian source densities.

The EM algorithm efficiently estimates model parameters.

PMOG-based BSS outperforms conventional methods in flexibility.

Abstract

We extend the mixtures of Gaussians (MOG) model to the projected mixture of Gaussians (PMOG) model. In the PMOG model, we assume that q dimensional input data points z_i are projected by a q dimensional vector w into 1-D variables u_i. The projected variables u_i are assumed to follow a 1-D MOG model. In the PMOG model, we maximize the likelihood of observing u_i to find both the model parameters for the 1-D MOG as well as the projection vector w. First, we derive an EM algorithm for estimating the PMOG model. Next, we show how the PMOG model can be applied to the problem of blind source separation (BSS). In contrast to conventional BSS where an objective function based on an approximation to differential entropy is minimized, PMOG based BSS simply minimizes the differential entropy of projected sources by fitting a flexible MOG model in the projected 1-D space while simultaneously optimizing the projection vector w. The advantage of PMOG over conventional BSS algorithms is the more flexible fitting of non-Gaussian source densities without assuming near-Gaussianity (as in conventional BSS) and still retaining computational feasibility.