Mixing and non-mixing local minima of the entropy contrast for blind source separation

TL;DR

This paper analyzes local minima of the entropy contrast in blind source separation, revealing conditions for non-mixing and mixing minima and proposing an entropy approximation method for multimodal sources.

Contribution

It demonstrates that the exact entropy function has non-mixing minima for any non-Gaussian source and introduces an entropy approximator with error bounds for multimodal densities.

Findings

Non-mixing minima occur when output aligns with any non-Gaussian source.

Existence of mixing minima when sources are strongly multimodal.

Proposed entropy approximator with error bounds applicable to multimodal densities.

Abstract

In this paper, both non-mixing and mixing local minima of the entropy are analyzed from the viewpoint of blind source separation (BSS); they correspond respectively to acceptable and spurious solutions of the BSS problem. The contribution of this work is twofold. First, a Taylor development is used to show that the \textit{exact} output entropy cost function has a non-mixing minimum when this output is proportional to \textit{any} of the non-Gaussian sources, and not only when the output is proportional to the lowest entropic source. Second, in order to prove that mixing entropy minima exist when the source densities are strongly multimodal, an entropy approximator is proposed. The latter has the major advantage that an error bound can be provided. Even if this approximator (and the associated bound) is used here in the BSS context, it can be applied for estimating the entropy of any random variable with multimodal density.