Statistical physics of independent component analysis

TL;DR

This paper applies statistical physics methods to analyze independent component analysis, revealing a phase transition in learning performance using an adapted cavity approach.

Contribution

It introduces an adapted cavity method for ICA, overcoming the limitations of the replica method, and characterizes the phase transition in learning curves.

Findings

Replica method fails for this analysis

Cavity approach provides valid results

Learning curves show a first order phase transition

Abstract

Statistical physics is used to investigate independent component analysis with polynomial contrast functions. While the replica method fails, an adapted cavity approach yields valid results. The learning curves, obtained in a suitable thermodynamic limit, display a first order phase transition from poor to perfect generalization.