Accelerating likelihood optimization for ICA on real signals

TL;DR

This paper investigates optimization methods for ICA, highlighting the limitations of quasi-Newton algorithms on real signals and demonstrating that the Picard algorithm effectively overcomes these issues for both constrained and unconstrained problems.

Contribution

The paper analyzes why existing quasi-Newton ICA algorithms underperform on real data and shows that the Picard algorithm provides a robust solution for both problem types.

Findings

Quasi-Newton methods perform well on simulated data but poorly on real signals.

The Picard algorithm overcomes convergence issues on real data.

Picard is effective for both constrained and unconstrained ICA problems.

Abstract

We study optimization methods for solving the maximum likelihood formulation of independent component analysis (ICA). We consider both the the problem constrained to white signals and the unconstrained problem. The Hessian of the objective function is costly to compute, which renders Newton's method impractical for large data sets. Many algorithms proposed in the literature can be rewritten as quasi-Newton methods, for which the Hessian approximation is cheap to compute. These algorithms are very fast on simulated data where the linear mixture assumption really holds. However, on real signals, we observe that their rate of convergence can be severely impaired. In this paper, we investigate the origins of this behavior, and show that the recently proposed Preconditioned ICA for Real Data (Picard) algorithm overcomes this issue on both constrained and unconstrained problems.