A 1/R Law for Kurtosis Contrast in Balanced Mixtures

TL;DR

This paper establishes a mathematical law describing how kurtosis contrast diminishes in wide, balanced mixtures and proposes a purification method to restore contrast, supported by synthetic experiments.

Contribution

The paper proves a sharp redundancy law for kurtosis contrast in balanced mixtures and introduces a purification technique to recover contrast independent of mixture width.

Findings

Kurtosis contrast decreases proportionally to the inverse of the effective width R.

Surpassing standard estimation error requires R to be smaller than a threshold related to sample size T.

Purification restores contrast independently of R, using a simple data-driven heuristic.

Abstract

Kurtosis-based Independent Component Analysis (ICA) weakens in wide, balanced mixtures. We prove a sharp redundancy law: for a standardized projection with effective width $R_{\mathrm{eff}}$ (participation ratio), the population excess kurtosis obeys $|\kappa(y)|=O(\kappa_{\max}/R_{\mathrm{eff}})$, yielding the order-tight $O(c_b\kappa_{\max}/R)$ under balance (typically $c_b=O(\log R)$). As an impossibility screen, under standard finite-moment conditions for sample kurtosis estimation, surpassing the $O(1/\sqrt{T})$ estimation scale requires $R\lesssim \kappa_{\max}\sqrt{T}$. We also show that \emph{purification} -- selecting $m\!\ll\!R$ sign-consistent sources -- restores $R$-independent contrast $\Omega(1/m)$, with a simple data-driven heuristic. Synthetic experiments validate the predicted decay, the $\sqrt{T}$ crossover, and contrast recovery.