Stochastic Approximation for Online Tensorial Independent Component Analysis

TL;DR

This paper analyzes the convergence of an online tensorial ICA algorithm, providing finite-sample error bounds under mild assumptions, advancing understanding of stochastic approximation methods in high-dimensional ICA.

Contribution

It offers a novel convergence analysis for online tensorial ICA as a nonconvex stochastic approximation, with sharp finite-sample error bounds.

Findings

Achieves a finite-sample error bound of ( ext{sqrt}(d/T)) under mild conditions.

Requires the data dimension and sample size to satisfy a scaling condition involving d^4/T.

Provides a dynamics-based analysis for the convergence of the online tensorial ICA algorithm.

Abstract

Independent component analysis (ICA) has been a popular dimension reduction tool in statistical machine learning and signal processing. In this paper, we present a convergence analysis for an online tensorial ICA algorithm, by viewing the problem as a nonconvex stochastic approximation problem. For estimating one component, we provide a dynamics-based analysis to prove that our online tensorial ICA algorithm with a specific choice of stepsize achieves a sharp finite-sample error bound. In particular, under a mild assumption on the data-generating distribution and a scaling condition such that $d^4/T$ is sufficiently small up to a polylogarithmic factor of data dimension $d$ and sample size $T$, a sharp finite-sample error bound of $\tilde{O}(\sqrt{d/T})$ can be obtained.