Bayesian ICA with super-Gaussian Source Priors

TL;DR

This paper introduces a Bayesian ICA framework with a horseshoe prior, providing scalable algorithms for estimation and inference, backed by theoretical guarantees and competitive empirical performance.

Contribution

It presents a novel hierarchical Bayesian approach to ICA with a horseshoe prior, unifying estimation strategies and establishing theoretical guarantees.

Findings

Competitive performance with existing ICA tools

Scalable algorithms for point estimation and posterior inference

Theoretical guarantees including posterior contraction

Abstract

Independent Component Analysis (ICA) plays a central role in modern machine learning as a flexible framework for feature extraction. We introduce a horseshoe-type prior with a latent Polya-Gamma scale mixture representation, yielding scalable algorithms for both point estimation via expectation-maximization (EM) and full posterior inference via Markov chain Monte Carlo (MCMC). This hierarchical formulation unifies several previously disparate estimation strategies within a single Bayesian framework. We also establish the first theoretical guarantees for hierarchical Bayesian ICA, including posterior contraction and local asymptotic normality results for the unmixing matrix. Comprehensive simulation studies demonstrate that our methods perform competitively with widely used ICA tools. We further discuss implementation of conditional posteriors, envelope-based optimization, and possible extensions to flow-based architectures for nonlinear feature extraction and deep learning. Finally, we outline several promising directions for future work.