Self-sufficient Independent Component Analysis via KL Minimizing Flows

TL;DR

This paper introduces a novel self-sufficient ICA method that learns disentangled signals by minimizing a conditional KL divergence, avoiding adversarial training and prior assumptions, demonstrated on toy and real datasets.

Contribution

It proposes a prior-free, likelihood-free ICA approach using KL minimizing flows that enhances stability and flexibility over traditional methods.

Findings

Effective on toy datasets

Successful on real-world data

Avoids adversarial training issues

Abstract

We study the problem of learning disentangled signals from data using non-linear Independent Component Analysis (ICA). Motivated by advances in self-supervised learning, we propose to learn self-sufficient signals: A recovered signal should be able to reconstruct a missing value of its own from all remaining components without relying on any other signals. We formulate this problem as the minimization of a conditional KL divergence. Compared to traditional maximum likelihood estimation, our algorithm is prior-free and likelihood-free, meaning that we do not need to impose any prior on the original signals or any observational model, which often restricts the model's flexibility. To tackle the KL divergence minimization problem, we propose a sequential algorithm that reduces the KL divergence and learns an optimal de-mixing flow model at each iteration. This approach completely avoids the unstable adversarial training, a common issue in minimizing the KL divergence. Experiments on toy and real-world datasets show the effectiveness of our method.