Binary Independent Component Analysis: A Non-stationarity-based Approach

TL;DR

This paper introduces a novel binary ICA method leveraging non-stationarity, providing a practical approach with pairwise marginals, and explores conditions for model identifiability based on the number of observed variables.

Contribution

It presents the first non-stationarity-based binary ICA method with a closed-form likelihood and analyzes model identifiability depending on observed variables.

Findings

Identifiability improves with more observed variables.

Pairwise marginals enable faster estimation.

Non-stationarity is crucial for binary ICA identifiability.

Abstract

We consider independent component analysis of binary data. While fundamental in practice, this case has been much less developed than ICA for continuous data. We start by assuming a linear mixing model in a continuous-valued latent space, followed by a binary observation model. Importantly, we assume that the sources are non-stationary; this is necessary since any non-Gaussianity would essentially be destroyed by the binarization. Interestingly, the model allows for closed-form likelihood by employing the cumulative distribution function of the multivariate Gaussian distribution. In stark contrast to the continuous-valued case, we prove non-identifiability of the model with few observed variables; our empirical results imply identifiability when the number of observed variables is higher. We present a practical method for binary ICA that uses only pairwise marginals, which are faster to compute than the full multivariate likelihood. Experiments give insight into the requirements for the number of observed variables, segments, and latent sources that allow the model to be estimated.