Discrete Independent Component Analysis (DICA) with Belief Propagation

TL;DR

This paper introduces a discrete version of Independent Component Analysis (DICA) using belief propagation on a Bayesian bipartite graph, demonstrating its effectiveness in modeling MNIST character images for generative and inference tasks.

Contribution

It presents a novel discrete ICA model implemented with belief propagation, enabling efficient inference and learning on discrete data.

Findings

Effective generative modeling of MNIST images

Belief propagation facilitates inference in discrete ICA

Discrete ICA captures factorial code structure

Abstract

We apply belief propagation to a Bayesian bipartite graph composed of discrete independent hidden variables and discrete visible variables. The network is the Discrete counterpart of Independent Component Analysis (DICA) and it is manipulated in a factor graph form for inference and learning. A full set of simulations is reported for character images from the MNIST dataset. The results show that the factorial code implemented by the sources contributes to build a good generative model for the data that can be used in various inference modes.