# Learning gradient-based ICA by neurally estimating mutual information

## TL;DR

This paper demonstrates how a neural network can learn linear independent component analysis by estimating and minimizing mutual information using a neural mutual information estimator, achieving results comparable to FastICA.

## Contribution

It introduces a neural network approach to perform linear ICA by integrating mutual information neural estimation, a novel method for this task.

## Key findings

- Achieves ICA-like separation of noisy sources
- Uses mutual information neural estimation (MINE) for learning
- Qualitatively comparable to FastICA results

## Abstract

Several methods of estimating the mutual information of random variables have been developed in recent years. They can prove valuable for novel approaches to learning statistically independent features. In this paper, we use one of these methods, a mutual information neural estimation (MINE) network, to present a proof-of-concept of how a neural network can perform linear ICA. We minimize the mutual information, as estimated by a MINE network, between the output units of a differentiable encoder network. This is done by simple alternate optimization of the two networks. The method is shown to get a qualitatively equal solution to FastICA on blind-source-separation of noisy sources.