# An information theoretic approach to the autoencoder

## TL;DR

This paper introduces the InfoMax Autoencoder (IMAE), a variation that maximizes mutual information to learn robust data representations, outperforming existing autoencoder models in clustering tasks.

## Contribution

The paper proposes IMAE, a novel autoencoder that explicitly maximizes mutual information, enhancing representation robustness and clustering ability compared to prior autoencoder variants.

## Key findings

- IMAE demonstrates superior clustering performance on MNIST and Fashion-MNIST datasets.
- IMAE outperforms Denoising and Contractive Autoencoders in single-layer settings.
- IMAE shows competitive results against Variational Autoencoders in multi-layer configurations.

## Abstract

We present a variation of the Autoencoder (AE) that explicitly maximizes the mutual information between the input data and the hidden representation. The proposed model, the InfoMax Autoencoder (IMAE), by construction is able to learn a robust representation and good prototypes of the data. IMAE is compared both theoretically and then computationally with the state of the art models: the Denoising and Contractive Autoencoders in the one-hidden layer setting and the Variational Autoencoder in the multi-layer case. Computational experiments are performed with the MNIST and Fashion-MNIST datasets and demonstrate particularly the strong clusterization performance of IMAE.