Information Theoretic-Learning Auto-Encoder

TL;DR

This paper introduces an information theoretic learning approach for autoencoders, providing a new regularization method that offers an alternative to existing generative models without requiring explicit partition functions.

Contribution

It develops ITL divergence measures for neural network regularization and formalizes generative moment matching networks within this framework, offering novel autoencoder variants.

Findings

ITL-regularized autoencoders effectively generate data

Provides a new perspective on generative modeling without explicit partition functions

Formalizes generative moment matching networks under ITL framework

Abstract

We propose Information Theoretic-Learning (ITL) divergence measures for variational regularization of neural networks. We also explore ITL-regularized autoencoders as an alternative to variational autoencoding bayes, adversarial autoencoders and generative adversarial networks for randomly generating sample data without explicitly defining a partition function. This paper also formalizes, generative moment matching networks under the ITL framework.