Probabilistic Autoencoder using Fisher Information

TL;DR

This paper introduces FisherNet, an extension of variational autoencoders that derives latent space uncertainty from the Fisher information metric, leading to improved data reconstruction and better scalability.

Contribution

FisherNet uniquely derives uncertainty from the decoder using Fisher information, enhancing theoretical soundness and performance over traditional VAEs.

Findings

FisherNet achieves more accurate data reconstructions.

FisherNet scales better with latent space dimensions.

Provides direct uncertainty quantification from the model.

Abstract

Neural Networks play a growing role in many science disciplines, including physics. Variational Autoencoders (VAEs) are neural networks that are able to represent the essential information of a high dimensional data set in a low dimensional latent space, which have a probabilistic interpretation. In particular the so-called encoder network, the first part of the VAE, which maps its input onto a position in latent space, additionally provides uncertainty information in terms of a variance around this position. In this work, an extension to the Autoencoder architecture is introduced, the FisherNet. In this architecture, the latent space uncertainty is not generated using an additional information channel in the encoder, but derived from the decoder, by means of the Fisher information metric. This architecture has advantages from a theoretical point of view as it provides a direct uncertainty quantification derived from the model, and also accounts for uncertainty cross-correlations. We can show experimentally that the FisherNet produces more accurate data reconstructions than a comparable VAE and its learning performance also apparently scales better with the number of latent space dimensions.