Robust Variational Inference

TL;DR

This paper introduces a robust modification to variational inference that improves performance when training data contains many noisy or uninformative samples, demonstrated through experiments on autoencoders with noisy datasets.

Contribution

It proposes a new robust variational objective and evidence lower bound that are effective in noisy data scenarios, enhancing inference robustness.

Findings

Improved performance on synthetic noisy datasets.

Small but consistent gains on original datasets.

Robust objective outperforms traditional evidence lower bound.

Abstract

Variational inference is a powerful tool for approximate inference. However, it mainly focuses on the evidence lower bound as variational objective and the development of other measures for variational inference is a promising area of research. This paper proposes a robust modification of evidence and a lower bound for the evidence, which is applicable when the majority of the training set samples are random noise objects. We provide experiments for variational autoencoders to show advantage of the objective over the evidence lower bound on synthetic datasets obtained by adding uninformative noise objects to MNIST and OMNIGLOT. Additionally, for the original MNIST and OMNIGLOT datasets we observe a small improvement over the non-robust evidence lower bound.