Reinterpreting Importance-Weighted Autoencoders

TL;DR

This paper offers a new perspective on importance-weighted autoencoders, showing they optimize the standard variational lower bound with a more complex distribution, and introduces a tighter lower bound and visualization of the implicit distribution.

Contribution

It provides an alternative interpretation of importance-weighted autoencoders and derives a tighter lower bound, enhancing understanding of their optimization process.

Findings

Importance-weighted autoencoders optimize the standard variational lower bound with a complex distribution.

A new, tighter lower bound for importance-weighted autoencoders is derived.

Visualization of the implicit importance-weighted distribution is presented.

Abstract

The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribution. We formally derive this result, present a tighter lower bound, and visualize the implicit importance-weighted distribution.