Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer

TL;DR

This paper introduces an adversarial regularizer for autoencoders that enhances the realism of interpolated outputs and improves the quality of learned representations for downstream tasks.

Contribution

It proposes a novel regularization method using a critic network to improve autoencoder interpolation and representation quality.

Findings

Regularizer significantly improves interpolation realism.

Enhanced interpolation correlates with better downstream task performance.

Benchmark results show increased interpolation accuracy.

Abstract

Autoencoders provide a powerful framework for learning compressed representations by encoding all of the information needed to reconstruct a data point in a latent code. In some cases, autoencoders can "interpolate": By decoding the convex combination of the latent codes for two datapoints, the autoencoder can produce an output which semantically mixes characteristics from the datapoints. In this paper, we propose a regularization procedure which encourages interpolated outputs to appear more realistic by fooling a critic network which has been trained to recover the mixing coefficient from interpolated data. We then develop a simple benchmark task where we can quantitatively measure the extent to which various autoencoders can interpolate and show that our regularizer dramatically improves interpolation in this setting. We also demonstrate empirically that our regularizer produces latent codes which are more effective on downstream tasks, suggesting a possible link between interpolation abilities and learning useful representations.