Improving Autoencoder Image Interpolation via Dynamic Optimal Transport

TL;DR

This paper introduces a novel autoencoder training method that uses dynamic optimal transport principles to produce more semantically meaningful image interpolations, even with limited training data.

Contribution

It proposes a new regularization based on optimal transport to improve the realism and semantic consistency of image interpolation in autoencoders.

Findings

Produces more realistic interpolations

Handles complex environments and unbalanced transport

Robust with limited training data

Abstract

Autoencoders are important generative models that, among others, have the ability to interpolate image sequences. However, interpolated images are usually not semantically meaningful.In this paper, motivated by dynamic optimal transport, we consider image interpolation as a mass transfer problem and propose a novel regularization term to penalize non-smooth and unrealistic changes in the interpolation result. Specifically, we define the path energy function for each path connecting the source and target images. The autoencoder is trained to generate the $L^2$ optimal transport geodesic path when decoding a linear interpolation of their latent codes. With a simple extension, this model can handle complicated environments, such as allowing mass transfer between obstacles and unbalanced optimal transport. A key feature of the proposed method is that it is physics-driven and can generate robust and realistic interpretation results even when only very limited training data are available.