Sliced-Wasserstein normalizing flows: beyond maximum likelihood training

TL;DR

This paper introduces a hybrid training method for normalizing flows that combines maximum likelihood with sliced-Wasserstein distance, improving data generation quality and out-of-distribution detection.

Contribution

It proposes a novel hybrid training paradigm for normalizing flows that enhances generative performance and out-of-distribution detection compared to traditional MLE training.

Findings

Improved likelihood and visual quality of generated samples.

Lower likelihood of out-of-distribution data.

Better data fidelity in generated flows.

Abstract

Despite their advantages, normalizing flows generally suffer from several shortcomings including their tendency to generate unrealistic data (e.g., images) and their failing to detect out-of-distribution data. One reason for these deficiencies lies in the training strategy which traditionally exploits a maximum likelihood principle only. This paper proposes a new training paradigm based on a hybrid objective function combining the maximum likelihood principle (MLE) and a sliced-Wasserstein distance. Results obtained on synthetic toy examples and real image data sets show better generative abilities in terms of both likelihood and visual aspects of the generated samples. Reciprocally, the proposed approach leads to a lower likelihood of out-of-distribution data, demonstrating a greater data fidelity of the resulting flows.