Normalizing Flow with Variational Latent Representation

TL;DR

This paper introduces a variational latent representation framework for normalizing flows, enhancing their ability to model complex, multi-modal data distributions by jointly learning discrete latent sequences and continuous data generation.

Contribution

It proposes replacing the standard normal latent variable with a learned variational latent, enabling normalizing flows to better handle multi-modal data distributions.

Findings

Improved modeling of multi-modal data distributions.

Enhanced generative performance over standard NF methods.

Successful implementation with Transformer-based latent sequences.

Abstract

Normalizing flow (NF) has gained popularity over traditional maximum likelihood based methods due to its strong capability to model complex data distributions. However, the standard approach, which maps the observed data to a normal distribution, has difficulty in handling data distributions with multiple relatively isolated modes. To overcome this issue, we propose a new framework based on variational latent representation to improve the practical performance of NF. The idea is to replace the standard normal latent variable with a more general latent representation, jointly learned via Variational Bayes. For example, by taking the latent representation as a discrete sequence, our framework can learn a Transformer model that generates the latent sequence and an NF model that generates continuous data distribution conditioned on the sequence. The resulting method is significantly more powerful than the standard normalization flow approach for generating data distributions with multiple modes. Extensive experiments have shown the advantages of NF with variational latent representation.