Tensorizing flows: a tool for variational inference

TL;DR

This paper introduces a novel tensorized normalizing flow method using tensor networks to enhance variational inference, especially for complex multimodal distributions, demonstrating improved performance over traditional approaches.

Contribution

The paper proposes replacing the Gaussian reference in normalizing flows with a tensor network-based distribution, enabling better modeling of multimodal distributions in variational inference.

Findings

Tensorized flows improve multimodal distribution modeling.

Combining tensor networks with flows outperforms individual methods.

Enhanced variational inference results on challenging tasks.

Abstract

Fueled by the expressive power of deep neural networks, normalizing flows have achieved spectacular success in generative modeling, or learning to draw new samples from a distribution given a finite dataset of training samples. Normalizing flows have also been applied successfully to variational inference, wherein one attempts to learn a sampler based on an expression for the log-likelihood or energy function of the distribution, rather than on data. In variational inference, the unimodality of the reference Gaussian distribution used within the normalizing flow can cause difficulties in learning multimodal distributions. We introduce an extension of normalizing flows in which the Gaussian reference is replaced with a reference distribution that is constructed via a tensor network, specifically a matrix product state or tensor train. We show that by combining flows with tensor networks on difficult variational inference tasks, we can improve on the results obtained by using either tool without the other.