Universal Approximation of Residual Flows in Maximum Mean Discrepancy

TL;DR

This paper proves that residual flows, a type of normalizing flow with Lipschitz residual blocks, are capable of universally approximating distributions under maximum mean discrepancy, with bounds on the number of blocks needed.

Contribution

It establishes the theoretical universality of residual flows in maximum mean discrepancy, providing bounds on residual block count for approximation.

Findings

Residual flows are universal approximators in MMD.

Upper bounds on residual blocks for approximation.

Theoretical insights into residual flow expressiveness.

Abstract

Normalizing flows are a class of flexible deep generative models that offer easy likelihood computation. Despite their empirical success, there is little theoretical understanding of their expressiveness. In this work, we study residual flows, a class of normalizing flows composed of Lipschitz residual blocks. We prove residual flows are universal approximators in maximum mean discrepancy. We provide upper bounds on the number of residual blocks to achieve approximation under different assumptions.