On the representation and learning of monotone triangular transport maps

TL;DR

This paper introduces a flexible framework for representing and learning monotone triangular transport maps, ensuring global optimality in estimation and demonstrating applications in density estimation, inference, and graphical models.

Contribution

It develops a general invertible transformation framework for monotone triangular maps, proving conditions for global optimality and proposing an adaptive algorithm for sparse approximation.

Findings

Framework guarantees no spurious local minima in the optimization landscape.

The proposed algorithm effectively estimates the KR map from data.

Applications show stable performance in density estimation and inference tasks.

Abstract

Transportation of measure provides a versatile approach for modeling complex probability distributions, with applications in density estimation, Bayesian inference, generative modeling, and beyond. Monotone triangular transport maps$\unicode{x2014}$approximations of the Knothe$\unicode{x2013}$Rosenblatt (KR) rearrangement$\unicode{x2014}$are a canonical choice for these tasks. Yet the representation and parameterization of such maps have a significant impact on their generality and expressiveness, and on properties of the optimization problem that arises in learning a map from data (e.g., via maximum likelihood estimation). We present a general framework for representing monotone triangular maps via invertible transformations of smooth functions. We establish conditions on the transformation such that the associated infinite-dimensional minimization problem has no spurious local minima, i.e., all local minima are global minima; and we show for target distributions satisfying certain tail conditions that the unique global minimizer corresponds to the KR map. Given a sample from the target, we then propose an adaptive algorithm that estimates a sparse semi-parametric approximation of the underlying KR map. We demonstrate how this framework can be applied to joint and conditional density estimation, likelihood-free inference, and structure learning of directed graphical models, with stable generalization performance across a range of sample sizes.