A geometric approach to the transport of discontinuous densities

TL;DR

This paper introduces a geometric method for transporting discontinuous densities by recovering underlying manifolds from partial observations, enhancing uncertainty quantification and system identification.

Contribution

It proposes a novel approach that incorporates observation process data to recover underlying manifolds, improving transport map estimation for discontinuous densities.

Findings

Inclusion of observation histories aids manifold recovery.

The method handles non-bijective source-target relations.

Limitations are discussed for complex manifold structures.

Abstract

Different observations of a relation between inputs ("sources") and outputs ("targets") are often reported in terms of histograms (discretizations of the source and the target densities). Transporting these densities to each other provides insight regarding the underlying relation. In (forward) uncertainty quantification, one typically studies how the distribution of inputs to a system affects the distribution of the system responses. Here, we focus on the identification of the system (the transport map) itself, once the input and output distributions are determined, and suggest a modification of current practice by including data from what we call "an observation process". We hypothesize that there exists a smooth manifold underlying the relation; the sources and the targets are then partial observations (possibly projections) of this manifold. Knowledge of such a manifold implies knowledge of the relation, and thus of "the right" transport between source and target observations. When the source-target observations are not bijective (when the manifold is not the graph of a function over both observation spaces, either because folds over them give rise to density singularities, or because it marginalizes over several observables), recovery of the manifold is obscured. Using ideas from attractor reconstruction in dynamical systems, we demonstrate how additional information in the form of short histories of an observation process can help us recover the underlying manifold. The types of additional information employed and the relation to optimal transport based solely on density observations is illustrated and discussed, along with limitations in the recovery of the true underlying relation.

Equations

$$\mu (A)={\int }_A{f}_{\mu }(x)d\lambda (x)\text{and}\nu (A)={\int }_A{f}_{\nu }(y)d\lambda (y).$$

$${\int }_A{f}_{\nu }(y)d\lambda (y)={\int }_{T^{-1}(A)}{f}_{\mu }(x)d\lambda (x).$$

$$f_{\nu }(y)=f_{\mu }\left(T^{-1}(y)\right)|\det D_y{T}^{-1}|.$$

$${\int }_{{\mathbb{R}}^n}c(x,T(x))f_{\mu }(x)d\lambda (x),$$

$$W(x)=\left(F_{\nu }^{-1}\circ F_{\mu }\right)(x).$$

$$\underset{y\to a^{-}}{\lim }{f}_{\nu }(y)\ne \underset{y\to a^{+}}{\lim }{f}_{\nu }(y)\text{and}\underset{y\to a^{-}}{\lim }{f}_{\nu }(y),\underset{y\to a^{+}}{\lim }{f}_{\nu }(y)\in \mathbb{R}.$$

$$\underset{y\to a^{-}}{\lim }{f}_{\nu }(y)=\infty \text{or}\underset{y\to a^{+}}{\lim }{f}_{\nu }(y)=\infty .$$

$$f_{\nu }(y)=\sum _{x\in T^{-1}(y)}\frac{f_{\mu }(x)}{|T^{\prime }(x)|},$$

$$c(x)=(x,T(x)),x\in [0,1].$$

$$s=\phi (x)=\mathrm{arcl}(x)={\int }_0^x||\dot{c}(t)||dt$$

$$f_c(s)=f_{\mu }({\mathrm{arcl}}^{-1}(s))|{{\mathrm{arcl}}^{-1}}^{\prime }(s)|.$$

$$f_c(s)=f_{\mu }({\mathrm{arcl}}^{-1}(s))\frac{1}{\sqrt{1+T^{\prime }({\mathrm{arcl}}^{-1}(s){)}^2}},$$

$$d(x)=(T(x-\tau ),T(x)),$$

$$S(x_1,x_2)=\left(x_1+x_2^3,x_2-x_1^3\right),$$

$$\left|\frac{d}{dy}{T}_1^{-1}(y)\right|={\rho }_1(y),\left|\frac{d}{dy}{T}_2^{-1}(y)\right|+\left|\frac{d}{dy}{T}_3^{-1}(y)\right|={\rho }_2(y)$$

$$T_1(\beta )=T_2(\beta ),T_2(\gamma )=T_3(\gamma ),T_2^{\prime }(\gamma )=T_3^{\prime }(\gamma )=0.$$

$$T_2(x)=F_{{\rho }_2}^{-1}\left(\frac{F_{{\rho }_2}(c)(x-\beta )}{\gamma -\beta }\right),T_3(x)=F_{{\rho }_2}^{-1}\left(\frac{F_{{\rho }_2}(c)(x-\beta -F_{{\rho }_2}(c))}{\gamma -\beta -F_{{\rho }_2}(c)}\right).$$

$$\gamma =F_{{\rho }_2}(c)\frac{{\rho }_1(b)}{{\rho }_2(b)}+\beta$$

$$T_2^{\prime \prime }(x)=-{\left(\frac{F_{{\rho }_2}(c)}{\gamma -\beta }\right)}^2\frac{{\rho }_2^{\prime }(T_2(x))}{{\rho }_2(T_2(x){)}^3},T_3^{\prime \prime }(x)=-{\left(\frac{F_{{\rho }_2}(c)}{\gamma -\beta -F_{{\rho }_2}(c)}\right)}^2\frac{{\rho }_2^{\prime }(T_2(x))}{{\rho }_2(T_2(x){)}^3},$$

$$\underset{y\to 1^{-}}{\lim }\frac{{\rho }_2^{\prime }(y)}{\rho (y{)}^3}=\frac{3}{2}.$$

$$k(y_i,y_j)=\exp \left(-\frac{(y_j-y_i{)}^T\left(J^T(y_i)J(y_i)+J^T(y_j)J(y_j)\right)(y_j-y_i)}{2\epsilon }\right),$$

$$\frac{1}{2}{\int }_{\mathcal{M}}d(x,T(x){)}^2{f}_{\mu }(x){\mathrm{dVol}}_g(x),$$

$${\Phi }_{(\varphi ,y)}(x)=\left(y(x),y(\varphi (x)),\dots ,y(\underset{2n\text{times}}{\underbrace{\varphi \circ \cdots \circ \varphi }}(x))\right)$$

$${\Phi }_{(F,y)}(p)=\left(y(p),y(F(p)),\dots ,y(\underset{2n\text{times}}{\underbrace{F\circ \cdots \circ F}}(p))\right),p\in \mathcal{M},$$

Full text

Keywords: discontinuous densities, singularities, manifold reconstruction, delay embedding, optimal transport

MSC: 58K05, 60G30, 37C20, 62-07

1 Introduction

We are interested in studying observations of an (unknown), manifold when the histograms of the observed quantity suggest the existence of singularities. Such cases arise in many physical or computational contexts: e.g., observations of tracer diffusion on a non-flat cell surface [1], particle flow in MC simulations [6], or density of state singularities in carbon nanotubes [18]. Figure 1 illustrates how such observation singularities might arise: A uniform density of points $f_{\mu}(x)$ on a segment of the $x$-axis is non-injectively mapped, through the black parabola, to a segment of the $y$ axis giving rise to the density $f_{\nu}(y)$. It is easy to see that this density has, by construction, both a jump discontinuity and an “infinity”. The two densities ($f_{\mu}(x)$ and $f_{\nu}(y)$), constitute two different observations of the same manifold (the black curve). If we only know the two distributions, and do not know the underlying pointwise correspondences, we can attempt to transport them to each other using the Wasserstein optimal transport [33, 5]. Attempting this, for example with numerical algorithms as in [9, 23, 22], does not recover the manifold (the black curve in Figure 1), but a different one, which is visibly non-differentiable at a single point (Figure 1, red map). Mapping $x$-point observations through this red function to the $y$-axis results in the same $f_{\nu}(y)$, but pointwise different $y$-values, compared to the black curve that we know, in this case, to be “the truth”.

The need to create useful pointwise mappings (as opposed to only mappings of distributions) arises in several contexts (domain adaptation or transfer learning, e.g. [35, 8]). This would be greatly facilitated if we could reconstruct, from our observations, the intrinsic manifold, i.e., if we can find the “intrinsic” state, the point on the black curve. In Figure 1, we can consider the “intrinsic state” to be the arclength on the black curve, starting, without loss of generality, from one of its two ends. Then both the “source” ($x$) and the “target” ($y$) observations are simply functions of this intrinsic state. We will discuss cases where it would make even less sense to use Wasserstein optimal transport, namely, when the dimensions of the source and target spaces differ: e.g. if one set of observations consists of scalars, while the other set is vector-valued.

Our goal is to devise conditions under which we can meaningfully reconstruct the original, underlying manifold. In general, starting with only the two distributions, this is not possible; so we must modify the problem. A transport between two densities is reinterpreted as a geometric object (the function is identified with its graph). To embed the object, a possibility arises through the use of embedding theorems [32, 26, 34]. To apply these theorems, we need to assume additional observation information, not just the density. This additional information can be provided through an observation process on the object, yielding observation histories, or, more generally, “ensembles” of observations around each point.

With this additional information, one can create diffeomorphic copies of the intrinsic manifold from different sets of observations (e.g. short observation histories in $x$ and short observation histories in $y$). The densities on the two intrinsic manifold copies can then be easily transported, e.g by solving the Optimal Mass Transport (OMT) problem.

Another way to implement this transport is by using the Mahalanobis-like metric Diffusion Map framework [7, 27, 28, 11], which goes beyond diffeomorphisms to create isometric embeddings of the intrinsic manifold—the transport in this case reduces to a global orthogonal transformation [4, 12]. For this, we also need the additional observation process data.

The “enhanced” reconstruction of the intrinsic manifold that exploits these additional process data allows us to deal with overlapping densities of observations, arising through non-invertible maps, see also [10]. These include densities with singularities, therefore providing a contrasting approach to [14, 16, 17] in the spirit of [13]. The approach also allows us to usefully transport marginal distributions if sufficient observation process data is available.

Even when the original function is invertible, the Wasserstein optimal transport might not give the “correct” intrinsic solution, but one that can be transformed to it through a measure-preserving map, see e.g. McCann’s polar factorization [19]. The Mahalanobis framework provides an isometry to the intrinsic manifold, so that the missing transformation to the solution is just a global orthogonal map. This is particularly interesting in higher dimensions; on $\mathbb{R}$ these concepts agree.

The paper is organized as follows: Section 2 introduces basic mathematical concepts concerning optimal transport and embedology. In Section 3 we illustrate the recovery of manifolds and corresponding transport maps through time-delay embeddings in the one-dimensional setting. In Section 4 we extend these results to the two-dimensional case, including a discussion of marginal distributions. Section 5 demonstrates the recovery of the intrinsic manifold up to an isometry. We compare this to the reconstruction with optimal transport, emphasizing the difference between measure-preserving and geometry-preserving maps.

2 Basic mathematical concepts and notation

2.1 Transport of densities

The optimal transport problem has been proposed by Monge [20], and seeks to find a mapping from one distribution of mass to another such that a cost function is minimized among all measure-preserving maps. We here introduce the mathematical background on $\mathbb{R}^{n}$ (the general Riemannian manifold case is discussed in Appendix C), mainly following [33].

On $\mathbb{R}^{n}$ we consider two measures $\mu$ and $\nu$. We say that $\nu$ is the push-forward of $\mu$ under $T:\mathbb{R}^{n}\to\mathbb{R}^{n}$, written as $\nu=T_{\sharp}\mu$, if $\nu(A)=\mu(T^{-1}(A)),A\subset\mathbb{R}^{n},$ where $T^{-1}(A)$ is the preimage of $A$ under $T$. If $\mu\ll\lambda$ and $\nu\ll\lambda$, i.e. both $\mu$ and $\lambda$ are absolutely continuous with respect to the Lebesgue measure $\lambda$ on $\mathbb{R}^{n}$, then there exist densities, that is, Lebesgue-integrable functions $f_{\mu},f_{\nu}:\mathbb{R}^{n}\to\mathbb{R}$ such that

[TABLE]

A short-hand notation for (1) is $\mu=f_{\mu}\,\lambda$, and similarly for $\nu$. The push-forward relation $\nu=T_{\sharp}\mu$ can then be rewritten as

[TABLE]

If $T$ is invertible and differentiable, by change of variables, we can formulate (2) as

[TABLE]

Given two densities $f_{\mu},f_{\nu}$, there might exist many transport maps satisfying (3). Thus one often seeks to find a transport that in addition to (3) is also unique in some sense—typically represented by an optimization problem. In the theory of optimal transport [33], the map $T$ is required to minimize a cost function of the form

[TABLE]

under the constraint (3). The cost $c$ is usually set to $c(x,y)=|x-y|^{p}$, and $p$ is typically set to $2$. The optimization of the Wasserstein problem with $p=2$ has a unique solution under reasonable assumptions [33, 5].

2.2 Embedology

In this paper, we use a dynamic observation process to provide additional information that will help us reconstruct intrinsic manifolds and define useful transport maps. The corresponding theory is broadly used when observing the time evolution of dynamical systems, and reconstructing attractors (long-term dynamics) through time series data. In that context, Packard et al. [21], Aeyels [2], and Takens [32] describe the observability of nonlinear state spaces, by using a number of delays of a single, real-valued function of the system state to reconstruct the system attractor. The results are mainly based on the theorems of Whitney [34] (see appendix), which describe conditions for smooth functions on a large class of manifolds to construct embeddings. Sauer, Yorke, and Casdagli [26] later refined the concept by proving that “almost all” smooth functions can be used to construct embeddings into Euclidean space. The authors also generalized the results to fractal sets rather than smooth manifolds. Here, we employ these embedding theorems as a theoretical underpinning for the reconstruction of intrinsic manifolds and transport maps.

3 One dimensional transport and discontinuous densities

In this section, we illustrate the main ideas of the paper in the simple one-dimensional case ($n=1$), where the intrinsic manifold is the graph of a function over a single, real variable $x$. In later sections (Sections 4 and E), we describe the general setting in which the coordinates $x,y$ are considered functions over a (higher-dimensional) manifold.

In one dimension, there exists a unique, monotonically increasing, solution to the minimization of the cost function (4) under the constraint (2). It is given by

[TABLE]

Here $F_{\mu}$ is the cumulative distribution function (cdf) of $f_{\mu}$, defined by $F_{\mu}(x)=\int_{-\infty}^{x}f_{\mu}(t)d\lambda(t),$ and similarly for $F_{\nu}$. Our only restriction on the densities $f_{\mu},f_{\nu}$ is that they be Lebesgue-integrable. Therefore, they may have discontinuities and singularities. In our illustrative example (Figure 1) we consider a jump discontinuity and divergence to infinity.

Definition 1**.**

A Lebesgue-integrable density $f_{\nu}$ possesses a jump discontinuity at $a\in\mathbb{R}$ if

[TABLE]

A Lebesgue-integrable density $f_{\nu}$ diverges to infinity at $a\in\mathbb{R}$ if

[TABLE]

Other types of discontinuities (e.g. derivative discontinuities) are also possible but not discussed here.

3.1 Discontinuities and non-bijective transport

Consider a density $f_{\nu}$ which is pushed by a transport $T$ from a continuous density $f_{\mu}$. The two types of discontinuity defined above are illustated in Figure 1. In this figure, the uniform density $f_{\mu}$ on the $x$-axis is transported to $f_{\nu}$ on the $y$-axis. We present two transports that push $f_{\mu}$ to $f_{\nu}$: The “folded” (i.e. non-injective) smooth map (black), and the Wasserstein optimal transport (5) (red), which is continuous, but not $C^{1}$. The divergence to infinity arises because both transport maps (black and red) have a maximum, i.e. satisfy $T^{\prime}(x)=0$ at some $x\in[0,1]$. The jump discontinuity of $f_{\nu}$ arises in each transport for a different reason:

through the derivative discontinuity in the red transport map, 2. 2.

through the folding in the black transport map (the map is surjective, but not injective).

Since we want to be able to work with noninvertible maps, like the black curve, we need to use (2) rather than (3) to define push-forward of measures. We now discuss case (2) in more detail. It illustrates that a continuous density can be pushed to a discontinuous one via a smooth transport. If the transport $T$ is not injective, there is a generalization of (3):

[TABLE]

if $T^{\prime}(x)\neq 0$ for all $x$. Here $T^{-1}$ denotes the preimage.

Transporting a density with discontinuities as in Figure 1 to a uniform density with the Wasserstein optimal transport map $W$ (5), gives rise to a function which is not $C^{1}$ everywhere, as we integrate a discontinuous function, see Figure 2 (top right). Thus the non-injective transport $T$ of Figure 2 (top left) is optimal in the sense that it is smooth and transports the parametrization induced by the coloring correctly. Note however that in general the Wasserstein cost (4) is smaller for $W$ than for $T$ ($W$ is closer to the identity map than $T$), hence $T$ is not optimal in the Wasserstein sense. For simple examples similar to the one shown in Figure 2, the cost can be computed explicitly 111For a quadratic version of the folded transport $T$, given by $y=T(x)=-\left(\frac{3}{2}+\sqrt{2}\right)x^{2}+(2+\sqrt{2})x$, the cost (4) is $(1/60)(6+\sqrt{2})\approx 0.12$, which is more than for the Wasserstein optimal transport $W$, where the cost is $(1/60)(-54+41\sqrt{2})\approx 0.07$..

In general, when dealing with a discontinuous density $f_{\nu}$ as in Figure 1, we do not know that it consists of two (or more) branches as indicated by the coloring in Figure 2. It is thus difficult to uncover the double folding and obtain the smooth transport $T$. In the next sections we describe how the density $f_{\nu}$ can be unfolded by assuming some additional information (an observation process). In Appendix A we also suggest a construction of a $C^{1}$ transport only from the knowledge of the distributions, but without additional information.

3.2 Unfolding discontinuous densities

We start by explaining the idea of unfolding discontinuous densities. Given a curve in $\mathbb{R}^{2}$, $c(s)=(c_{1}(s),c_{2}(s))$, and a density $f_{c}$ over $s$, we can push this density to the two axes by projecting it to the coordinates $c_{1}$ and $c_{2}$. Even in the case that both $c_{1}$ and $c_{2}$ are non-injective, we can push the density with the generalized formula (6).

A simple example is the graph of a function $T$, given by the curve

[TABLE]

Here the projection to the first coordinate is the identity, and the projection to the second coordinate is given through the function $T$. Now we can reparametrize the curve $c$ with a bijective map $s=\varphi(x)$ resulting in a curve $\tilde{c}(s)=c(\varphi^{-1}(s))=(\varphi^{-1}(s),T(\varphi^{-1}(s))).$ We then consider a transport from $s$ to $x$. Figure 2 illustrates the effect of the coordinate transformation $\varphi$. The uniform density $f_{\mu}$ on $[0,1]$ ($x$-axis) is transported to a discontinuous density $f_{\nu}$ on the $y$-axis via a non-injective transport map $T$. Reparametrizing the curve $c(x)=(x,T(x))$ by its arclength

[TABLE]

gives rise to the density

[TABLE]

The density $f_{c}$ on the arclength $s$ is continuous, in contrast to the discontinuous density $f_{\nu}$ on $y$ (Figure 2). This can also be seen by rewriting (9):

[TABLE]

where the denominator is never zero. Projecting from $s$ to the second component of $\tilde{c}$, we obtain the transport from $f_{c}$ to $f_{\nu}$, which is the original transport $T$ up to the one-to-one transport between $x$ and $s$. As we will see below, by using additional information from an observation process, we can recover the fact that $T$ is folded.

3.3 Time-delay embedding

In the previous section we assumed the knowledge of the full curve $c(x)=(x,T(x))$ to explain the general concept.

In this section, we show how equivalent results can be achieved by employing a (systematic but unknown) “observation process”—considering, for example, that values of $y$ are always observed along trajectories of a systematic constant motion in $x$. We do not assume that all the $y$ observations have been produced along a single constant-speed $x$ trajectory; we may have many short trajectories initialized at various random $x$ values. Yet all of the observations follow the same process (the same, unknown, $x$-sampling dynamics).

Now the curve $c$ is embedded in a higher-dimensional (here, two-dimensional) space by using the time-delay embedding $d$:

[TABLE]

with time-delay parameter $\tau$. By Takens’ results [32], we know that such a time-delay embedding is diffeomorphic to the original curve (see Figure 3), if enough time delays and generic observations are used. In this particular example we only need a single delay to obtain a diffeomorphic embedding (Figure 4). In general, to embed an $n$-dimensional manifold, $2n+1$ observation functions (e.g., one real-valued observation plus $2n$ delays) will be sufficient (see Appendix E).

As in Section 3.2, we can transport the uniform density $f_{\mu}(x)$ on $[0,1]$ to a density $f_{d}(s^{\prime})$ along the arclength $s^{\prime}$ of the curve embedded through time-delays (10). The density $f_{d}$ is shown on the original curve $c$ in Figure 4 (right).

With the same approach we can also recover the underlying manifold in more complicated examples. In Figure 5 we consider a function consisting of many folds (left). The histogram obtained by transporting the uniform density on $[0,1]$ ($x$-axis) with this function consists of many discontinuities (middle). By using delay coordinates $(y_{n+1},y_{n})$ (and PCA) we can recover a manifold that is diffeomorphic to the original function. A parametrization of this curve, here obtained with DMAP (right) can be used to transport to the uniform density on $[0,1]$ ($x$-axis).

Instead of functions, we can even consider relations: In Figure 6 the underlying manifold it is a closed curve. In this case, the recorded histogram is obtained by pushing the uniform density on the arclength of the curve to the $y$-axis. Note that in these examples we can also consider the histogram obtained by pushing the uniform density on the arclength to the $x$-axis—in the examples where we consider functions (Figures 3 and 5) this is not interesting, as we already parametrize the manifolds by the $x$-axis.

Also in the case of relations we can reconstruct a diffeomorphic copy of the underlying manifold with a time-delay embedding in $(y_{n+2},y_{n+1},y_{n})$. As in the case for functions, a parametrization (obtained by, e.g. DMAP) can be used to transport to the uniform density.

To conclude: Diffeomorphic copies of one-dimensional curves can be constructed through time-delay embeddings. The densities on these curves are continuous, in contrast to the original target densities.

4 Densities on two dimensional manifolds

In two dimensions, we illustrate cases exhibiting one-parameter families of discontinuities and study marginals of two-dimensional distributions. We show how the idea of “unfolding” discontinuities through process observations (in the form of delays) can be applied to such singular densities, and how the same approach can help construct joint distributions from marginal ones; both cases involve the construction of a copy of an underlying manifold from process observation histories.

4.1 Unfolding two-dimensional discontinuous densities

Consider the cusp surface (Figure 7, top right) for $x,\beta_{1}\in[-1,1]$ and $\beta_{2}(x,\beta_{1})=x^{3}-\beta_{1}\,x$. Sampling uniformly on the $(x,\beta_{1})$-square and observing the distribution of points in $(\beta_{1},\beta_{2})$ gives rise to a density with a one-parameter family of discontinuities at which the density approaches infinity (see Figure 7, left), including the cusp point $(\beta_{1},\beta_{2})=(0,0)$.

To “unfold” this two-dimensional singular density, we assume access to data from an observation process on the $(x,\beta_{1})$ plane, starting from randomly chosen initial conditions and moving in the positive $\beta_{1}$-direction. The observations in this example are $([\beta_{1}]_{n+1},[\beta_{2}]_{n},[\beta_{2}]_{{n+1}})$; the $x$ coordinate is not recorded. Figure 7 illustrates the embedding of the $(x,\beta_{1})$-plane into the space of the three principal components of the collection of these delayed observations (bottom right panel). A two-dimensional parametrization of this reconstructed surface (e.g. through DMAP) can be used to transport the points to the original parametrization in $(x,\beta_{1})$.

4.2 Converting one-dimensional marginal distributions to joint distributions

Given the marginal density in $\beta_{2}$ (Figure 8, top left), we cannot transport it to the uniform density in $(x,\beta_{1})$, as the respective dimensions disagree. Using history from an observation process starting at randomly selected initial points in $(x,\beta_{1})$ and taking two steps in the $x$-direction, we could attempt (in the spirit of Section 3.3) to “unfold” the singularity at $([\beta_{2}]_{n}=0)$ using the delays $([\beta_{2}]_{n},[\beta_{2}]_{{n+1}})$. In contrast to the example of Section 3.3, this clearly does not lead to a curve in $\mathbb{R}^{2}$, but visibly to overlaps of a surface in the projection, see Figure 8 (bottom left). This indicates that the intrinsic dimension is not one, but two, and that we need a third coordinate to properly embed the underlying manifold. This visual assertion can be also quantified by applying dimension estimation algorithms to the ensemble of possible observation histories.

In delay-coordinates $([\beta_{2}]_{{n}},[\beta_{2}]_{{n+1}},[\beta_{2}]_{{n+2}})$ we are able to unfold the scalar $[\beta_{2}]_{n}$ observations (Figure 8 top left), to a two-dimensional surface (bottom right, top of the figure, showing the data set embedded into the three principal components of the observation delay-coordinates). The resulting surface is diffeomorphic to the original cusp surface. As in the previous section, a two-dimensional parametrization of this reconstructed surface could be used to transport the points to the original $(x,\beta_{1})$-parametrization (illustrated schematically through black lines from the surface to the $(x,\beta_{1})$ square in Figure 8, bottom right).

4.3 Parametrizing the embedded surface

One could attempt to map the two-dimensional surface reconstructed from “unfolding” the one-dimensional marginal distribution of $\beta_{2}$ values in Figure 8 (as detailed in Section 4.2) to the two original coordinates $(x,\beta_{1})$ through Wasserstein optimal transport. Alternatively, we can also recover the original parametrization $(x,\beta_{1})$ using a non-linear manifold learning technique, Diffusion Maps (DMAP) [7] (see Appendix B). To do this, we must employ the Mahalanobis distance [7, 27], see Figure 9, which, however, requires more information than just densities or even ensembles of points on each two-dimensional surface: It needs estimates of the covariance matrices (the Jacobian of the mapping between the two manifolds) at each point. We discuss the difference between the Wasserstein and Mahalanobis frameworks for transport in Section 5.

5 Recovering parametrizations up to isometries

In the previous sections, we illustrated the recovery of manifolds from (individually) non-invertible observations. In this section, we discuss the conceptual similarities and differences between the reconstruction of geometry through Mahalanobis-Diffusion Maps and the construction of transport maps optimal in the Wasserstein sense. We simplify the presentation by assuming that the observations are now already embeddings of the manifold (i.e., invertible on their image). Figure 10 illustrates the concept of maps changing metrics and measures.

Let $S$ be a smooth, invertible function, mapping a manifold $\mathcal{M}$ to itself. This map changes the metric $g$ into the metric $S_{\ast}g$. The density of points on $\mathcal{M}$ also changes through $S$, indicating that measures are also transformed. This section discusses how the change in the metric and the resulting change in the measures relate to each other, and how the information about either metrics or measures can be used to recover different aspects of the map $S$.

Given two measures $\mu$ and $\nu$ on a Riemannian manifold $(\mathcal{M},g)$, absolutely continuous with respect to its volume form $\operatorname{Vol}_{g}$ (defined in Appendix C), we endeavor to reconstruct a fixed, but unknown, invertible, measure-preserving map $S:\mathcal{M}\to\mathcal{M}$ such that $\nu(A)=\mu(S^{-1}(A)),\ A\subset\mathcal{M},$ which we write as $S_{\sharp}\mu=\nu$. As discussed in Section 2, there may exist many such maps for fixed $\mu$ and $\nu$. When trying to recover the original function $S$, we may choose an optimality criterion to at least select a unique map—for example, we could use the map that is minimizing the Wasserstein cost. In this section, we consider measures that are induced by metrics, which can lead to a notion of optimality via metric preservation.

By solving the Wasserstein optimal transport problem between $\mu$ and $\nu$ on $\mathcal{M}$, we can reconstruct the map $S$ only up to maps $U:\mathcal{M}\to\mathcal{M}$ with $U_{\sharp}\mu=\mu$. This is due to the polar factorization ([5, 19] and 2 in the Appendix): Given a measure $\mu\ll\operatorname{Vol}_{g}$, a Borel map $S:\mathcal{M}\to\mathcal{M}$ can be written uniquely ($\mu$-a.e) as $S=T\circ U$, where $T$ is the optimal transport between $\mu$ and $S_{\sharp}\mu$ and $U$ is measure-preserving w.r.t. $\mu$, i.e. $U_{\sharp}\mu=\mu$.

Instead of optimizing the Wasserstein cost and reconstructing $S$ up to measure-preserving maps $U$, we employ here additional information from our observation processes to obtain a map with far less ambiguity. To this end, we use the construction of metric-preserving maps described in [27, 4]: Given a metric $g$ and its push-forward metric $S_{\ast}g$ by a diffeomorphism $S:\mathcal{M}\to\mathcal{M}$, the map $S$ can be reconstructed up to a linear, orthogonal map. The reconstruction can even be done in a data-driven way, employing diffusion map (DMAP) embeddings [7] and a Mahalanobis distance [27]. The reconstruction up to an orthogonal map is justified through the following argument: By choosing the metric $S_{\ast}g$ based on the metric $g$, $S$ is an isometry. Laplace-Beltrami operators of isometric manifolds have the same eigenvalues, and eigenfunctions associated to the same eigenvalue are related by an orthogonal map [4, 25]. Therefore, an isometry $S$ between the base manifolds $(\mathcal{M},g)$ and $(\mathcal{M},S_{*}g)$ turns into an orthogonal map in eigenfunction coordinates (Mahalanobis-DMAP coordinates) of the manifolds, which can be computed easily [4]. Similar to [4], we summarize this in a commutative diagram (Figure 11).

The metric $S_{\ast}g$ can be computed from $g$ by $(S_{\ast}g)_{y}(\xi,\eta)=g_{S^{-1}(y)}(J(y)\xi,J(y)\eta),$ where $\xi,\eta\in T_{y}\mathcal{M}$ and $J(y)$ denotes the Jacobian of $S^{-1}$ at $y\in\mathcal{N}$. If $\mathcal{N}$ is embedded in Euclidean space, to use the push-forward metric $S_{\ast}g$ instead of the induced Euclidean metric on $\mathcal{N}$ for DMAP embeddings, a special kernel can be employed (see (15) in Appendix B, and [27]). The kernel requires estimations of the Jacobian matrices of $S$ at every point, which can be obtained by computing local covariance matrices as described in [27]. The induced metric $S_{\ast}g$ computed via covariance matrices is also referred to as the Mahalanobis distance, see [11].

Even if $S$ is not an isometry, but only measure-preserving between the manifolds $(\mathcal{M},g)$ and $(\mathcal{M},g^{\prime})$, the map between their diffusion map embeddings is still measure-preserving (see fig. 12): An embedding with $\ell$ eigenfunctions is an isometry between the original manifold $(\mathcal{M},g)$ and $\Phi_{g}(\mathcal{M})\subset\mathbb{R}^{\ell}$ with the metric induced by the Euclidean metric, up to the truncation error caused by using a finite number of eigenfunctions [24]. Therefore, measures are also preserved up to this accuracy.

Note that on $\mathbb{R}$, a measure-preserving map is automatically metric-preserving as well. In higher dimensions, however, there is a difference between these concepts, which is also apparent by comparing the Wasserstein and Mahalanobis frameworks. To show the difference, we consider an example from [27], which is used to explain the Mahalanobis concept.

Consider the map

[TABLE]

which maps the unit square $[0,1]^{2}\subset\mathbb{R}^{2}$ to a mushroom-like subset of $\mathbb{R}^{2}$ (therefore, as in [27], we call $S$ the mushroom-map), see the upper part of Figure 13. In this figure, the blue points $x_{i}$ arise from a uniform sampling of the unit square, and the points $y_{i}$ are their images under the mushroom-map $S$ (generating “observations” of the points $x_{i}$). If we apply DMAP with the Mahalanobis-kernel (15) (induced by the mushroom-map $S$, i.e. with Jacobians of $S^{-1}$) to the points $y_{i}$, we obtain an embedding denoted by $\phi^{M}(y_{i})$: We are able to recover a (rotated) square (Figure 13 lower right). A similar square is obtained by applying standard DMAPS (i.e. with Euclidean distances) to the sampling points $x_{i}$, denoted by $\phi(x_{i})$, see Figure 13 lower left.

We now map the distribution of points $y_{i}$ back to the uniform distribution on the square $[0,1]^{2}$ with Wasserstein optimal transport, denoted by $T$. Applying the mapping $T^{-1}$ pointwise to the $y_{i}$, we obtain the points $\tilde{x}_{i}$ (Figure 13 upper left, red) on the unit square $[0,1]^{2}$. DMAP with Euclidean distance applied to $\tilde{x}_{i}$ results in the the red square described by the points $\psi(\tilde{x}_{i})$ (Figure 13 lower left, red).

All three squares (blue and red in lower left and blue in lower right in Figure 13) in DMAP space have the same distribution of points, i.e. can be transformed into each other with a measure preserving map (Figure 12). The two blue squares are even related point-wise by an orthogonal map due to the isometry induced by the Mahalanobis-kernel (Figure 11).

We show the underlying mappings, rather than just the distributions of points, in Figure 14. The first row of Figure 14 is the orthogonal mapping from $(\phi_{1},\phi_{2})$ to $(\phi_{1}^{M},\phi_{2}^{M})$ presented in the form of contour plots. The second row shows the map from $(\phi_{1},\phi_{2})$ to $(\psi_{1},\psi_{2})$ in the same manner. This map is measure-preserving, but not orthogonal.

These plots show that the Wasserstein and Mahalanobis concepts produce the same result with respect to measures, but not with respect to point-wise mappings. Wasserstein creates a point mapping from distributions; Mahalanobis creates a point mapping (and thus a way to transport distributions) via local covariances.

6 Conclusion

In this paper, we studied densities that arose from observations of an unknown manifold; we focused on the case where the histograms of the observed quantities suggest the existence of singularities in the densities. Attempting to transport these histogram observations with Wasserstein optimal transport may not recover the intrinsic structure of the manifold. By assuming access to additional information from an observation process, we can employ embedding theorems to construct meaningful realizations of the underlying manifold. If enough additional information is available, through this construction, the density of points on the recovered manifold is no longer singular.

Even in case the direct observations do provide enough information to embed the manifold, Wasserstein optimal transport may not recover the “correct” functional relations (a fact embodied in the polar factorization theorem). To obtain such a functional relation with less ambiguity we can employ metric-preserving maps, for example constructed through diffusion maps with the Mahalanobis-metric. This approach again relies on additional process observation data. The reconstruction of useful pointwise maps (as opposed to mappings of distributions) and manifolds can be especially interesting and useful in contexts such as domain adaptation.

Acknowledgements

This work was partially supported by DARPA (Lagrange Program, Drs. F. Fahroo and C. Lewis) and by the ARO through a MURI (Drs. S. Stanton and M. Munson).

Appendix

Appendix A Construction of a $C^{1}$ transport without additional information

Starting only with two distributions and no additional information (e.g. histories), it is difficult to meaningfully reconstruct the underlying transport map. For the example of Figure 1, we suggest an easy construction which at least produces a $C^{1}$ transport (in contrast to the Wasserstein transport, which is only continuous).

The aim is to find a $C^{1}$ transport that pushes the uniform density on $[\alpha,\delta]$ (x-axis) to the discontinuous density $f_{\nu}$ on $[a,c]$ (y-axis), see Figure 15. For our construction we assume that the transport we are looking for consists of three parts $T_{1},T_{2},T_{3}$, as indicated in Figure 15. Even with this restriction there exist many solutions.

To find $T_{1},T_{2},T_{3}$, we have to solve the following problem:

[TABLE]

under the constraints

[TABLE]

Here we use notation as in Figure 15; note that we denote the two parts of the density $f_{\nu}$ by $\rho_{1}$ and $\rho_{2}$, respectively. The starting point $\alpha$ of our transport can be chosen freely, note however that $\delta=\alpha+1$, if we assume that the density $f_{\nu}$ integrates to $1$.

The first problem in (12) has a unique monotonically increasing solution (given by the inverse of the cdf $F_{\rho_{1}}$ of $\rho_{1}$, compare (5)). This determines $T_{1}$, and also $\beta=\alpha+F_{\rho_{1}}(b)$.

The following choices of $T_{2}$ and $T_{3}$ (depending on the choice of the location $\gamma$ of the maximum) solve the second problem in (12) under the constraints (13):

[TABLE]

In this construction $T_{2}$ is monotonically increasing, and $T_{3}$ is decreasing.

The transport consisting of the three parts $T_{1},T_{2},T_{3}$ is continuous, but only the choice

[TABLE]

gives rise to a $C^{1}$ transport, i.e. satisfies $T_{1}^{\prime}(\beta)=T_{2}^{\prime}(\beta)$. Note that if the original density does not have a jump discontinuity at $b$, i.e. $\rho_{1}(b)=\rho_{2}(b)$, then $\gamma=\delta$, i.e. the maximum is at the end of the interval, and our construction gives rise to the Wasserstein optimal transport (the inverse cdf as in (5)). Note that only $T_{1}$ and $T_{2}$ are needed then—$T_{3}$ is not well-defined.

In general, our construction does not reconstruct the original transport (the green function versus the black function in Figure 15). Nevertheless it gives rise to a $C^{1}$ transport that pushes $f_{\mu}$ to $f_{\nu}$, in contrast to the Wasserstein transport, which is only continuous in this example.

Note that we fit the first derivatives at $\beta$ and $\gamma$ to obtain a $C^{1}$ transport; in general the constructed transport is not smoother than that. The second derivatives of $T_{2}$ and $T_{3}$, for example, are given by

[TABLE]

which, in general give different values at $\gamma$. The second derivatives might be unbounded at $\gamma$, in case $\rho_{2}^{\prime}$ grows faster than $\rho_{2}^{3}$.

In our specific example of $f_{\nu}$ (pushed from the uniform density on $[0,1]$ by the cubic polynomial $y=T(x)=-2(1-x)^{3}+1.5(1-x)+0.5$ defined in Figure 1), we have $c=1$ and

[TABLE]

This implies that the transport constructed with $T_{1},T_{2},T_{3}$ has bounded second derivatives at $\gamma$, but is not $C^{2}$, as $T_{2}^{\prime\prime}(\gamma)\neq T_{3}^{\prime\prime}(\gamma)$.

Appendix B Diffusion maps (DMAP)

We briefly outline the Diffusion Maps algorithm, and refer to the special case when using the Mahalanobis-metric below. Given $N$ (possibly noisy) data points $\mathcal{D}=\{y_{1},\ldots,y_{N}\}$ in ambient Euclidean space $\mathbb{E}=\mathbb{R}^{m}$ close to a smooth, compact manifold $\mathcal{M}$, the DMAP algorithm constructs a parametrization of $\mathcal{M}$ based on the convergence of the normalized graph Laplacian on the data to the Laplace–Beltrami operator on $\mathcal{M}$. First, we construct a graph between the points, where connectivity is based on a similarity measure given by a Gaussian kernel employing the Euclidean distance in the ambient space $\mathbb{E}$: For a given scale parameter $\epsilon>0$, the similarity between two distinct points $y_{i}$ and $y_{j}$ in $\mathbb{E}$ is defined through $K_{ij}=k(y_{i},y_{j})=\rm{exp}\left(-r^{2}/\epsilon\right)$, where $r:=d(y_{i},y_{j})$. Appropriate choices of the parameter $\epsilon$ depend on the data [7, 4]. Second, if the data points $\mathcal{D}$ are not sampled uniformly in $\mathcal{M}$, the matrix $K$ has to be normalized by an estimation of the density on the diagonal of a matrix $P\in\mathbb{R}^{N\times N}$, $P_{ii}=\sum_{j=1}^{N}K_{ij}$, $\widetilde{K}=P^{-\alpha}KP^{-\alpha}$ where $\alpha=0$ (no normalization, [3]) can be used in the case of uniform sampling, and $\alpha=1$ otherwise [7]. Third, the kernel matrix $\widetilde{K}$ is normalized by the diagonal matrix $D\in\mathbb{R}^{N\times N}$, where $D_{ii}=\Sigma_{j=1}^{N}\widetilde{K}_{ij}$ for $i=1,\dots,N$. The non-linear parametrization (embedding) of the manifold is then given by a certain number $\ell$ of eigenvectors of $A=D^{-1}\widetilde{K}\in\mathbb{R}^{N\times N}$, scaled by their respective eigenvalue (and removing redundant eigenvectors that are functions of eigenvectors associated to larger eigenvalues [11]). The new embedding dimension $\ell$ may be much smaller than the ambient space dimension $m$, in which case DMAP achieves dimensionality reduction.

The similarity between points defined through the kernel can also include information about non-linear maps $S:\mathcal{M}\to\mathbb{R}^{m}$ (which have to be invertible on their image), employing the so-called “Mahalanobis-metric” in the kernel, first introduced in  [27]:

[TABLE]

where $J(y)$ is the Jacobian matrix of the inverse transformation $S^{-1}$ at the point $y$. The product $J^{T}J$ can be approximated through a covariance, for example, obtained by short bursts of a stochastic dynamical system with subsequent mapping by $S$ [28].

Appendix C Optimal transport on Riemannian manifolds

The results presented in Section 2 can be extended to Riemannian manifolds, which we summarize here, following [19].

Let $(\mathcal{M},g)$ be a smooth (for the purpose of [19], at least $C^{3}$) Riemannian manifold, where $g$ denotes the metric. The volume form $\operatorname{dVol}_{g}$, given in coordinates by $\operatorname{dVol}_{g}=\sqrt{|\det(g)|}d^{k}x$, induces a measure on $\mathcal{M}$ via $\operatorname{Vol}_{g}(A)=\int_{\mathcal{M}}\mathbf{1}_{A}\operatorname{dVol}_{g}=\int_{A}\operatorname{dVol}_{g},$ where $A\subseteq\mathcal{M}$ and $\mathbf{1}_{A}$ is the indicator function. If a measure $\mu$ on $\mathcal{M}$ is absolutely continuous with respect to $\operatorname{Vol}_{g}$, again written as $\mu\ll\operatorname{Vol}_{g}$, then there exists a density $f_{\mu}$, such that $\mu(A)=\int_{A}f_{\mu}(x)\operatorname{dVol}_{g}(x),$ with $A\subseteq\mathcal{M}$. This is also written as $\mu=f_{\mu}\operatorname{Vol}_{g}$.

Given two measures $\mu,\nu\ll\operatorname{Vol}_{g}$ on $\mathcal{M}$, the optimal transport problem seeks to find a smooth map $T:\mathcal{M}\to\mathcal{M}$ such that $\nu=T_{\sharp}\mu$ (where, as in Section 2.1, $T_{\sharp}\mu(A)=\mu(T^{-1}(A))$) and such that the cost

[TABLE]

is minimized. Here $d$ denotes the Riemannian distance function on $\mathcal{M}$ induced by $g$. The push-forward condition $\nu=T_{\sharp}\mu$ can be replaced by (3) for bijective $T$. The existence of an optimal transport map is proved in [19]:

Theorem 2**.**

[19, Results 9–11]** Let $(\mathcal{M},g)$ be a connected, compact Riemannian manifold, $C^{3}$-smooth and without boundary. Then we have

If $\mu\ll\operatorname{Vol}_{g}$ and $\nu$ arbitrary, then there exists a smooth map $T$ satisfying $T_{\sharp}\mu=\nu$, and minimzing (16). Only one $T$ can arise in this way (up to sets of $\mu$-measure zero). $T$ is the optimal transport pushing $\mu$ to $\nu$. 2. 2.

If also $\nu\ll\operatorname{Vol}_{g}$, then there exists an optimal transport $T^{\ast}$ pushing $\nu$ to $\mu$. $T$ and $T^{\ast}$ are inverses of each other ($\mu$* resp. $\nu$ almost everywhere).* 3. 3.

If $S:\mathcal{M}\to\mathcal{M}$ is a Borel map, $\mu$ a Radon measure, and $\nu:=S_{\sharp}\mu\ll\operatorname{Vol}_{g}$. Then $S=T\circ U$, where $T$ is the optimal transport pushing $\mu$ to $\nu$ and $U$ satisfies $U_{\sharp}\mu=\mu$.

Appendix D Takens theorems, embedology and Whitney theorems

Let $k\geq n\in\mathbb{N}$, and $\mathcal{M}\subset\mathbb{R}^{k}$ be a $n$-dimensional, compact, smooth, connected, oriented manifold with Riemannian metric $g$ induced by its embedding in $k$-dimensional Euclidean space. This setting is sufficient to understand the main concepts in the paper, but is more restrictive than needed for the theorems.

Together with the results from Packard et al. [21] and Aeyels [2], the definitions and theorems of Takens [32] describe embedding constructions of state spaces of nonlinear dynamical systems from observations. A dynamical system is defined through its state space (here, the manifold $\mathcal{M}$) and a diffeomorphism $\phi:\mathcal{M}\to\mathcal{M}$. Here, the map $\phi$ is a discrete time dynamical system, or represents the time-$\tau$ map of a continuous-time system.

Theorem 3** (Generic delay embeddings).**

For pairs $(\phi,y)$, $\phi:\mathcal{M}\to\mathcal{M}$ a smooth diffeomorphism and $y:\mathcal{M}\to\mathbb{R}$ a smooth function, it is a generic property that the map $\Phi_{(\phi,y)}:\mathcal{M}\to\mathbb{R}^{2n+1}$, defined by

[TABLE]

is an embedding of $\mathcal{M}$; here, “smooth” means at least $C^{2}$.

Genericity as defined by Takens [32] refers to “an open and dense set of pairs $(\phi,y)$” in the $C^{2}$ function space. In general, open and dense sets can have measure zero, so Sauer et al. [26] later refined Takens’ results significantly by introducing the concept of prevalence (a “probability one” analog in infinite dimensional spaces, see 4).

Stark et al. have extended the Takens’ theorems to deterministically forced, input-output, irregularly sampled, and stochastic systems [30, 29, 31]. We do not discuss these results here.

The results of Sauer et al. [26] are presented in relation to Whitney’s theorems [34].

Definition 4**.**

A Borel subset $S$ of a normed linear space $V$ is prevalent if there is a finite-dimensional subspace $E$ of $V$ such that for each $v\in V$, $v+e$ belongs to $S$ for (Lebesgue-) almost every $e$ in $E$.

Theorem 5** (Whitney, prevalence form, embedology).**

The set $S\subset C^{1}$ of smooth maps $F:\mathbb{R}^{k}\to\mathbb{R}^{2n+1}$ that are embeddings of $\mathcal{M}$ is prevalent.

Given any smooth map $F$, there are maps arbitrarily near F that are embeddings, which is the notion of genericity from Takens. The notion of prevalence and 5 assert that “almost all” (in the sense of prevalence) of the maps near $F$ are embeddings. In these theorems, the space $E$ used in the definition of prevalence is the $k(2n+1)$-dimensional space of linear maps from $\mathbb{R}^{k}$ to $\mathbb{R}^{2n+1}$.

Appendix E Embedding theorems and optimal transport

We now prove some results concerning optimal transport in the framework of time-delay embeddings. In particular, we show that the transport maps constructed in Sections 3 and 4 exist and are invertible. We argue that in our embedding constructions, optimal transport theory is applicable.

Lemma 6**.**

Let $\mathcal{M}$ be a smooth, orientable manifold and let $g,g^{\prime}$ be two Riemannian metrics on $\mathcal{M}$. Then $\operatorname{Vol}_{g}=f\operatorname{Vol}_{g^{\prime}}$, for smooth $f:\mathcal{M}\to\mathbb{R},f>0$ or $f<0$. In particular $\operatorname{Vol}_{g}\ll\operatorname{Vol}_{g^{\prime}}$.

Proof.

On an orientable manifold, there are two possible orientations, which differ by sign. Therefore there exists a smooth function $f:\mathcal{M}\to\mathbb{R}$ such that $\operatorname{dVol}_{g}=f\operatorname{dVol}_{g^{\prime}}$ and $f>0$ or $f<0$, see [15, Chapter 15]. ∎

Lemma 7**.**

Let $(\mathcal{M},g)$ and $(\mathcal{N},g^{\prime})$ be two smooth, orientable Riemannian manifolds, both of dimension $n$ and let $\Phi:\mathcal{M}\to\mathcal{N}$ be a diffeomorphism. If $\mu$ is a measure on $\mathcal{M}$ and $\mu\ll\operatorname{Vol}_{g}$, then $\Phi_{\sharp}\mu\ll\operatorname{Vol}_{g^{\prime}}$.

Proof.

We can push $g$ forward via $\Phi$, to obtain a metric $\Phi_{\ast}g$ on $\mathcal{N}$. Then 6 implies $\operatorname{Vol}_{\Phi_{\ast}g}\ll\operatorname{Vol}_{g^{\prime}}$. Thus $\Phi_{\sharp}\mu\ll\Phi_{\sharp}\operatorname{Vol}_{g}=\operatorname{Vol}_{\Phi_{\ast}g}\ll\operatorname{Vol}_{g^{\prime}}.$ ∎

Corollary 8**.**

Let $(\mathcal{M},g)$ be a smooth, orientable Riemannian manifold of dimension $n$. Let $F:\mathcal{M}\to\mathcal{M}$ be a smooth diffeomorphism, and let $y\in C^{2}(\mathcal{M},\mathbb{R})$ a generic observable. Let $\Phi_{(F,y)}:\mathcal{M}\to\mathbb{R}^{2n+1}$ given by

[TABLE]

be the embedding from 3. Let $g^{\prime}$ be the Riemannian metric on $\Phi_{(F,y)}(\mathcal{M})$ induced by the Euclidean metric of $\mathbb{R}^{2n+1}$. If $\mu\ll\operatorname{Vol}_{g}$, then ${\Phi_{(F,y)}}_{\sharp}\mu\ll\operatorname{Vol}_{g^{\prime}}$.

Lemma 9**.**

Let $(\mathcal{M},g)$ and $(\mathcal{N},g^{\prime})$ be smooth (at least $C^{3}$), compact, connected, orientable Riemannian manifolds, without boundary and of the same dimension $n$. Let $\mu$ be a measure on $\mathcal{M}$ such that $\mu\ll\operatorname{Vol}_{g}$. Let $\Phi_{x},\Phi_{y}:\mathcal{M}\to\mathcal{N}$ be two diffeomorphisms. Consider the measures $\nu_{x}:={\Phi_{x}}_{\sharp}\mu$ and $\nu_{y}:={\Phi_{y}}_{\sharp}\mu$ on $\mathcal{N}$. Then $\nu_{x},\nu_{y}\ll\operatorname{Vol}_{g^{\prime}}$. Also the optimal transport map $T:\mathcal{N}\to\mathcal{N}$ satisfying $T_{\sharp}\nu_{x}=\nu_{y}$ exists and is invertible ($\nu_{x}$- resp. $\nu_{y}$-a.e.).

Proof.

This follows from 7 and 2. ∎

Corollary 10**.**

Let $(\mathcal{M},g)$ a be smooth (at least $C^{3}$), compact, connected, orientable Riemannian manifolds, without boundary, of dimension $n$. Let $F:\mathcal{M}\to\mathcal{M}$ be a smooth ($C^{2}$) diffeomorphism, and let $x,y\in C^{2}(\mathcal{M},\mathbb{R})$ be generic observables. Let $\Phi_{(F,x)},\Phi_{(F,y)}:\mathcal{M}\to\mathbb{R}^{2n+1}$ be the embedding defined in 3. We further assume that $\mathcal{N}:=\Phi_{(F,x)}(M)=\Phi_{(F,y)}(M)$. Let $\mu$ be a measure on $\mathcal{M}$ such that $\mu\ll\operatorname{Vol}_{g}$. The optimal transport $T:\mathcal{N}\to\mathcal{N}$ that pushes ${\Phi_{(F,x)}}_{\sharp}\mu$ to ${\Phi_{(F,y)}}_{\sharp}\mu$, exists and is invertible (a.e. with respect to these measures).

In general it is difficult to derive results on the regularity of the optimal transport map, see [33, Chapter 12]. However, for our purpose, the existence and invertability of the optimal transport map is sufficient.