Entropic Regularisation of Robust Optimal Transport

TL;DR

This paper reinterprets a recent colour transfer method as a robust optimal transport framework with entropy regularisation, providing a new perspective on the approach.

Contribution

It introduces a novel robust optimal transport formulation with entropy regularisation over marginals, unifying and extending previous colour transfer methods.

Findings

Reinterprets a colour transfer method within a robust optimal transport framework

Demonstrates the effectiveness of entropy regularisation in robust optimal transport

Provides theoretical insights connecting colour transfer and optimal transport

Abstract

Grogan et al [11,12] have recently proposed a solution to colour transfer by minimising the Euclidean distance L2 between two probability density functions capturing the colour distributions of two images (palette and target). It was shown to be very competitive to alternative solutions based on Optimal Transport for colour transfer. We show that in fact Grogan et al's formulation can also be understood as a new robust Optimal Transport based framework with entropy regularisation over marginals.

Figures

Equations

$$\hat{\varphi }=\arg \underset{\varphi }{\min }\mathcal{C}(\varphi )$$

$$\begin{array}[]{l|lcl}\mathcal{C}(\phi)=&+\frac{1}{n^{2}}\sum_{j_{1}=1}^{n}\sum_{j_{2}=1}^{n}\mathcal{N}(0;y^{(j_{1})}-y^{(j_{2})},2h^{2}\mathrm{I})&&(\mathcal{T}_{0})\\ &&&\\ &+\frac{1}{\tilde{n}^{2}}\sum_{i_{1}=1}^{\tilde{n}}\sum_{i_{2}=1}^{\tilde{n}}\mathcal{N}(0;\tilde{y}^{(i_{1})}-\tilde{y}^{(i_{2})},2h^{2}\mathrm{I})&&(\mathcal{T}_{1})\\ &&&\\ &-\frac{2}{n\tilde{n}}\sum_{i=1}^{\tilde{n}}\sum_{j=1}^{n}\mathcal{N}(0;\tilde{y}^{(i)}-y^{(j)},2h^{2}\mathrm{I})&&(\mathcal{T}_{2})\\ &&&\\ &-\lambda_{1}\frac{1}{\tilde{\tilde{n}}}\sum_{k=1}^{\tilde{\tilde{n}}}\mathcal{N}(0;\tilde{y}^{(k)}-y^{(k)},2h^{2}\mathrm{I})&&(\mathcal{T}_{3})\\ &&&\\ &+\lambda_{2}\ \mathcal{P}(\tilde{y})&&(\mathcal{T}_{4})\\ &&&\\ &+\lambda_{3}\ \mathcal{P}(\phi)&&(\mathcal{T}_{5})\\ &&&\\ \end{array}$$

$$\mu (y)=\frac{1}{n}\sum _{y^{(j)}\in \mathcal{S}}\mathcal{N}(y;y^{(j)},h^2)$$

$$\tilde{\mu }(y|\varphi )=\frac{1}{\tilde{n}}\sum _{y^{(i)}\in \tilde{\mathcal{S}}}\mathcal{N}(y;{\tilde{y}}^{(i)},h^2)$$

$${\mathcal{L}}_2(\mu ,\tilde{\mu })=\Vert \mu -\tilde{\mu }{\Vert }^2=\int (\mu (y)-\tilde{\mu }(y|\varphi ){)}^{2}dy=\underset{{\mathcal{T}}_0}{\underbrace{\Vert \mu {\Vert }^2}}\underset{{\mathcal{T}}_2}{\underbrace{-2⟨\mu |\tilde{\mu }⟩}}+\underset{{\mathcal{T}}_1}{\underbrace{\Vert \tilde{\mu }{\Vert }^2}}$$

$$\tilde{\mu }(y)=\frac{1}{\widetilde{\stackrel{~}{n}}}\sum _{y^{(k_1)}\in \widetilde{\stackrel{~}{\mathcal{S}}}}\mathcal{N}(y;y^{(k_1)},h^2\mathrm{I})\text{and}\tilde{\mu }(y)=\frac{1}{\widetilde{\stackrel{~}{n}}}\sum _{y^{(k_2)}\in \widetilde{\stackrel{~}{\mathcal{S}}}}\mathcal{N}(y;{\tilde{y}}^{(k_2)},{\tilde{h}}^2\mathrm{I})$$

$$⟨\mu |\tilde{\mu }⟩=\frac{1}{{\widetilde{\stackrel{~}{n}}}^2}\sum _{y^{(k_1)}\in \widetilde{\stackrel{~}{\mathcal{S}}}}\sum _{y^{(k_2)}\in \widetilde{\stackrel{~}{\mathcal{S}}}}\mathcal{N}(y^{(k_1)};{\tilde{y}}^{(k_2)},(h^2+{\tilde{h}}^2)\mathrm{I})$$

$$\mathcal{W}(\mu ,\tilde{\mu })=\underset{\gamma }{\min }\left\{\int \int c(y,\tilde{y})\gamma (y,\tilde{y})dyd\tilde{y}=⟨c|\gamma ⟩\right\}$$

$${\gamma }_u(y,\tilde{y}|\varphi )=\left(\frac{1}{n}\sum _{y^{(j)}\in \mathcal{S}}\mathcal{N}(y;y^{(j)},h^2\mathrm{I})\right)\times \left(\frac{1}{\tilde{n}}\sum _{{\tilde{y}}^{(i)}\in \tilde{\mathcal{S}}}\mathcal{N}(\tilde{y};{\tilde{y}}^{(i)},{\tilde{h}}^2\mathrm{I})\right)$$

$${\mu }_u(y)=\frac{1}{n}\sum _{y^{(j)}\in \mathcal{S}}\mathcal{N}(y;y^{(j)},h^2\mathrm{I})$$

$${\tilde{\mu }}_u(\tilde{y}|\varphi )=\frac{1}{\tilde{n}}\sum _{{\tilde{y}}^{(i)}\in \tilde{\mathcal{S}}}\mathcal{N}(\tilde{y};{\tilde{y}}^{(i)},{\tilde{h}}^2\mathrm{I})$$

$${\gamma }_s(y,\tilde{y}|\varphi )=\frac{1}{\widetilde{\stackrel{~}{n}}}\sum _{(y^{(k)},{\tilde{y}}^{(k)})\in \widetilde{\tilde{\mathcal{S}}}}\mathcal{N}(y;y^{(k)},h^2\mathrm{I})\mathcal{N}(\tilde{y};{\tilde{y}}^{(k)},{\tilde{h}}^2\mathrm{I})$$

$${\mu }_s(y)=\frac{1}{\widetilde{\stackrel{~}{n}}}\sum _{y^{(k)}\in \widetilde{\tilde{\mathcal{S}}}}\mathcal{N}(y;y^{(k)},h^2\mathrm{I})$$

$${\tilde{\mu }}_s(\tilde{y}|\varphi )=\frac{1}{\widetilde{\stackrel{~}{n}}}\sum _{{\tilde{y}}^{(k)}\in \widetilde{\tilde{\mathcal{S}}}}\mathcal{N}(\tilde{y};{\tilde{y}}^{(k)},{\tilde{h}}^2\mathrm{I})$$

$${\gamma }_{s+u}(y,\tilde{y}|\varphi )=(1-\lambda ){\gamma }_u(y,\tilde{y}|\varphi )+\lambda {\gamma }_s(y,\tilde{y}|\varphi )$$

$${\mu }_{s+u}(y)=(1-\lambda ){\mu }_u(y)+\lambda {\mu }_s(y)$$

$${\tilde{\mu }}_{s+u}(\tilde{y}|\varphi )=(1-\lambda ){\tilde{\mu }}_u(\tilde{y}|\varphi )+\lambda {\tilde{\mu }}_s(\tilde{y}|\varphi )$$

$$c_G(y,\tilde{y})=A-\mathcal{N}(y;\tilde{y},h_c^2\mathrm{I})$$

$$⟨c_G|\gamma ⟩=A-\int \int \mathcal{N}(y;\tilde{y},h_c^2\mathrm{I})\gamma (y,\tilde{y})dyd\tilde{y}$$

$${\rho }_G(ϵ)=1-\exp \left(-\frac{1}{2}{\left(\frac{ϵ}{\sigma }\right)}^2\right)$$

$${\rho }_G(ϵ)=1-\exp \left(-\frac{{ϵ}^2}{2{\sigma }^2}\right)\sim \frac{{ϵ}^2}{2{\sigma }^2}\text{for}ϵ<<\sigma$$

$$⟨c_G|{\gamma }_u⟩=\frac{1}{n\tilde{n}}\sum _{i=1}^{\tilde{n}}\sum _{j=1}^n\mathcal{N}\left(0;{\tilde{y}}^{(i)}-y^{(j)},(h^2+{\tilde{h}}^2+h_c^2)\mathrm{I}\right)$$

$$⟨c_G|{\gamma }_s⟩=\frac{1}{n_c}\sum _{k=1}^{n_c}\mathcal{N}\left(0;{\tilde{y}}^{(k)}-y^{(k)},(h^2+{\tilde{h}}^2+h_c^2)\mathrm{I}\right)$$

$$⟨c_G|{\gamma }_{s+u}⟩=(1-\lambda )⟨c_G|{\gamma }_u⟩+\lambda ⟨c_G|{\gamma }_s⟩(\mathcal{T})$$

$$\hat{\varphi }=\arg \underset{\varphi }{\min }\left\{\mathcal{W}({\mu }_{s+u},{\tilde{\mu }}_{s+u})=⟨c_G|{\gamma }_{s+u}⟩\right\}$$

$${\mathcal{T}}_5={\lambda }_3\int {\Vert \frac{{\partial }^2\varphi (x)}{{\partial }^{2}x}\Vert }^{2}dx$$

$$\hat{\varphi }=\arg \underset{\varphi }{\max }\int \int \mathcal{N}(y;\tilde{y},h_c^2\mathrm{I}){\gamma }_{\varphi }(y,\tilde{y})dyd\tilde{y}$$

$$\hat{\varphi }=\arg \underset{\varphi }{\max }⟨{\gamma }_m|{\gamma }_{\varphi }⟩$$

Full text

Keywords: M-estimation, $\mathcal{L}_{2}E$ estimator, Optimal Transport, Colour Transfer

1 Introduction

Optimal transport (OT) [16] has been successfully used as a way for defining cost functions for optimisation when performing colour transfer [17] and more recently in machine learning [4, 16]. The optimal transport cost (e.g Wasserstein distance) itself is also used as a similarity metric for retrieval [18]. For colour transfer (see Fig. 1111Images extracted from video https://youtu.be/FfrdyKMBVRc (demo for [12]) have been used for designing Fig. 1.), Grogan et al. [11, 12] have recently proposed an alternative approach for designing the cost function based on the $\mathcal{L}_{2}$ divergence (see section 2). This $\mathcal{L}_{2}$ based cost function is a weighted sum of multiple terms (terms $\mathcal{T}_{0,1,2,3}$ in Eq. 1) able to take into account correspondences between images (via term $\mathcal{T}_{3}$ in Eq. 1) when these are available, as well as the unsupervised scenario when no correspondence is available (via term $\mathcal{T}_{2}$ in Eq. 1). In addition, $\mathcal{L}_{2}$ includes entropies (terms $\mathcal{T}_{0}$ and $\mathcal{T}_{1}$ in Eq. 1). To further constrain the cost function when estimating the colour transformation $\phi$, additional penalties can be added to prevent colours exceeding a certain range or forcing the estimated solution $\phi$ to be smooth (resp. terms $\mathcal{T}_{4}$ and $\mathcal{T}_{5}$ in Eq. 1). The estimate $\hat{\phi}$ is computed as

[TABLE]

with:

[TABLE]

with $\mathcal{N}(z;a,\Sigma)$ indicating a normal distribution for random vector $z$ with expectation $a$ and covariance matrix $\Sigma$. $\mathrm{I}$ is the identity matrix, $h$ is a user defined bandwidth and $\lambda_{1,2,3}$ are weights. This paper aims at proposing an OT formulation for the terms $\mathcal{T}_{2}$ and $\mathcal{T}_{3}$ (see Sec. 3) as an alternative to $\mathcal{L}_{2}$ (presented in Sec. 2). In particular we show that these terms corresponds to robust Wasserstein distances where the bandwidth $h$ (Eq. 1) enables the seamless control of the level of robustness in a similar fashion as the scale parameter controlling M-estimators [13]. This reformulation allows the following contributions: first, to extend OT in supervised and semi-supervised scenarios, and second to propose a robust Wasserstein cost (Sec. 3). We start first by explaining in more detail the notations used and the $\mathcal{L}_{2}$ cost function.

2 $\mathcal{L}_{2}$ divergence

We consider that the following are available:

• •

a dataset $\mathcal{S}=\left\{y^{(j)}\right\}_{j=1,\cdots,n}$: the term $\mathcal{T}_{0}$ (Eq. 1) uses the samples from this dataset.

• •

a dataset $\widetilde{\mathcal{S}}=\left\{\tilde{y}^{(i)}=\phi(x^{(i)})\right\}_{i=1,\cdots,\tilde{n}}$ computed using a transfer (or mapping) function $\phi$ on data points $\{x^{(i)}\}_{i=1,\cdots,\tilde{n}}$. The term $\mathcal{T}_{1}$ (Eq. 1) uses the samples from this dataset.

• •

a dataset of correspondences $\widetilde{\widetilde{\mathcal{S}}}=\left\{(y^{(k)},\tilde{y}^{(k)}=\phi(x^{(k)}))\right\}_{k=1,\cdots,\tilde{\tilde{n}}}$ : the term $\mathcal{T}_{3}$ (Eq. 1) uses the samples from this dataset.

All data points have the same dimension (i.e. $\dim(y^{(l_{1})})=\dim(\tilde{y}^{(l_{2})})$) for any samples taken from $\mathcal{S}$, $\widetilde{\mathcal{S}}$ or $\widetilde{\widetilde{\mathcal{S}}}$. Figure 2 shows an illustration of our datasets in the context of colour transfer222Images from the video posted at https://twitter.com/gabrielpeyre/status/979605863295053826 have been used for designing Fig. 2..

In this $\mathcal{L}_{2}$ framework [11], only one random vector (r.v.) $y$ is defined. Using $\mathcal{S}$ and $\widetilde{\mathcal{S}}$, two probability density functions noted $\mu(y)$ and $\tilde{\mu}(y|\phi)$ respectively are computed for r.v. $y$ as kernel density estimates with a Normal kernel (or Gaussian Mixture Models):

[TABLE]

and

[TABLE]

The unknown mapping function $\phi$ transforms the samples in $\widetilde{\mathcal{S}}$ that act as the means of the normal kernels in the mixture $\tilde{\mu}(y|\phi)$. Hence, $\tilde{\mu}$ can be warped onto $\mu$ by finding the appropriate function $\phi$. The best choice for function $\phi$ can be chosen as minimising the Euclidean $\mathcal{L}_{2}$ distance between $\mu$ and $\tilde{\mu}$ defined as [14]:

[TABLE]

from which terms $\mathcal{T}_{0,1,2}$ in the cost function $\mathcal{C}(\phi)$ originate (Eq. 1). Such a formulation of $\mathcal{L}_{2}$ has been used for colour transfer [11] and shape registration [14, 1]. The connection between $\mathcal{L}_{2}$ with robust M-estimators has also been shown [3, 19, 14].

Removing $\mathcal{T}_{0}$ from the cost function $\mathcal{C}$.

$\mathcal{T}_{0}$ does not depends on $\phi$ and can be discarded, shortening $\mathcal{L}_{2}$ into $\mathcal{L}_{2}E$ [19] for estimating $\phi$. Both $\mathcal{T}_{0}$ and $\mathcal{T}_{1}$ correspond to entropies since $-\log\left(\|\mu\|^{2}\right)$ and $-\log\left(\|\tilde{\mu}\|^{2}\right)$ are the quadratic Renyi entropies of $\mu$ and $\tilde{\mu}$ respectively [11].

Using correspondences.

The term $\mathcal{T}_{3}$ to account for correspondences in $\widetilde{\widetilde{\mathcal{S}}}$, is explained intuitively with notation $-2\langle\mu|\tilde{\mu}\rangle$ by Grogan et al [12], where this time $\mu$ and $\tilde{\mu}$ are likewise kernel density estimates (with Normal kernel) using only observations in the dataset of correspondences $\widetilde{\widetilde{\mathcal{S}}}$:

[TABLE]

and the scalar product $\langle\mu|\tilde{\mu}\rangle$ then corresponds to:

[TABLE]

Hence the notation $\langle\mu|\tilde{\mu}\rangle$ is not mathematically correct to explain $\mathcal{T}_{3}$ (i.e note the single sum for $\mathcal{T}_{3}$ in Eq. 1 versus the double sum appearing in Eq. 3). So even if the intuition for $\mathcal{T}_{3}$ is sound and proves to be efficient in practice against the state of the art techniques for colour transfer [12, 11], its origin cannot be explained mathematically with $\mathcal{L}_{2}$ and we provide next a better explanation for $\mathcal{T}_{3}$ based on Optimal Transport.

3 Optimal Transport

We propose to reformulate both $\mathcal{T}_{2}$ and $\mathcal{T}_{3}$ from an OT perspective. OT aims at choosing $\phi$ with the minimum transport (displacement) cost between two random vectors noted $y$ and $\tilde{y}$. The OT cost function is expressed here with the Wasserstein distance [16] as follow:

[TABLE]

where $c$ is a cost often chosen as $c(y,\tilde{y})=\|y-\tilde{y}\|^{2}$ (quadratic Wasserstein distance), and $\gamma$ is the joint probability density function of $y$ and $\tilde{y}$ having $\mu$ and $\tilde{\mu}$ for marginals respectively i.e. $\int\gamma(y,\tilde{y})\ dy=\tilde{\mu}(\tilde{y})$ and $\int\gamma(y,\tilde{y})\ d\tilde{y}=\mu(y)$. We first present our choices for these distributions (Sec. 3.1) and then propose a new robust cost (in Sec. 3.2). An alternative OT based explanation for terms $\mathcal{T}_{2}$ and $\mathcal{T}_{3}$ then emerges (Sec. 3.3).

3.1 Models for $\gamma_{\phi}$, $\mu$ and $\tilde{\mu}_{\phi}$

Kernel density estimates with Normal kernels are used as joint density functions $\gamma_{\phi}$ and using the datasets available, three estimates of $\gamma_{\phi}\in\{\gamma_{u},\gamma_{s},\gamma_{s+u}\}$ can be proposed:

• •

using independent sets $\mathcal{S}$ and $\widetilde{\mathcal{S}}$ (unsupervised scenario i.e without correspondences):

[TABLE]

with the marginals

[TABLE]

and

[TABLE]

• •

using the set of correspondences $\widetilde{\widetilde{\mathcal{S}}}$ (supervised):

[TABLE]

providing the marginals

[TABLE]

and:

[TABLE]

• •

Using all datasets, the following mixture can be considered (semi-supervised):

[TABLE]

where $0\leq\lambda\leq 1$ is a parameter controlling the importance between the estimates $\gamma_{u}$ and $\gamma_{s}$. In this case, the marginals are:

[TABLE]

and

[TABLE]

Note that these models noted $\gamma_{\phi}\in\{\gamma_{u},\gamma_{s},\gamma_{s+u}\}$ are parameterized by $\phi$ via the samples $\tilde{y}^{(l)}$ in $\widetilde{\mathcal{S}}$ and $\widetilde{\widetilde{\mathcal{S}}}$. The bandwidths $h$ and $\tilde{h}$ are user defined and using $h=\tilde{h}=0$ enables the recovery of the empirical pdf estimates with Dirac kernels.

3.2 Robust cost $c_{G}(y,\tilde{y})$

Concave functions $g$ to define costs $c$ of the form $c(y,\tilde{y})=g(|y-\tilde{y}|)$ have been suggested for robustness [8]. Here, we go further by proposing the following robust cost:

[TABLE]

where $A$ is a constant that can be added if one need to enforce a positive cost $c_{G}$. Our cost $c_{G}$ is convex near the origin $\|y-\tilde{y}\|\sim 0$ and then becomes concave as the difference $\|y-\tilde{y}\|$ increases. We also note that:

[TABLE]

since $\gamma$ integrates to 1 by definition. In practice, for estimation of $\phi$ that minimizes this cost, the constant $A$ does not matter and can be set $A=0$.

3.2.1 Relation to M-estimators

With the more familiar notation for error $\epsilon=\|y-\tilde{y}\|$, our robust cost $c_{G}$ is proportional to the Welsch-Leclerc loss $\rho_{G}$ [2]:

[TABLE]

which is a well-known hard redescending M-estimating function with scale parameter $\sigma=h_{c}$ [13, 15, 7, 2]. The more the chosen function $\rho$ penalises large errors $\epsilon$, the more it is robust to outliers. See for instance in Fig. 3(a) how the hard redescending functions $\rho_{GM}$ (for Geman-McClure loss [6, 2]) and $\rho_{G}$ have an upper finite limit (equal to 1) when $\epsilon\rightarrow+\infty$ and thus prevent high residuals (outliers) to overly contribute too much when estimating $\hat{\phi}$. The non-robust Least Square function $\rho_{LS}$ is also shown and corresponds here to the quadratic Wasserstein cost $c(y,\tilde{y})=\|y-\tilde{y}\|^{2}$ that is not robust to gross errors.

3.2.2 Relation of robust cost $c_{G}$ to Wasserstein distance

When the bandwidth $h_{c}$ (or scale parameter $\sigma$) is very very large compared to $\epsilon$, using Taylor approximation of the cost shows that (cf. Fig. 3(b)):

[TABLE]

making our cost $c_{G}$ proportional to the one used in the quadratic Wasserstein distance. The bandwidth $h_{c}$ allows for the modulation of the cost from the non robust Euclidean distance ($h_{c}\rightarrow\infty$) to a more robust cost ($h_{c}$ small) for penalising high differences $\|y-\tilde{y}\|$ (or outliers).

3.3 OT perspective for terms $\mathcal{T}_{2}$ and $\mathcal{T}_{3}$

Using the definitions of our cost $c_{G}=-\mathcal{N}(y;\tilde{y},h_{c}^{2}\mathrm{I})$ and our joint probability density functions $\gamma_{\phi}\in\{\gamma_{u},\gamma_{s},\gamma_{s+u}\}$ (cf. Sec. 3.1), we note that:

[TABLE]

hence it is equivalent to the term $\mathcal{T}_{2}$ (since the bandwidths are user defined). Likewise we note

[TABLE]

which is equivalent to $\mathcal{T}_{3}$ (Eq. 1) introduced by Grogan et al to take advantage of correspondences [11]. Since the weight $\lambda_{1}$ was chosen in an ad hoc fashion, we can propose a more elegant alternative form combining $\mathcal{T}_{2}$ and $\mathcal{T}_{3}$ into a new term $\mathcal{T}$ using the estimate $\gamma_{s+u}$:

[TABLE]

With the OT formulation (Eq. 4), Grogan et al’s estimation (terms $\mathcal{T}_{2}$ and $\mathcal{T}_{3}$, Eq. 1) can be rewritten:

[TABLE]

to which entropic terms on the marginals $\mu$ and $\tilde{\mu}$ ($\mathcal{T}_{0}$ and $\mathcal{T}_{1}$) can be added along with other constraints on $\phi$ (e.g. $\mathcal{T}_{4}$ and $\mathcal{T}_{5}$).

When setting $h=\tilde{h}=0$ for simplicity (i.e. using empirical pdf estimates with Dirac kernels, Sec. 3.1), Grogan et al’s terms $\mathcal{T}_{2}$ and $\mathcal{T}_{3}$ are robust OT distances where the parameter $h_{c}$ in the robust cost $c_{G}$ controls the influence of outliers when performing estimation of the mapping function $\phi$ in the same way as the scale parameter for M-estimation.

3.4 Parametric Modelling of the transfer function $\phi$

In practice, a parametric form of $\phi$ is used: Thin Plate Splines (TPS) have been used for colour transfer and shape registration [14, 11, 12]. The term $\mathcal{T}_{5}$ in Eq. 1 corresponds to a smoothness constraint on the TPS solution [14, 11]:

[TABLE]

However TPS is not a convenient formulation when modelling transfer functions in high dimensional spaces and Deep Neural Networks are now providing more powerful formulations for $\phi$.

3.5 Interpretation and Generalization of the cost $c_{G}$

Our formulation of OT is equivalent to :

[TABLE]

where more generally $\mathcal{N}(y;\tilde{y},h_{c}^{2}\mathrm{I})$ can be understood as a conditional pdf ($y$ given $\tilde{y}$ or vice versa since the Normal distribution is symmetric w.r.t. its mean). Using a flat prior for $\tilde{y}$ (e.g. $\tilde{y}\sim\mathcal{N}(\tilde{y};0,a\mathrm{I})$ with bandwidth $a$ very large to approximate a flat prior), then a model for the joint probability density function is available $\gamma_{m}(y,\tilde{y})=\mathcal{N}(y;\tilde{y},h_{c}^{2}\mathrm{I})\times\mathcal{N}(\tilde{y};0,a\mathrm{I})$ and our OT formulation (Eq. 17) is equivalent to:

[TABLE]

which has the same form as the cross product $\langle\mu|\tilde{\mu}\rangle$ appearing in $\mathcal{L}_{2}$ (cf. Eq. 2): as indicated in [11], the main difference between the two frameworks lies in the modelling of one r.v. ($y$ in $\mathcal{L}_{2}$, with notation $\langle.|.\rangle$ indicating integration over this one vector) or two r.v. ($y$ and $\tilde{y}$ in OT, $\langle.|.\rangle$ indicating integration over these two vectors). These scalar products between probability densities functions (joint, marginals or conditionals) are frequent for robust estimation including for instance the Hough transform widely used in image processing [5, 7, 9]. While some robust costs can be identified as a negative log likelihood [6, 2], we identify directly our robust cost $c_{G}$ as a negative Multivariate Normal distribution instead.

4 Final remarks

We have proposed a new generic formulation for Optimal Transport with the following advantages:

• •

it is robust: our new robust cost $c_{G}(y,\tilde{y})=-\mathcal{N}(y;\tilde{y},h_{c}^{2}\mathrm{I})$ is parameterised by a bandwidth $h_{c}$ that acts like the scale parameter of M-estimators. This bandwidth enables the control of the level of robustness and when chosen very large, it makes our cost converge towards the standard (non robust) quadratic Wasserstein distance.

• •

Our formulation can seamlessly consider various scenarios e.g. unsupervised, supervised (with correspondences) or semi-supervised depending on the dataset(s) available.

• •

Grogan et al [11] propose the use of entropy terms for the marginals (e.g. $\tilde{\mu}$) that can be used in addition to (or instead of) an entropy on the joint pdf $\gamma$ [16].

• •

More generally, we have shown the commonality of these formulations ($\mathcal{L}_{2}$ and OT) in using scalar products between two p.d.fs. The main difference between $\mathcal{L}_{2}$ and OT is then in the number of random vectors used in the formulation of this scalar product. We believe this thinking extends to the Gromov-Wasserstein formulation which defines 4 random vectors [20].

Beyond the impact of our formulation for colour transfer [12, 11], future work will investigate shape registration with correspondences (e.g. for user interactions) and with kernels other than Gaussian better suited to directional data [10].

Acknowledgments

This work is partly supported by a scholarship from Umm Al-Qura University, Saudi Arabia, and in part by a research grants from Science Foundation Ireland (SFI) (Grant Number 15/RP/2776), and the ADAPT Centre for Digital Content Technology (www.adaptcentre.ie) that is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.