Hausdorff and Wasserstein metrics on graphs and other structured data

TL;DR

This paper extends optimal transport metrics, including Wasserstein and Hausdorff, from set matching to structured data like graphs, enabling efficient, structure-preserving comparisons across diverse data types.

Contribution

It introduces a framework for Wasserstein and Hausdorff metrics on structured data, generalizing optimal transport to graphs and other structures via category theory.

Findings

Wasserstein metrics on structured data are convex relaxations of Hausdorff metrics.

The Wasserstein metric on $C$-sets is computable via linear programming.

The approach applies to various graph types and other structured data.

Abstract

Optimal transport is widely used in pure and applied mathematics to find probabilistic solutions to hard combinatorial matching problems. We extend the Wasserstein metric and other elements of optimal transport from the matching of sets to the matching of graphs and other structured data. This structure-preserving form of optimal transport relaxes the usual notion of homomorphism between structures. It applies to graphs, directed and undirected, labeled and unlabeled, and to any other structure that can be realized as a $C$-set for some finitely presented category $C$. We construct both Hausdorff-style and Wasserstein-style metrics on $C$-sets and we show that the latter are convex relaxations of the former. Like the classical Wasserstein metric, the Wasserstein metric on $C$-sets is the value of a linear program and is therefore efficiently computable.

Equations

$$\mathsf{Th}\left(\mathsf{Graph}\right)=\left\{\leavevmode\hbox to57.24pt{\vbox to24.24pt{\pgfpicture\makeatletter\hbox{\hskip 28.618pt\lower-17.15967pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-28.618pt}{-7.07637pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 8.2847pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.97916pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${E}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 8.2847pt\hfil&\hfil\hskip 32.33328pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.02777pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${V}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 8.33331pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-11.8486pt}{-2.16527pt}\pgfsys@lineto{11.35141pt}{-2.16527pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{11.55139pt}{-2.16527pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.35556pt}{0.1875pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{src}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-11.8486pt}{-6.98747pt}\pgfsys@lineto{11.35141pt}{-6.98747pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{11.55139pt}{-6.98747pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.52084pt}{-13.6458pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{tgt}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{ {}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\right\}.$$

$$\mathsf{Th}\left(\mathsf{SGraph}\right)=\left\{\leavevmode\hbox to81.58pt{\vbox to24.24pt{\pgfpicture\makeatletter\hbox{\hskip 52.96309pt\lower-17.15967pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-28.618pt}{-7.07637pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 8.2847pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.97916pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${E}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 8.2847pt\hfil&\hfil\hskip 32.33328pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.02777pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${V}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 8.33331pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{} {{}{{}{}}{}}{{}{{}{}}{}}{{}{}}{{}} {{}{{}{}}{}}{{{}}{{}}}{{}}{{}{{}{}}{}}{{{}}{{}}}{ {}{}{}}{}{{}}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{{{{{{}}{ {}{}}{}{}{{}{}}}}}{}{}{}{}}{}{}{}{}{{}}{}{{}}{}{}{}{}{}{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-28.818pt}{-6.8494pt}\pgfsys@curveto{-42.55963pt}{-10.5321pt}{-42.55963pt}{1.37936pt}{-29.20432pt}{-2.19981pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{0.96591}{-0.25887}{0.25887}{0.96591}{-29.01115pt}{-2.25157pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{{}{}}}{{}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-50.81032pt}{-6.91385pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{inv}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-11.8486pt}{-2.16527pt}\pgfsys@lineto{11.35141pt}{-2.16527pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{11.55139pt}{-2.16527pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.35556pt}{0.1875pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{src}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-11.8486pt}{-6.98747pt}\pgfsys@lineto{11.35141pt}{-6.98747pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{11.55139pt}{-6.98747pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.52084pt}{-13.6458pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{tgt}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{ {}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\middle|\ {\footnotesize\begin{aligned} \operatorname{inv}^{2}=1_{E}\\ \operatorname{inv}\cdot\operatorname{src}=\operatorname{tgt}\\ \operatorname{inv}\cdot\operatorname{tgt}=\operatorname{src}\end{aligned}}\right\}.$$

$$X(c)$$

$$X(E)$$

$$\mathsf{Th}\left(\mathsf{RGraph}\right)=\left\{\leavevmode\hbox to57.24pt{\vbox to31.35pt{\pgfpicture\makeatletter\hbox{\hskip 28.618pt\lower-21.1763pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-28.618pt}{-7.07637pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 8.2847pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.97916pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${E}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 8.2847pt\hfil&\hfil\hskip 32.33328pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.02777pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${V}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 8.33331pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-11.8486pt}{2.65694pt}\pgfsys@lineto{11.35141pt}{2.65694pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{11.55139pt}{2.65694pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.35556pt}{5.0097pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{src}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-11.8486pt}{-6.98747pt}\pgfsys@lineto{11.35141pt}{-6.98747pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{11.55139pt}{-6.98747pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.52084pt}{-3.2736pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{tgt}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{11.75137pt}{-11.80968pt}\pgfsys@lineto{-11.44864pt}{-11.80968pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{-1.0}{0.0}{0.0}{-1.0}{-11.64862pt}{-11.80968pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-5.01668pt}{-19.02353pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{refl}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{ {}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\middle|\ {\footnotesize\begin{aligned} \operatorname{refl}\cdot\operatorname{src}=1_{V}\\ \operatorname{refl}\cdot\operatorname{tgt}=1_{V}\end{aligned}}\right\}.$$

$$\mathsf{Th}\left(\mathsf{BGraph}\right)=\left\{\leavevmode\hbox to97.77pt{\vbox to14.25pt{\pgfpicture\makeatletter\hbox{\hskip 48.88254pt\lower-7.1763pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-48.88254pt}{-7.07637pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 8.26456pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.95901pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${U}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 8.26456pt\hfil&\hfil\hskip 32.28467pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.97916pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${E}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 8.2847pt\hfil&\hfil\hskip 32.33328pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.02777pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${V}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 8.33331pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-8.55345pt}{-4.57637pt}\pgfsys@lineto{-31.75346pt}{-4.57637pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{-1.0}{0.0}{0.0}{-1.0}{-31.95345pt}{-4.57637pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-24.66039pt}{-2.2236pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{src}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{8.41594pt}{-4.57637pt}\pgfsys@lineto{31.61595pt}{-4.57637pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{31.81593pt}{-4.57637pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{15.7437pt}{-0.8625pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{tgt}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{ {}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\right\}.$$

$$\mathsf{\Delta}_{+}^{2}:=\left\{\leavevmode\hbox to97.08pt{\vbox to32.08pt{\pgfpicture\makeatletter\hbox{\hskip 48.53984pt\lower-20.61798pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-48.53984pt}{-7.07637pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 7.92186pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.61632pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${T}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 7.92186pt\hfil&\hfil\hskip 32.28467pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.97916pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${E}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 8.2847pt\hfil&\hfil\hskip 32.33328pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.02777pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${V}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 8.33331pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-32.49612pt}{2.65694pt}\pgfsys@lineto{-9.29611pt}{2.65694pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-9.09613pt}{2.65694pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-23.32582pt}{6.29858pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{e_{0}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-32.49612pt}{-6.98747pt}\pgfsys@lineto{-9.29611pt}{-6.98747pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-9.09613pt}{-6.98747pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-23.32582pt}{-3.34583pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{e_{1}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-32.49612pt}{-11.80968pt}\pgfsys@lineto{-9.29611pt}{-11.80968pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-9.09613pt}{-11.80968pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-23.32582pt}{-17.17633pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{e_{2}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{8.07324pt}{-2.16527pt}\pgfsys@lineto{31.27325pt}{-2.16527pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{31.47324pt}{-2.16527pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{17.05113pt}{1.47638pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{v_{0}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{8.07324pt}{-6.98747pt}\pgfsys@lineto{31.27325pt}{-6.98747pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{31.47324pt}{-6.98747pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{17.05113pt}{-12.35413pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{v_{1}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{ {}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\middle|\ {\footnotesize\begin{aligned} e_{1}v_{0}=e_{0}v_{0}\\ e_{2}v_{0}=e_{0}v_{1}\\ e_{2}v_{1}=e_{1}v_{1}\end{aligned}}\right\}.$$

$$\mathsf{Th}\left(\mathsf{DDS}\right)=\left\{\leavevmode\hbox to33.69pt{\vbox to15.23pt{\pgfpicture\makeatletter\hbox{\hskip 6.80554pt\lower-9.24544pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-6.80554pt}{-5.9861pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 6.80554pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${*}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 6.80554pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{} {{}{{}{}}{}}{{}{{}{}}{}}{{}{}}{{}} {{}{{}{}}{}}{{{}}{{}}}{{}}{{}{{}{}}{}}{{{}}{{}}}{ {}{}{}}{}{{}}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{{{{{{}}{ {}{}}{}{}{{}{}}}}}{}{}{}{}}{}{}{}{}{{}}{}{{}}{}{}{}{}{}{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{7.00554pt}{-1.60944pt}\pgfsys@curveto{20.74718pt}{2.07326pt}{20.74718pt}{-9.04546pt}{7.39186pt}{-5.4663pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{-0.96591}{0.25887}{-0.25887}{-0.96591}{7.1987pt}{-5.41454pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{19.6645pt}{-5.87775pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{T}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{ {}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\right\}.$$

$$\mathsf{Th}\left(\mathsf{ASet}\right)=\left\{\leavevmode\hbox to53.72pt{\vbox to14.25pt{\pgfpicture\makeatletter\hbox{\hskip 26.86107pt\lower-7.1763pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-26.86107pt}{-7.07637pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 6.80554pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${*}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 6.80554pt\hfil&\hfil\hskip 32.05551pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.75pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${A}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 8.05554pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-13.04999pt}{-4.57637pt}\pgfsys@lineto{10.15002pt}{-4.57637pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{10.35pt}{-4.57637pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-7.09306pt}{-2.2236pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{attr}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{ {}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\right\}.$$

$$\mathsf{Th}\left(\mathsf{VGraph}\right)=\left\{\leavevmode\hbox to97.35pt{\vbox to24.24pt{\pgfpicture\makeatletter\hbox{\hskip 48.67352pt\lower-17.15967pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-48.67352pt}{-7.07637pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 8.2847pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.97916pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${E}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 8.2847pt\hfil&\hfil\hskip 32.33328pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.02777pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${V}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 8.33331pt\hfil&\hfil\hskip 32.05551pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.75pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${A}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 8.05554pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-31.90413pt}{-2.16527pt}\pgfsys@lineto{-8.70412pt}{-2.16527pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-8.50414pt}{-2.16527pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-24.41109pt}{0.1875pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{src}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{}{{{}{}}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{{{}}}{{{}}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{{{}}{{}}}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}} {}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-31.90413pt}{-6.98747pt}\pgfsys@lineto{-8.70412pt}{-6.98747pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-8.50414pt}{-6.98747pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-24.57637pt}{-13.6458pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{tgt}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{8.76247pt}{-4.57637pt}\pgfsys@lineto{31.96248pt}{-4.57637pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{32.16246pt}{-4.57637pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{14.71939pt}{-2.2236pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\operatorname{attr}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{ {}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\right\}.$$

$$\mathsf{Graph}\cong [\mathsf{Th}\left(\mathsf{Graph}\right),\mathsf{Set}].$$

$$X(c)$$

$$\leavevmode\hbox to85.11pt{\vbox to52.74pt{\pgfpicture\makeatletter\hbox{\hskip 42.55585pt\lower-26.41937pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-42.55585pt}{-26.31944pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 14.89293pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-10.58739pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${X(c)}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 14.89293pt\hfil&\hfil\hskip 39.6629pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-11.35739pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${X(c^{\prime})}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 15.66293pt\hfil\cr\vskip 18.00005pt\cr\hfil\hskip 14.37212pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-10.06657pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${Y(c)}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 14.37212pt\hfil&\hfil\hskip 39.14207pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-10.83656pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${Y(c^{\prime})}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 15.1421pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-12.56999pt}{11.50003pt}\pgfsys@lineto{10.63002pt}{11.50003pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{10.83pt}{11.50003pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-6.03459pt}{15.2139pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{Xf}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-27.66292pt}{2.64032pt}\pgfsys@lineto{-27.66292pt}{-14.55977pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{0.0}{-1.0}{1.0}{0.0}{-27.66292pt}{-14.75975pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{{}{}}}{{}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-35.91754pt}{-7.9097pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\phi_{c}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{26.89291pt}{2.64032pt}\pgfsys@lineto{26.89291pt}{-14.55977pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{0.0}{-1.0}{1.0}{0.0}{26.89291pt}{-14.75975pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{29.24568pt}{-7.9097pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\phi_{c^{\prime}}}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-13.0908pt}{-23.81944pt}\pgfsys@lineto{11.15085pt}{-23.81944pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{11.35083pt}{-23.81944pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-5.67001pt}{-20.10558pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{Yf}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\qquad\leadsto\qquad\leavevmode\hbox to125.97pt{\vbox to52.74pt{\pgfpicture\makeatletter\hbox{\hskip 62.93147pt\lower-26.41937pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{\offinterlineskip{}{}{{{}}{{}}{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-55.5559pt}{-26.31944pt}\pgfsys@invoke{ }\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip 14.89293pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-10.58739pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${X(c)}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip 14.89293pt\hfil&\hfil\hskip 65.66301pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-11.35739pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${X(c^{\prime})}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 15.66293pt\hfil\cr\vskip 18.00005pt\cr\hfil\hskip 14.37212pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-10.06657pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${Y(c)}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 14.37212pt\hfil&\hfil\hskip 65.14218pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-10.83656pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${Y(c^{\prime})}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip 15.1421pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}{{{{}}}{{}}{{}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-25.57004pt}{11.50003pt}\pgfsys@lineto{23.63008pt}{11.50003pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{23.83006pt}{11.50003pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-11.96515pt}{15.6028pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\mathcal{M}(Xf)}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-40.66296pt}{2.64032pt}\pgfsys@lineto{-40.66296pt}{-14.55977pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{0.0}{-1.0}{1.0}{0.0}{-40.66296pt}{-14.75975pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{{}{}}}{{}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-60.7787pt}{-7.9097pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\mathcal{M}(\phi_{c})}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{39.89297pt}{2.64032pt}\pgfsys@lineto{39.89297pt}{-14.55977pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{0.0}{-1.0}{1.0}{0.0}{39.89297pt}{-14.75975pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{42.24574pt}{-7.9097pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\mathcal{M}(\phi_{c^{\prime}})}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{ {}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{ }{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-26.09085pt}{-23.81944pt}\pgfsys@lineto{24.15091pt}{-23.81944pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{24.35089pt}{-23.81944pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-11.60056pt}{-19.71667pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\mathcal{M}(Yf)}$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}.$$

$${\Phi }_c:X(c)\to Y(c),c\in \mathsf{C}$$

$$Xf\cdot {\Phi }_{c^{\prime }}={\Phi }_c\cdot Yf,\forall f:c\to c^{\prime }\text{ in }\mathsf{C},$$

$${\Phi }_c\in {\mathbb{R}}^{|X(c)|\times |Y(c)|},c\in \mathsf{C}$$

$${\Phi }_c\ge 0,{\Phi }_c\cdot \text{mathbbm}1=\text{mathbbm}1,\forall c\in \mathsf{C}$$

$$Xf\cdot {\Phi }_{c^{\prime }}={\Phi }_c\cdot Yf,\forall f:c\to c^{\prime },$$

$$X(E)$$

$${\Phi }_V(v)\sim \mathrm{Unif}(C_n(V)),v\in C_m(V),{\Phi }_E(e)\sim \mathrm{Unif}(C_n(E)),e\in C_m(E).$$

$${\mu }_{X}f:={\mu }_x\circ f^{-1}={\mu }_Y.$$

$${\Phi }_c:X(c)\to Y(c),c\in \mathsf{C}$$

$${\mu }_{X(c)}{\Phi }_c={\mu }_{Y(c)},\forall c\in \mathsf{C}$$

$$Xf\cdot {\Phi }_{c^{\prime }}={\Phi }_c\cdot Yf,\forall f:c\to c^{\prime }\text{ in }\mathsf{C},$$

$$X(c)$$

$$d_X(x,x^{\prime \prime })\le d_X(x,x^{\prime })+d_X(x^{\prime },x^{\prime \prime }),\forall x,x^{\prime },x^{\prime \prime }\in X.$$

$$d_{\infty }(f,g):=\underset{x\in X}{\sup }{d}_Y(f(x),g(x)).$$

$$\underset{x\in X}{\sup }{d}_Y(y_0,f(x))<\infty$$

$$d_{L^p}(f,g):=\{\begin{array}{ll}{\left({\int }_X{d}_Y(f(x),g(x){)}^p{\mu }_X(dx)\right)}^{1/p}& \text{when 1≤p<∞}\\ \underset{x\in X}{\mathrm{esssup}}{d}_Y(f(x),g(x))& \text{when p=∞}.\end{array}$$

$${\int }_X{d}_Y(y_0,f(x){)}^p{\mu }_X(dx)<\infty \text{for some (and hence any) y0∈Y},$$

$$d_Y(f(x),f(x^{\prime }))\le d_X(x,x^{\prime }),\forall x,x^{\prime }\in X.$$

$$d_{\infty }(fg,f{g}^{\prime })=\underset{y\in f(X)}{\sup }{d}_Z(g(y),g^{\prime }(y))\le \underset{y\in Y}{\sup }{d}_Z(g(y),g^{\prime }(y))=d_{\infty }(g,g^{\prime }).$$

$$d_{\infty }(hf,h^{\prime }f)=\underset{w\in W}{\sup }\underset{\le d_X(h(w),h^{\prime }(w))}{\underbrace{d_Y(f(h(w)),f(h^{\prime }(w)))}}\le d_{\infty }(h,h^{\prime }).$$

Full text

1. Introduction

How do you measure the distance between two graphs or, in a broader sense, quantify the similarity or dissimilarity of two graphs? Metrics and other dissimilarity measures are useful tools for mathematicians who study graphs, and also for practitioners in statistics, machine learning, and other fields who analyze graph-structured data. Many distances and dissimilarities have been proposed as part of the general study of graph matching [16, 39], yet current methods tend to suffer from one of two problems. Methods that fully exploit the graph structure, such as graph edit distances and related distances based on maximum common subgraphs [6], are generally NP-hard to compute and must be approximated by heuristic search algorithms. Methods based on efficiently computable graph substructures, such as random walks [26], shortest paths [4], or graphlets [43], are computationally tractable by design but are only sensitive to the particular substructure under consideration. Most kernel methods for graphs or other discrete structures fall into this category [21, 53]. Our aim is to construct a metric on graphs that fully accounts for the graph structure, but attains computational tractability in a principled way through convex relaxation.

The fundamental difficulty in graph matching is that the optimal correspondence of vertices between two graphs is unknown and must be estimated from a combinatorially large set of possibilities. The theory of optimal transport [51, 52, 37], now routinely used to find probabilistic matchings of metric spaces [35], suggests itself as a general strategy to circumvent this combinatorial problem. Several authors have proposed specific methods to match graphs or other structured data using optimal transport [2, 11, 36, 49].

Simplifying somewhat, current applications of optimal transport to graph matching draw on two major ideas, the Wasserstein distance between measures supported on a common metric space and the Gromov-Wasserstein distance between metric measure spaces [48, 35]. If you have a way of embedding the vertices of two graphs into a common metric space, say a Euclidean space, then you can compute the Wasserstein distance between these two subspaces [36]. Alternatively, if you have a way of converting each vertex set into its own metric space, then you can compute the Gromov-Wasserstein distance between these disjoint spaces. In the latter case, the distance between two vertices in a graph is often defined to be the length of the shortest path between them, although there are other possibilities. The two approaches, via the Wasserstein and Gromov-Wasserstein distances, can also be combined [49].

Methods of this style reduce the problem of matching graphs to that of matching metric spaces, and then apply the usual tools of optimal transport for metric matching. While this may suffice for some purposes, it is conceptually unsatisfying for the simple reason that graphs are not identical with metric spaces. Any information that cannot be encoded in the metric is lost to optimal transport. Thus, if we take the shortest path distance on vertices, the optimal coupling of vertices depends on the graph’s edges only through the lengths of the shortest paths.

Here we describe a form of optimal transport between graphs that makes no such reduction. Probabilistic mappings are established between both vertex and edge sets, and compatibility between the mappings is enforced according to the nearest analogue of graph homomorphism. These probabilistic graph homomorphisms are defined as solutions to linear programs and are therefore efficiently computable.

Our methodology is not ultimately about graphs, but about how the idea of a homomorphism, or structure-preserving map, can be deformed both probabilistically and metrically. We set forth a general notion of structure-preserving optimal transport that applies to a limited but important class of structures. This class encompasses directed, undirected, and bipartite graphs; graphs with vertex attributes, edge attributes, or both; simplicial sets, the higher-dimensional generalization of graphs; other variants of graphs, such as hypergraphs; and unrelated structures. The ensuing optimization problems are in all cases linear programs.

Other authors have proposed convex or otherwise tractable relaxations of graph matching, based on spectral methods [10], semidefinite programming [42], and doubly stochastic matrices [1]. Closest to ours is the last method, which relaxes the vertex permutation of a graph isomorphism into a doubly stochastic matrix. Unlike ours, this method does not straightforwardly generalize from graph isomorphism to graph homomorphism or to metrics on graphs, nor to graphs with vertex labels or to structures other than graphs.

Let us outline in more detail the major concepts and sections of the paper. In the mainly expository Section 2, we review $\mathsf{C}$-sets, the class of structures treated throughout. We give numerous examples of $\mathsf{C}$-sets, including several kinds of graphs. We also introduce their functorial semantics, a device we use incessantly to equip $\mathsf{C}$-sets with probabilistic and metric structure, beginning in Section 3. There we relax the $\mathsf{C}$-set homomorphism problem by replacing functions with their probabilistic analogue, Markov kernels.111Readers unfamiliar with Markov kernels will find a brief review in Appendix A. We arrive at a feasibility problem reminiscent of optimal transport, although it is expressed using Markov kernels instead of couplings.

We devote the rest of the paper to studying metrics on $\mathsf{C}$-sets, exact and probabilistic. Section 4 isolates the purely metric aspects of the problem. We establish a general method for lifting metrics on the hom-sets of a category $\mathsf{S}$ to a metric on $\mathsf{C}$-sets in $\mathsf{S}$, and we instantiate the theorem to define a Hausdorff-style metric on $\mathsf{C}$-sets. This metric, which generalizes the classical Hausdorff metric on subsets of a metric space, is generally hard to compute, but may be of theoretical interest.

We then set out to construct a Wasserstein-style metric on $\mathsf{C}$-sets, using the same framework. To do this, we must take a detour in Section 5 to define a Wasserstein metric on Markov kernels. The definition strikes us as very natural, though we can find no source for it in the literature. It is possibly of independent interest, being the probabilistic analogue of the $L^{p}$ metrics and the functional analogue of the usual Wasserstein metric. Finally, in Section 6, we bring together these threads to construct a Wasserstein metric on $\mathsf{C}$-sets. It is a convex relaxation of the Hausdorff metric on $\mathsf{C}$-sets and it is, like the classical optimal transport problem, expressible as a linear program.

The relations between the major concepts are summarized in Figure 1.1, which also serves as a dependency graph for the sections of the paper. If the meaning of this diagram is not now entirely clear, we hope that it will become so by the end.

2. Graphs, $\mathsf{C}$-sets, and functorial semantics

Graphs belong to a class of algebraic structures known as $\mathsf{C}$-sets. As a logical system, this class is extremely simple, yet it is broad enough to encompass a range of useful and important structures, such as directed and undirected graphs and their higher-dimensional generalizations. It also easily accommodates the attachment of extra data to arbitrary substructures, as in vertex- or edge-attributed graphs.

In this section we describe the essential elements of $\mathsf{C}$-sets, their morphisms, and their functorial semantics. We give many examples that should be useful in applications. Most of what we say appears in the literature on category theory [32, 33, 34, 38, 46, 47], but we assume of the reader nothing more than the definitions of a category, a functor, and a natural transformation.

Definition 2.1** ($\mathsf{C}$-sets).**

Let $\mathsf{C}$ be a small category. A $\mathsf{C}$-set222$\mathsf{C}$-sets are more commonly called “presheaves” and defined to be contravariant functors $\mathsf{C}^{\mathrm{op}}\to\mathsf{Set}$. The name “$\mathsf{C}$-sets” [38] arises from category actions, which generalize group actions ($G$-sets). is a functor $X:\mathsf{C}\to\mathsf{Set}$ from the category $\mathsf{C}$ to the category of sets and functions.

Thus, a $\mathsf{C}$-set $X$ consists of, for every object $c$ in $\mathsf{C}$, a set $X(c)$, and for every morphism $f:c\to c^{\prime}$ in $\mathsf{C}$, a function $X(f):X(c)\to X(c^{\prime})$, such that the assignment of functions preserves composition and identities.

Our categories $\mathsf{C}$ will always have finite presentations, which we regard as logical theories. A $\mathsf{C}$-set is then an instance or model of that theory. A few examples will bring this out.

Example 2.2* (Graphs).*

The theory of graphs is the category with two objects and two parallel morphisms:

[TABLE]

A functor $X:\mathsf{Th}\left(\mathsf{Graph}\right)\to\mathsf{Set}$ consists of a vertex set $X(V)$ and an edge set $X(E)$, together with source and target maps $X(\operatorname{src}),X(\operatorname{tgt}):X(E)\to X(V)$ which assign the source and target vertices of each edge. Thus, a $\mathsf{Th}\left(\mathsf{Graph}\right)$-set is simply a graph.

In this paper, a “graph” without qualification is a directed graph, possibly with multiple edges and self-loops (Figure 2.1).333Such graphs are also called “directed pseudographs,” by graph theorists, and “quivers,” by representation theorists. Different kinds of graphs arise as $\mathsf{C}$-sets for different categories $\mathsf{C}$.

Example 2.3* (Symmetric graphs).*

The theory of symmetric graphs extends the theory of graphs with an edge involution:

[TABLE]

A $\mathsf{Th}\left(\mathsf{SGraph}\right)$-set, or symmetric graph, is a graph $X$ endowed with an involution on edges, that is, a self-inverse, orientation-reversing edge map $X(\operatorname{inv}):X(E)\to X(E)$. Loosely speaking, a symmetric graph is a graph in which every edge has a matching edge going in the opposite direction (Figure 2.2). Symmetric graphs are essentially the same as undirected graphs.444In the absence of self-loops, symmetric graphs correspond one-to-one with undirected graphs, but a self-loop in an undirected graph has two possible representations in a symmetric graph: it can be fixed or not under the involution.

Before giving further examples, we define the notion of homomorphism appropriate for $\mathsf{C}$-sets.

Definition 2.4** ($\mathsf{C}$-set morphisms).**

Let $\mathsf{C}$ be a small category and let $X$ and $Y$ be $\mathsf{C}$-sets. A morphism of $\mathsf{C}$-sets from $X$ to $Y$ is a natural transformation $\phi:X\to Y$.

Thus, a $\mathsf{C}$-set morphism $\phi:X\to Y$ assigns a function $\phi_{c}:X(c)\to Y(c)$ to each object $c\in\mathsf{C}$ in such a way that for every morphism $f:c\to c^{\prime}$ in $\mathsf{C}$, there is a commutative diagram

[TABLE]

A morphism of graphs, according to this definition, is a graph homomorphism as ordinarily understood, consisting of a vertex map $\phi_{V}:X(V)\to Y(V)$ and an edge map $\phi_{E}:X(E)\to Y(E)$ that preserves the assignment of source and target vertices:

[TABLE]

In the commutative diagrams, we adopt the convention of writing simply $f$ for $X(f)$ or $Y(f)$ where no confusion can arise. Similarly, a morphism of symmetric graphs is a graph homomorphism that preserves the edge involution.

The next example shows that different categories $\mathsf{C}$ can define essentially the same $\mathsf{C}$-sets while yielding genuinely different $\mathsf{C}$-set morphisms.

Example 2.5* (Reflexive graphs).*

The theory of reflexive graphs is

[TABLE]

A reflexive graph is a graph whose every vertex is endowed with a distinguished loop (Figure 2.3). As objects, reflexive graphs are the same as graphs, inasmuch as they are in one-to-one correspondence with each other. However, morphisms of reflexive graphs can “collapse” edges into vertices by mapping them onto distinguished loops, a possibility not permitted of a graph homomorphism. For this reason reflexive graph morphisms are sometimes called “degenerate maps.”

Symmetry and reflexivity combine straightforwardly in symmetric reflexive graphs, in which the distinguished loops are fixed by the edge involution. Bipartite graphs form another important class of graphs.

Example 2.6* (Bipartite graphs).*

The theory of bipartite graphs is

[TABLE]

A bipartite graph $X$ consists of two vertex sets, $X(U)$ and $X(V)$, and a set $X(E)$ of edges with sources in $X(U)$ and targets in $X(V)$ (Figure 2.4). A morphism of bipartite graphs $\phi:X\to Y$ has two vertex maps, $\phi_{U}:X(U)\to Y(U)$ and $\phi_{V}:X(V)\to Y(V)$, and an edge map $\phi_{E}:X(E)\to Y(E)$ that preserves the source and target vertices.

We have not exhausted the list of graph-like structures that can be defined as $\mathsf{C}$-sets. For example, hypergraphs, which generalize graphs by allowing edges with multiple sources and multiple targets, are $\mathsf{C}$-sets [18, 46]. So are simplicial sets, the higher-dimensional analogue of graphs and combinatorial analogue of simplicial complexes.

Example 2.7* (Semi-simplicial sets).*

The semi-simplicial category, truncated to two dimensions, is

[TABLE]

A $\mathsf{\Delta}_{+}^{2}$-set, or two-dimensional semi-simplicial set, is a collection of triangles, edges, and vertices. Each triangle has three edges, in a definite order, and each edge has two vertices, also in a definite order, in such a way that the induced assignment of vertices to triangles is consistent, according to the simplicial identities (Figure 2.5).

Semi-simplicial sets up to any dimension $n$, or in all dimensions $n$, can be defined as $\mathsf{C}$-sets, as can several other kinds of simplicial sets [17, 19, 46]. We will not present the simplicial categories here, but we summarize the idea that graphs are one-dimensional simplicial sets in the table below.

[TABLE]

We take this list of examples to establish that many graph-like structures can be represented as $\mathsf{C}$-sets. The next example is rather trivial, but later we use its attributed variant to recover the classical Hausdorff and Wasserstein distances as special cases of metrics on $\mathsf{C}$-sets.

Example 2.8* (Sets).*

If $\mathsf{1}=\{*\}$ is the discrete category on one object, then $\mathsf{1}$-sets are sets and morphisms of $\mathsf{1}$-sets are functions.

Example 2.9* (Bisets).*

If $\mathsf{2}$ is the discrete category on two objects, then $\mathsf{2}$-sets are pairs of sets and morphisms of $\mathsf{2}$-sets are pairs of functions.

Example 2.10* (Dynamical systems).*

The theory of discrete dynamical systems is

[TABLE]

A discrete dynamical system is a set $X=X(*)$ together with a function $T:X\to X$. The set $X$ is the state space of the system and the transformation $T$ defines the transitions between states.

In applications, graphs and other structures often bear additional data in the form of discrete labels or continuous measurements. Data attributes are easily attached to $\mathsf{C}$-sets by extending the theory $\mathsf{C}$.

Example 2.11* (Attributed sets).*

The theory of attributed sets is

[TABLE]

An attributed set is thus a set $X=X(*)$ equipped with an map $X\xrightarrow{\operatorname{attr}}X(A)$ that assigns to each element $x\in X$ an attribute value $\operatorname{attr}(x)$.

Example 2.12* (Vertex-attributed graphs).*

The theory of vertex-attributed graphs is

[TABLE]

A vertex-attributed graph is a graph $X$ equipped with a map $X(V)\xrightarrow{\operatorname{attr}}X(A)$ that assigns an attribute to each vertex. Such graphs are usually called “vertex-labeled” when the attribute set $X(A)$ is discrete.

Edge-attributed, or vertex- and edge-attributed, graphs can be defined similarly. Indeed, any number of attributes can be attached to any substructure of a $\mathsf{C}$-set, making the class of $\mathsf{C}$-sets closed under attachment of extra data.

Often all the $\mathsf{C}$-sets under consideration take attributes in a common space $\mathcal{A}$, such as a fixed set of labels or a Euclidean space. In this case, we restrict the $\mathsf{C}$-set morphisms $\phi:X\to Y$ to those whose attribute maps $X(A)=\mathcal{A}\xrightarrow{\phi_{A}}\mathcal{A}=Y(A)$ are the identity $1_{\mathcal{A}}$. We will note when this restriction is in force by describing the attribute space as fixed.

At this juncture, the reader may wonder what is gained by the formalism of $\mathsf{C}$-sets over, say, an equational fragment of first-order logic or even ordinary, informal mathematics. This question has many valid answers, but the most pertinent is that viewing theories as categories in their own right makes it extremely simple to define models with extra structure, be it topological, metric, measure-theoretic, or otherwise. We simply replace the category $\mathsf{Set}$ of sets and functions with a category $\mathsf{S}$ having that extra structure.

Definition 2.13** (Functorial semantics).**

Let $\mathsf{C}$ be a small category and let $\mathsf{S}$ be any category. The functor category $[\mathsf{C},\mathsf{S}]$ has functors $\mathsf{C}\to\mathsf{S}$ as objects and natural transformations between them as morphisms. We call the objects of this category $\mathsf{S}$-valued $\mathsf{C}$-sets or $\mathsf{C}$-sets in $\mathsf{S}$.

Functorial semantics goes back to Lawvere’s pioneering thesis [30]. When $\mathsf{S}=\mathsf{Set}$, we recover the original definitions of $\mathsf{C}$-sets and $\mathsf{C}$-set morphisms. So, in our main example, the category of graphs and graph homomorphisms is the category of functors $\mathsf{Th}\left(\mathsf{Graph}\right)\to\mathsf{Set}$, that is,

[TABLE]

The starting point for much subsequent development is the category $\mathsf{Meas}$ of measurable spaces and measurable functions, defined more carefully below. A measurable $\mathsf{C}$-set, or $\mathsf{C}$-set in $\mathsf{Meas}$, is a $\mathsf{C}$-set $X$ whose internal sets $X(c)$ are equipped with $\sigma$-algebras and whose internal maps $X(f)$ are measurable with respect to these $\sigma$-algebras. We will introduce other categories as we need them. Throughout the paper we explore the consequences of enriching graphs and other $\mathsf{C}$-sets with metrics, measures, and Markov morphisms.

3. Markov morphisms of measurable $\mathsf{C}$-spaces

On the topic of matching of $\mathsf{C}$-sets, the seemingly most elementary question one can ask is:

Problem 3.1** ($\mathsf{C}$-set homomorphism).**

Given $\mathsf{C}$-sets $X$ and $Y$, does there exist a $\mathsf{C}$-set morphism $\phi:X\to Y$?

For $\mathsf{C}=\mathsf{1}$, the problem is trivial. A function $\phi:X\to Y$ exists if and only if the codomain $Y$ is nonempty. But for other categories $\mathsf{C}$ the problem is computationally hard. The graph homomorphism problem, occurring when $\mathsf{C}=\mathsf{Th}\left(\mathsf{Graph}\right)$, is a famous NP-complete problem.555The graph homomorphism problem usually refers to undirected graphs, but it is no easier for directed graphs [22, Proposition 5.10]. In the case of reflexive graphs, the homomorphism problem becomes trivial again, because there is a reflexive graph morphism $X\to Y$ if and only if codomain $Y$ is nonempty (contains a vertex). Yet the same cannot be said of similar matching problems, such as:

Problem 3.2** ($\mathsf{C}$-set isomorphism).**

Given $\mathsf{C}$-sets $X$ and $Y$, does there exist a $\mathsf{C}$-set isomorphism $X\to Y$?

The isomorphism problem for reflexive graphs is equivalent to the graph isomorphism problem, so is once again computationally hard. In summary, while the complexity depends on the category $\mathsf{C}$ and on the specific $\mathsf{C}$-sets under consideration, it is generally computationally intractable to find $\mathsf{C}$-set morphisms, to say nothing of enumerating them or optimizing over them.

A popular strategy for solving hard combinatorial problems, especially when inexact solutions are acceptable, is to relax the problem to a continuous one that is easier to solve. Functorial semantics offer a simple way to implement this strategy: replace the category $\mathsf{Set}$ with a category having better computational properties. In what will be a recurring theme, we replace categories of functions with categories of Markov kernels, which are the probabilistic analogue of functions. The reader unfamiliar with Markov kernels will find references and a short review in Appendix A.

Definition 3.3** (Category of Markov kernels).**

The category $\mathsf{Markov}$ has Polish measurable spaces666In other words, the objects are topological spaces homeomorphic to a complete, separable metric space, equipped with their Borel $\sigma$-algebras. Many results hold under weaker or no assumptions on the measurable spaces, but for simplicity we assume this regularity condition everywhere. as objects and Markov kernels between them as morphisms.

Example 3.4* (Markov chains).*

A discrete dynamical system in $\mathsf{Markov}$ is a Markov chain. Morphisms of Markov chains, as stipulated by this definition, are known to probabilists as intertwinings [55, 14].

Functions are Markov kernels that contain no randomness. To be more formal, a Markov kernel $M:X\to Y$ is deterministic if for every $x\in X$, the distribution $M(x)$ is a Dirac delta measure. Given any measurable function $f:X\to Y$, a deterministic Markov kernel $\mathcal{M}(f):X\to Y$ is defined by $\mathcal{M}(f)(x):=\delta_{f(x)}$, and every deterministic Markov kernel arises uniquely in this way. Measurable functions can therefore be identified with deterministic Markov kernels. Moreover, the identification is functorial. Given composable measurable functions $X\xrightarrow{f}Y\xrightarrow{g}Z$, one easily checks that $\mathcal{M}(f\cdot g)=\mathcal{M}f\cdot\mathcal{M}g$. Also, $\mathcal{M}(1_{X})=1_{X}$. We summarize these statements by saying that $\mathcal{M}:\mathsf{Meas}\to\mathsf{Markov}$ is an identity-on-objects embedding functor. In what follows we will not always distinguish notationally between a function $f$ and its corresponding Markov kernel $\mathcal{M}(f)$.

Let us now make precise the relaxation of the $\mathsf{C}$-set homomorphism problem. Strictly speaking, the relaxation is not from $\mathsf{C}$-sets, but from measurable $\mathsf{C}$-sets.

Definition 3.5** (Measurable $\mathsf{C}$-spaces).**

The category $\mathsf{Meas}$ has Polish measurable spaces as objects and measurable functions as morphisms.

A measurable $\mathsf{C}$-space is a $\mathsf{C}$-set in $\mathsf{Meas}$.

The sought-after relaxation is a nearly immediate consequence of the embedding $\mathsf{Meas}$ in $\mathsf{Markov}$.

Definition 3.6** (Markov morphisms).**

A Markov morphism $\Phi:X\to Y$ of measurable $\mathsf{C}$-spaces $X$ and $Y$ consists of a Markov kernel $\Phi_{c}:X(c)\to Y(c)$ for each object $c\in\mathsf{C}$, such that for every morphism $f:c\to c^{\prime}$ in $\mathsf{C}$, the diagram

[TABLE]

in $\mathsf{Markov}$ commutes.

Proposition 3.7** (Relaxation of $\mathsf{C}$-set homomorphism).**

Given measurable $\mathsf{C}$-spaces $X$ and $Y$, the problem of finding a Markov morphism $\Phi:X\to Y$ is a convex relaxation of the problem of finding a (measurable) morphism $\phi:X\to Y$.

Proof.

The proposition makes two assertions, concerning relaxation and convexity.

To establish the relaxation, observe that if $\phi:X\to Y$ is a measurable morphism, then $\mathcal{M}(\phi):=(\mathcal{M}(\phi_{c}))_{c\in\mathsf{C}}:X\to Y$ is a Markov morphism, because, by functoriality, the embedding $\mathcal{M}:\mathsf{Meas}\to\mathsf{Markov}$ preserves naturality squares:

[TABLE]

Thus, if the measurable morphism problem has a solution, so does the Markov morphism problem. To state the argument more pithily, the functor $\mathcal{M}:\mathsf{Meas}\to\mathsf{Meas}$ induces a functor $\mathcal{M}_{*}:[\mathsf{C},\mathsf{Meas}]\to[\mathsf{C},\mathsf{Markov}]$ by post-composition.

As for the convexity, the Markov morphism problem,

[TABLE]

is a convex feasibility problem, possibly in infinite dimensions. The variables, namely Markov kernels $\Phi_{c}:X(c)\to Y(c)$ indexed by $c\in\mathsf{C}$, form a convex space, and the constraints are linear in the variables. ∎

One way to think about this result is that the constraints defining a $\mathsf{C}$-set morphism, which are only formally linear, become actually linear upon relaxation. In the case of greatest practical interest, when the $\mathsf{C}$-sets are finite, the result is a linear program.

To make this as transparent as possible, let us write out the linear program. Given finite $\mathsf{C}$-sets $X$ and $Y$, identify a function $X(f):X(c)\to X(c^{\prime})$ with a binary matrix $X(f)\in\{0,1\}^{|X(c)|\times|X(c^{\prime})|}$ whose rows sum to 1 and identify a Markov kernel $\Phi_{c}:X(c)\to Y(c)$ with a right stochastic matrix $\Phi_{c}\in\mathbb{R}^{|X(c)|\times|Y(c)|}$. The Markov morphism problem is then

[TABLE]

where $\cdot$ denotes the usual matrix multiplication and $\mathbbm{1}$ denotes the column vector of all 1’s (whose dimensionality is left implicit in the notation). This feasibility problem is a linear program with linear equality constraints and nonnegativity constraints.777The category $\mathsf{C}$ may contain infinitely many morphisms, but it suffices to enforce naturality on a generating set of morphisms. Thus, assuming $\mathsf{C}$ is finitely presented, we can always write the linear program with finitely many constraints.

It will be helpful to see how Markov morphisms behave in a concrete situation.

Example 3.8* (Markov morphisms of graphs).*

Let $X$ and $Y$ be finite graphs. A Markov morphism $\Phi:X\to Y$ is a Markov kernel on vertices, $\Phi_{V}:X(V)\to Y(V)$, and a Markov kernel on edges, $\Phi_{E}:X(E)\to Y(E)$, such that the two diagrams

[TABLE]

in $\mathsf{Markov}$ commute. Since the vertex and edge maps are nondeterministic, it does not make sense to ask that the source and target vertices be preserved exactly, as in a graph homomorphism. The naturality squares assert the next best thing, that for every edge $e$ in $X(E)$, the distribution of $e$’s source vertex under $\Phi_{V}$ is equal to that of the source vertices in the edge distribution of $e$ under $\Phi_{E}$, and similarly for target vertices.

We bring out the difference between graph homomorphisms and Markov morphisms in a series of examples. Between the graphs $X$ and $Y$ of Figure 3.1, there are two graph homomorphisms $\phi_{1},\phi_{2}:X\to Y$, corresponding to the two directed paths in $Y$. Both are, of course, Markov morphisms, as is any mixture $\Phi=t\phi_{1}+(1-t)\phi_{2}$, where $t\in[0,1]$. In this case, every Markov morphism is a mixture of graph homomorphisms, so little is lost (or gained) by the relaxation. Figure 3.2 presents a similar picture. The graph $X$ is a loop and the graph $Y$ is a cycle, though not a directed one. There are no Markov graph morphisms from $X$ to $Y$, deterministic or otherwise.

Figure 3.3 looks superficially similar, with the graph $Y$ now a directed cycle, but the outcome is more interesting. As before, there is no graph homomorphism from $X$ to $Y$, but there is a Markov morphism. In fact, if $C_{n}$ is the directed cycle of length $n$, then for any $m,n\geq 1$, a Markov morphism $\Phi:C_{m}\to C_{n}$ is given by assigning the uniform distributions on all vertices and edges:

[TABLE]

In particular, it is sometimes possible to find a Markov graph morphism that is not a mixture of deterministic morphisms, proving that the notion of Markov homomorphism is genuinely weaker than graph homomorphism. That should not be surprising, given that the graph homomorphism problem is NP-hard, while the Markov graph morphism problem is a linear program, hence solvable in polynomial time.

Finally, Figure 3.4 shows the terminal graph for both deterministic and Markov morphisms. Any graph $X$ has a unique graph homomorphism, indeed a unique Markov morphism, into the loop $Y$.

One might suppose that the strategy for relaxing $\mathsf{C}$-set homomorphism carries over directly to $\mathsf{C}$-set isomorphism, but that is not so. The constraints imposed by isomorphism are bilinear, not linear, so convexity would be lost in a direct translation. The problem is not just computational, though, as the following result shows.

Proposition 3.9** (Isomorphism in $\mathsf{Markov}$ [3]).**

All isomorphisms in $\mathsf{Markov}$ are deterministic. That is, any Markov kernels $M:X\to Y$ and $N:Y\to X$ between Polish measurable spaces satisfying $M\cdot N=1_{X}$ and $N\cdot M=1_{Y}$ have the form $M=\mathcal{M}(f)$ and $N=\mathcal{M}(g)$ for measurable functions $f:X\to Y$ and $g:Y\to X$.

Proof.

Under the given assumptions, the extreme points of the convex set $\operatorname{Prob}(X)$ of probability measures on $X$ are exactly the point masses [45, Example 8.16]. If $M:X\to Y$ is an isomorphism in $\mathsf{Markov}$, then it acts as a linear isomorphism on $\operatorname{Prob}(X)$ and hence preserves the extreme points of $\operatorname{Prob}(X)$. Thus, for every $x\in X$, there exists $y\in Y$ such that $M(x)=\delta_{x}M=\delta_{y}$, which proves that $M$ is deterministic. ∎

As there is nothing to be gained, computationally or mathematically, by looking at the isomorphisms in $\mathsf{Markov}$, we will formulate the isomorphism problem in a different way. Let $X$ and $Y$ be finite $\mathsf{C}$-sets and equip every set $X(c)$ and $Y(c)$ with the counting measure. A $\mathsf{C}$-set morphism $\phi:X\to Y$ is an isomorphism if and only if each component $\phi_{c}:X(c)\to Y(c)$ is an isomorphism of sets (a bijection), and this happens if and only if each component $\phi_{c}$ preserves the counting measure, that is, for every subset $B$ of $Y(c)$, the set $B$ and its preimage $\phi_{c}^{-1}(B)$ are of the same size. With this motivation, we define:

Definition 3.10** (Measure $\mathsf{C}$-spaces).**

The category $\mathsf{Meas}_{*}$ has $\sigma$-finite measures on Polish measurable spaces as objects and measurable maps as morphisms. The category $\mathsf{Markov}_{*}$ has the same objects and Markov kernels as morphisms.

A measure $\mathsf{C}$-space is a $\mathsf{C}$-set in $\mathsf{Meas}_{*}$.

Recall that a measurable map $f:(X,\mu_{X})\to(Y,\mu_{Y})$ of measure spaces is measure-preserving if

[TABLE]

Similarly, a Markov kernel $M:(X,\mu_{X})\to(Y,\mu_{Y})$ between measure spaces is measure-preserving if $\mu_{X}M=\mu_{Y}$, as expressed in the operator notation of Definition A.3. When the measure spaces coincide, that is, $X=Y$ and $\mu_{X}=\mu_{Y}$, the measure $\mu_{X}$ is also called an invariant measure of $f$ or $M$. Let $\operatorname{\mathsf{Fix}}(\mathsf{Meas}_{*})$ and $\operatorname{\mathsf{Fix}}(\mathsf{Markov}_{*})$ denote the subcategories of $\mathsf{Meas}_{*}$ and $\mathsf{Markov}_{*}$ whose morphisms preserve measure.

Example 3.11* (Invariant dynamics).*

A discrete dynamical system in $\operatorname{\mathsf{Fix}}(\mathsf{Meas}_{*})$ is a measure-preserving dynamical system, the basic object of study in ergodic theory. A discrete dynamical system in $\operatorname{\mathsf{Fix}}(\mathsf{Markov}_{*})$ is a Markov chain together with an invariant measure.

Problem 3.12** (Measure-preserving $\mathsf{C}$-space homomorphism).**

Given measure $\mathsf{C}$-spaces $X$ and $Y$, does there exist a measure-preserving morphism $X\to Y$, i.e., a morphism $\phi:X\to Y$ whose components $\phi_{c}:X(c)\to Y(c)$ are all measure-preserving?

Due to the motivating case of finite sets and counting measures, the problem of measure $\mathsf{C}$-space homomorphism is no easier to solve than $\mathsf{C}$-set isomorphism; however, its relaxation to measure-preserving Markov morphism is easier to solve, being a convex feasibility problem.

Proposition 3.13** (Relaxation of measure-preserving $\mathsf{C}$-set morphism).**

Given measure $\mathsf{C}$-spaces $X$ and $Y$, the problem of finding a measure-preserving Markov morphism $\Phi:X\to Y$ is a convex relaxation of the problem of finding a measure-preserving morphism $\phi:X\to Y$.

Proof.

For any function $f$ on a measure space $\mu$, we have $\mu f=\mu\mathcal{M}(f)$. Thus, measure-preserving functions correspond to measure-preserving deterministic kernels, and the embedding $\mathcal{M}:\mathsf{Meas}_{*}\to\mathsf{Meas}_{*}$ restricts to an embedding $\operatorname{\mathsf{Fix}}(\mathsf{Meas}_{*})\to\operatorname{\mathsf{Fix}}(\mathsf{Markov}_{*})$. The relaxation follows by the argument of Proposition 3.7. To prove the convexity, observe that the measure-preserving Markov morphism problem,

[TABLE]

merely adds linear equality constraints to the Markov morphism problem. ∎

As before, when the $\mathsf{C}$-sets are finite, the feasibility problem is a linear program.

Insisting that Markov kernels preserve measure brings us closer to classical optimal transport, formulated in terms of couplings. For any Markov kernel $M:X\to Y$ preserving finite measures $\mu_{X}$ and $\mu_{Y}$, the product measure $\mu_{X}\otimes M$ has marginals $\mu_{X}$ and $\mu_{X}M=\mu_{Y}$ and hence is a coupling of $\mu_{X}$ and $\mu_{Y}$. Conversely, if $\pi$ is a product measure on $X\times Y$ with marginals $\mu_{X}$ and $\mu_{Y}$, then by the disintegration theorem (Theorem A.4), there exists a Markov kernel $M:X\to Y$, unique up to sets of $\mu_{X}$-measure zero, such that $\pi=\mu_{X}\otimes M$, and any such kernel $M$ satisfies $\mu_{X}M=\mu_{Y}$. So, up to null sets, couplings and measure-preserving Markov kernels are the same. They are even the same as morphisms of measure spaces. Couplings have a standard composition law, known in optimal transport as the gluing lemma [51, Lemma 7.6], and the composition laws for couplings and kernels are compatible.

Despite this equivalence, we formulate the content of this paper entirely using Markov kernels, not couplings. In order to compose couplings, one must first compute disintegrations, and, while disintegration is a linear operation, introducing it complicates the optimization problem. Also, and more importantly, we routinely use Markov kernels that are not measure-preserving. There is no correspondence between general Markov kernels and couplings. The difference, roughly speaking, is that Markov kernels are the probabilistic analogue of functions, while couplings are an analogue of bijections, or of correspondences [35].

Incidentally, workers in optimal transport have long observed that the measure-preserving property of a coupling, which in particular requires that the coupled measures have equal mass, is burdensome in applications. In response various notions of unbalanced optimal transport have been proposed [37, Section 10.2]. Markov kernels offer another alternative to couplings.

4. Hausdorff metric on metric $\mathsf{C}$-spaces

The $\mathsf{C}$-set homomorphism problem is too stringent for practical matching of graphs and other structures. Morphisms of $\mathsf{C}$-sets, even Markov morphisms, are all-or-nothing: either they exist or they do not, and when they do exist, they are distinguished only by coarse qualitative distinctions, like that of homomorphism versus isomorphism. This is problematic in scientific and statistical applications, in which the data, be it structural or numerical, is generally subject to randomness and measurement error. To be tolerant to noise, we should use an inexact, quantitative measure of structural similarity or dissimilarity. One approach to dissimilarity, possibly the most important, is via the ubiquitous mathematical concept of metric.

Throughout the rest of this paper, we develop the metric approach to matching $\mathsf{C}$-sets. We will eventually, in Section 6, construct a computationally tractable, Wasserstein-style metric on $\mathsf{C}$-sets. In this section, we focus on the purely metric aspects of the problem. The Hausdorff-style metric on $\mathsf{C}$-sets that we propose is generally hard to compute, but is helpful in isolating the metric concepts from the probabilistic. It may also be interesting in its own right, as a theoretical tool.

The central idea is to weaken the constraints defining a $\mathsf{C}$-set homomorphism from exact equality to approximate equality, with the quality of approximation determined by a metric on morphisms. Schematically, if $X$ and $Y$ are $\mathsf{C}$-sets and $\phi:X\to Y$ is a transformation, not necessarily natural, then for each morphism $f:c\to c^{\prime}$ in $\mathsf{C}$, we have a “lax” naturality square888The mapping $\phi:X\to Y$ can be seen as an enriched lax natural transformation. In this section we occasionally use the language of enriched category theory [31, 27], but always in such a way that the meaning of the terms is clear from context. We assume no knowledge of this subject.

[TABLE]

where the double arrow represents the value $d(Xf\cdot\phi_{c^{\prime}},\phi_{c}\cdot Yf)$ of some metric $d$ defined on functions $X(c)\to Y(c^{\prime})$. We aggregate these values over all morphisms, or all morphism generators, $f:c\to c^{\prime}$ in $\mathsf{C}$ to obtain an total nonnegative weight for the transformation. The Hausdorff distance $d_{H}(X,Y)$ is the weight attained by optimizing over all transformations $\phi:X\to Y$.

As a first step in rendering this idea precise, we recall the basic concepts of metric spaces and metrics on function spaces. It is convenient to work with a definition of metric that is more general than the classical notion.

Definition 4.1** (Metric spaces).**

A Lawvere metric space, which we call simply a metric space, is a set $X$ together with a function $d_{X}:X\times X\to[0,\infty]$, taking values in the extended nonnegative real numbers, that satisfies the identity law, $d_{X}(x,x)=0$ for all $x\in X$, and the triangle inequality,

[TABLE]

A metric space $(X,d_{X})$ is classical if three further axioms are satisfied:999Metrics that fail to satisfy one or more of these axioms occur often in metric geometry [5, 7], under names like “extended metrics,” “pseudometrics,” and “quasimetrics.” The term “Lawvere metric space” derives from Lawvere’s study [31] of metric spaces as categories enriched in $[0,\infty]$.

1. (i)

Finiteness: $d_{X}(x,x^{\prime})<\infty$ for all $x,x^{\prime}\in X$; 2. (ii)

Positive definiteness: $x=x^{\prime}$ whenever $d_{X}(x,x^{\prime})=0$; 3. (iii)

Symmetry: $d_{X}(x,x^{\prime})=d_{X}(x^{\prime},x)$ for all $x,x^{\prime}\in X$.

The category $\mathsf{Met}$ has metric spaces as objects and functions between them as morphisms.

Unless otherwise noted, all metrics in this paper are in the above generalized sense.

Example 4.2* (Shortest path distance).*

To reprise an example from Section 1, if $X$ is a graph, then a metric is defined on its vertices by letting $d_{X(V)}(v,v^{\prime})$ be the length of the shortest directed path from $v$ to $v^{\prime}$. The metric is finite if $X$ is finite and strongly connected, it is always positive definite, and it is symmetric if $X$ is a symmetric graph. Generalizing slightly, if $X$ is a weighted graph, where each edge carries a nonnegative weight, a metric on its vertices is defined by the shortest weighted path. This metric is positive definite if the edge weights are strictly positive.

As outlined above, we will work with $\mathsf{C}$-sets in categories $\mathsf{S}$ admitting a measure of distance between morphisms.

Definition 4.3** (Metric categories).**

A metric category is a category enriched in $\mathsf{Met}$, i.e., a category $\mathsf{S}$ whose hom-sets $\mathsf{S}(X,Y)$ each have the structure of a metric space.

Under the supremum metric, $\mathsf{Met}$ is itself a metric category. Recall that for any set $X$ and metric space $Y$, the supremum metric on functions $f,g:X\to Y$ is defined by

[TABLE]

When $Y$ is a classical metric space, the supremum metric is also classical when restricted to bounded functions, i.e., to functions $f:X\to Y$ such that

[TABLE]

for some (and hence any) $y_{0}\in Y$.

Another prominent example of a metric category comes from metric measure spaces. Later, in Section 6, we will see other examples.

Definition 4.4** (Metric measure spaces).**

A metric measure space, or mm space, is a Polish measurable space $X$ together with a metric $d_{X}$ and a $\sigma$-finite measure $\mu_{X}$. We do not assume that $d_{X}$ is a classical metric or that it metrizes the topology of $X$, although we do require that $d_{X}$ be lower-semicontinuous with respect to the topology of $X$ and so, in particular, be Borel measurable.101010Our definition of an mm space is weaker than usual. Most authors assume that $d_{X}$ metrizes the topology of $X$ [20, 52, 44], and some also require that $\mu_{X}$ has full support [35]. Here, the Polish topology of $X$ serves only as a regularity condition to exclude pathological $\sigma$-algebras.

The category $\mathsf{MM}$ has metric measure spaces as objects and measurable functions as morphisms.

Under any of the $L^{p}$ metrics, $\mathsf{MM}$ is a metric category. Recall that for any measure space $X$ and metric space $Y$, the $L^{p}$ metric on measurable functions $f,g:X\to Y$ is

[TABLE]

The essential supremum metric differs from the supremum metric only in being insensitive to sets of $\mu_{X}$-measure zero.

When $Y$ is a classical metric space, the $L^{p}$ metrics are also classical when restricted to the $L^{p}$ spaces. In this context, the space $L^{p}(X,Y)$ consists of equivalence classes of functions $f:X\to Y$ that have finite moments of order $p$,

[TABLE]

when $1\leq p<\infty$, or that are essentially bounded, when $p=\infty$. The equivalence relation is that of equality $\mu_{X}$-almost everywhere.

Definition 4.5** (Metric $\mathsf{C}$-spaces).**

A metric $\mathsf{C}$-space is a $\mathsf{C}$-set in $\mathsf{Met}$. Likewise, a metric measure $\mathsf{C}$-space, or mm $\mathsf{C}$-space, is a $\mathsf{C}$-set in $\mathsf{MM}$.

As $\mathsf{Met}$ and $\mathsf{MM}$ are metric categories, we will be able to define quantitative measures of dissimilarity between metric $\mathsf{C}$-spaces and between metric measure $\mathsf{C}$-spaces. However, in order that the dissimilarity measures be metrics, we must restrict the $\mathsf{C}$-set transformations to those whose components do not increase distances. We now formulate this requirement abstractly, for a general metric category $\mathsf{S}$.

Definition 4.6** (Short morphisms).**

A morphism $f:X\to Y$ in a metric category $\mathsf{S}$ is short if it does not increase distances upon pre-composition or post-composition. In other words, we have $d(fg,fg^{\prime})\leq d(g,g^{\prime})$ for all morphisms $g,g^{\prime}:Y\to Z$ in $\mathsf{S}$, and we have $d(hf,h^{\prime}f)\leq d(h,h^{\prime})$ for all morphisms $h,h^{\prime}:W\to X$ in $\mathsf{S}$.

The class of morphisms in $\mathsf{S}$ that do not increases distances upon pre-composition is clearly closed under composition and includes the identities, and likewise for post-composition. The short morphisms in $\mathsf{S}$ therefore form a subcategory of $\mathsf{S}$, which we denote by $\operatorname{\mathsf{Short}}(\mathsf{S})$.

Characterizing the short morphisms of metric spaces and metric measure spaces is straightforward. The first example shows that our terminology is consistent with standard usage.

Proposition 4.7** (Short morphisms of metric spaces).**

The short morphisms of metric spaces are short maps,111111Other names for short maps are “contractions,” “distance-decreasing maps,” “metric maps,” and “nonexpansive maps.” Alternatively, short maps are Lipschitz functions with Lipschitz constant 1. The category of metric spaces and short maps was first studied by Isbell [23], in the case of classical metric spaces, and by Lawvere [31], in the case of generalized metric spaces. namely functions $f:X\to Y$ of metric spaces $X,Y$ such that

[TABLE]

Consequently, $\operatorname{\mathsf{Short}}(\mathsf{Met})$ is the category of metric spaces and short maps.

Proof.

For any functions $f:X\to Y$ and $g,g^{\prime}:Y\to Z$, we have

[TABLE]

If, moreover, $f$ is a short map, then for any functions $h,h^{\prime}:W\to X$,

[TABLE]

Thus short maps are short morphisms of $\mathsf{Met}$. Conversely, if $f:X\to Y$ is a short morphism, let $e_{x}:I\to X$ denote the unique map from the terminal space $I=\{*\}$ onto $x\in X$. Then for any $x,x^{\prime}\in X$,

[TABLE]

proving that $f$ is a short map. ∎

Example 4.8* (Short maps of graphs [22, Corollary 1.2]).*

Let $X$ and $Y$ be graphs with the shortest path distance making the vertex sets into metric spaces. For any graph homomorphism $\phi:X\to Y$, the vertex map $\phi_{V}:X(V)\to Y(V)$ is a short map of metric spaces, since $\phi$ transforms any path in $X$ into a path in $Y$ of the same length. On the other hand, not every short map can be extended to a graph homomorphism (consider a constant map). Thus, shorts map of graphs are a weakening of graph homomorphisms.

Proposition 4.9** (Short morphisms of mm spaces).**

The short morphisms between metric measure spaces $X$ and $Y$ are measurable maps $f:X\to Y$ that are both measure-decreasing,121212A contravariance is involved in describing $\mu_{X}f\leq\mu_{Y}$ as $f:X\to Y$ being measure-decreasing. In fact, it is the induced set map $f^{-1}:\Sigma_{Y}\to\Sigma_{X}$ that decreases measure; the map $f$ does not act on measurable sets.

[TABLE]

and distance-decreasing,

[TABLE]

Consequently, $\operatorname{\mathsf{Short}}(\mathsf{MM})$ is the category of mm spaces and distance- and measure-decreasing maps.

Proof.

We prove only the case $1\leq p<\infty$.

First, we show that $f:X\to Y$ is measure-decreasing if and only if pre-composing with $f$ decreases $L^{p}$ distances. If $f$ is measure-decreasing, then for measurable functions $g,g^{\prime}:Y\to Z$,

[TABLE]

Conversely, if $f$ is not measure-decreasing, then there exists a set $B\in\Sigma_{Y}$ such that $\mu_{X}f(B)>\mu_{Y}(B)$. Choose a classical metric space $Z$ with at least two points $z$ and $z^{\prime}$, let $g:Y\to Z$ be the constant function at $z$, and let $g^{\prime}:Y\to Z$ be the function equal to $z^{\prime}$ on $B$ and equal to $z$ outside of $B$. Then, by construction, $d_{L^{p}}(fg,fg^{\prime})>d_{L^{p}}(g,g^{\prime})$.

Now we show that $f:X\to Y$ is distance-decreasing (a short map of metric spaces) if and only if post-composing with $f$ decreases $L^{p}$ distances. If $f$ is distance-decreasing, then for any measurable functions $h,h^{\prime}:W\to X$,

[TABLE]

For the converse direction, let $I=\{*\}$ be the singleton probability space and let $e_{x}:I\to X$ denote the generalized elements, as in the proof of Proposition 4.7. Then for any $x,x^{\prime}\in X$,

[TABLE]

proving that $f$ is distance-decreasing. ∎

We saw in Section 3 that a measure-preserving function is a surrogate for a bijection. Likewise, a measure-decreasing function is a surrogate for an injection, since a function on finite sets is measure-decreasing with respect to counting measure if and only if it is injective. For functions on finite measure spaces of equal mass, particularly probability spaces, being measure-decreasing is the same as being measure-preserving. The elementary version of this fact is that on finite sets of equal size, injections are the same as bijections.

By definition, a metric category is a category enriched in $\mathsf{Met}$. The enrichment extends to short morphisms in that for any category $\mathsf{S}$ enriched in $\mathsf{Met}$, the subcategory $\operatorname{\mathsf{Short}}(\mathsf{S})$ is enriched in $\operatorname{\mathsf{Short}}(\mathsf{Met})$. We digress slightly to clarify this statement.

Proposition 4.10**.**

Let $\mathsf{S}$ be a category. The following are equivalent:

1. (i)

The category $\mathsf{S}$ is enriched in $\operatorname{\mathsf{Short}}(\mathsf{Met})$; 2. (ii)

For any morphisms * {X}$${Y}$${Z}$$\scriptstyle{f}$$\scriptstyle{g}$$\scriptstyle{h}$$\scriptstyle{k} * in $\mathsf{S}$,

[TABLE] 3. (iii)

*For any morphisms * {X}$${Y}$${Z}$$\scriptstyle{f}$$\scriptstyle{h}$$\scriptstyle{k} * in $\mathsf{S}$, we have $d(fh,fk)\leq d(h,k)$, and for any morphisms * {X}$${Y}$${Z}$$\scriptstyle{f}$$\scriptstyle{g}$$\scriptstyle{h} , we have $d(fh,gh)\leq d(f,g)$.

Proof.

Conditions (i) and (ii) are equivalent by the definition of an enriched category. We prove that (ii) and (iii) are equivalent. If (ii) holds, then taking $f=g$ gives $d(fh,fk)\leq d(f,f)+d(h,k)=d(h,k)$. Similarly, taking $h=k$ gives $d(fh,gh)\leq d(f,g)+d(h,h)=d(f,g)$. Conversely, if (iii) holds, then by the triangle inequality,

[TABLE]

We now define a Hausdorff-style metric on $\mathsf{C}$-sets in a general metric category $\mathsf{S}$. We instantiate it with $\mathsf{S}=\mathsf{Met}$ and $\mathsf{S}=\mathsf{MM}$ below and with other metric categories in Section 6. In the definition, fix $1\leq p\leq\infty$ and write the $\ell^{p}$ norm on numbers in $[0,\infty]$ as the binary operator $+_{p}$, meaning that $a+_{p}b=(a^{p}+b^{p})^{1/p}$ when $p<\infty$ and $a+_{p}b=\max\{a,b\}$ when $p=\infty$.

Definition 4.11** (Hausdorff metric on $\mathsf{C}$-sets).**

Let $\mathsf{C}$ be a finitely presented category and let $\mathsf{S}$ be a metric category. The Hausdorff metric on $\mathsf{C}$-sets in $\mathsf{S}$ is given by, for $X,Y\in[\mathsf{C},\mathsf{S}]$,

[TABLE]

where the $\ell^{p}$ sum is over a fixed, finite generating set of morphisms in $\mathsf{C}$ and the infimum is over all transformations $\phi:X\to Y$, not necessarily natural, whose components $\phi_{c}:X(c)\to Y(c)$ in $\mathsf{S}$ are short morphisms (so belong to $\operatorname{\mathsf{Short}}(\mathsf{S})$).

The Hausdorff metric is indeed a metric, albeit not a classical one.

Theorem 4.12**.**

As defined above, the Hausdorff metric on $\mathsf{C}$-sets in $\mathsf{S}$ is a metric.

Proof.

For any $\mathsf{C}$-set $X$ in $\mathsf{S}$, taking the identity transformation $1_{X}:X\to X$ gives

[TABLE]

proving that $d_{H}(X,X)=0$.

So we need only verify the triangle inequality. Let $X$, $Y$, and $Z$ be $\mathsf{C}$-sets in $\mathsf{S}$. Fixing a morphism $f:c\to c^{\prime}$ in $\mathsf{C}$, define the weight of a transformation $\phi:X\to Y$ at $f$ to be the number $|\phi|_{f}:=d(Xf\cdot\phi_{c^{\prime}},\phi_{c}\cdot Yf)$. We will show that, for any composable transformations $X\xrightarrow{\phi}Y\xrightarrow{\psi}Z$ with components in $\operatorname{\mathsf{Short}}(\mathsf{S})$, we have a triangle inequality

[TABLE]

The proof then follows readily. By the triangle inequalities for the weights $|{-}|_{f}$ and for the $\ell^{p}$ sum,

[TABLE]

Take the infimum over the transformations $\phi:X\to Y$ and $\psi:Y\to Z$ to conclude that $d_{H}(X,Y)\leq d_{H}(X,Y)+d_{H}(Y,Z)$.

To prove the triangle inequality for the weights, let $X\xrightarrow{\phi}Y\xrightarrow{\psi}Z$ be composable transformations and consider a “pasting diagram” of form

[TABLE]

Here is the argument encoded informally by this diagram, which is easy to understand if cumbersome to write down. Since the components of $\phi$ and $\psi$ are short morphisms, we have

[TABLE]

and

[TABLE]

Therefore, by the triangle inequality in the hom-space $\mathsf{S}(X(c),Z(c^{\prime}))$,

[TABLE]

This completes the proof of the triangle inequality for $|{-}|_{f}$ and hence also for $d_{H}$. ∎

The assumptions of the theorem can be weakened or strengthened with concomitant effects on the conclusion.

Remark 4.13* (General cost functions).*

As the proof shows, the restriction to short morphisms is what guarantees the triangle inequality. Thus, in situations where the triangle inequality is inessential, we are free to take the infimum over arbitrary transformations $\phi:X\to Y$ with components in $\mathsf{S}$. We may as well also allow the hom-sets of $\mathsf{S}$ to carry general cost functions, not necessarily metrics.

Remark 4.14* (Generators and composites).*

From the coordinate-free perspective of categorical logic, the restriction to a finite generating set of morphisms in $\mathsf{C}$ is open to criticism, since it makes the Hausdorff metric depend on how the category $\mathsf{C}$ is presented. Can anything be said about the deviation from naturality for a generic morphism in $\mathsf{C}$? In general, no. If, however, we strengthen the assumptions to include $X$ and $Y$ being $\mathsf{C}$-sets in $\operatorname{\mathsf{Short}}(\mathsf{S})$, not merely in $\mathsf{S}$, then for any composable morphisms $c\xrightarrow{f}c^{\prime}\xrightarrow{g}c^{\prime\prime}$ in $\mathsf{C}$, an argument by pasting diagram of form

[TABLE]

shows that for all transformations $\phi:X\to Y$,

[TABLE]

Thus, for $\mathsf{C}$-sets $X$ and $Y$ in $\operatorname{\mathsf{Short}}(\mathsf{S})$, bounds on the weights of $\phi:X\to Y$ at generators yield bounds at composites.

We emphasize that the Hausdorff metric on $\mathsf{S}$-valued $\mathsf{C}$-sets is not a classical metric. Even when the underlying metrics on $\mathsf{S}$ are symmetric, the Hausdorff metric is not, although it can be symmetrized in one of the usual ways, such as

[TABLE]

As a more fundamental matter, the Hausdorff metric is not positive definite, since $d_{H}(X,Y)=0$ whenever there exists a $\mathsf{C}$-set homomorphism $\phi:X\to Y$ with components in $\operatorname{\mathsf{Short}}(\mathsf{S})$. Similar statements apply to the symmetrized metrics. Indeed, under any reasonable definition, the distance between isomorphic $\mathsf{C}$-sets should be zero, so we cannot expect to get positive definiteness without passing to equivalence classes of isomorphic $\mathsf{C}$-sets. This is not a matter we will pursue.

We turn now to concrete examples. We frequently need to make a metric space out of a set with no metric structure. There are several generic ways to do this, but the most useful is the discrete metric, defined on a set $X$ as

[TABLE]

A map $f:X\to Y$ out of a discrete metric space $X$ is always short.131313The discrete metric is usually taken to be 1, not $\infty$, off the diagonal. Both metrics generate the same topology—the discrete topology—but only the infinite discrete metric satisfies a universal property in $\operatorname{\mathsf{Short}}(\mathsf{Met})$.

We can make a $\mathsf{C}$-set $X$ into a metric $\mathsf{C}$-space by equipping every set $X(c)$, $c\in\mathsf{C}$, with the discrete metric. On such discrete metric $\mathsf{C}$-spaces, the Hausdorff metric reduces to the $\mathsf{C}$-set homomorphism problem:

[TABLE]

It can therefore be at least as hard to compute the Hausdorff metric on metric $\mathsf{C}$-spaces as it is to solve the $\mathsf{C}$-set homomorphism problem, which is generally NP-hard. Overcoming such computational difficulties is a major motivation for Sections 5 and 6.

More interesting things happen when the underlying metrics are not all discrete. As a first example, we show that the classical Hausdorff metric on subsets of a metric space is a special case of the Hausdorff metric on $\mathsf{C}$-sets, justifying our terminology.

Example 4.15* (Classical Hausdorff metric).*

Let $\mathsf{C}=\left\{*\xrightarrow{\operatorname{attr}}A\right\}$ be the theory of attributed sets, as in Example 2.11. Let $X$ and $Y$ be attributed sets under the discrete metric, with attributes valued in a fixed metric space $\mathcal{A}=X(A)=Y(A)$. Then $X$ and $Y$ are metric $\mathsf{C}$-spaces and the Hausdorff distance between them is

[TABLE]

where the first infimum is over all functions $f:X\to Y$. If $X\xrightarrow{\operatorname{attr}}\mathcal{A}$ and $Y\xrightarrow{\operatorname{attr}}\mathcal{A}$ are injective, we can identify $X$ and $Y$ with subsets of the metric space $\mathcal{A}$. The Hausdorff distance then simplifies to

[TABLE]

which is the classical Hausdorff metric in non-symmetric form. Assuming $\mathcal{A}$ is a symmetric metric space, we recover the standard Hausdorff metric upon symmetrization:

[TABLE]

In the next two examples, we define several possible Hausdorff metrics on graphs.

Example 4.16* (Hausdorff metric on attributed graphs).*

Let $X$ and $Y$ be vertex-attributed graphs, as in Example 2.12, with discrete metrics on the vertex and edge sets and arbitrary metrics on the attribute sets. Then $X$ and $Y$ are attributed graphs in $\mathsf{Met}$ and the Hausdorff distance between them is

[TABLE]

where the infimum is over graph homomorphisms $\phi=(\phi_{V},\phi_{E}):X\to Y$ and short maps $\phi_{A}:X(A)\to Y(A)$.

This metric optimizes both the graph homomorphism and the matching of the attribute spaces $X(A)$ and $Y(A)$. If instead we fix a metric space $\mathcal{A}$ for the attributes, the Hausdorff distance is

[TABLE]

where the infimum is now only over graph homomorphisms $\phi:X\to Y$.

Example 4.17* (Weak Hausdorff metric on graphs).*

Let $X$ and $Y$ be graphs, with discrete metrics on the edge sets, the shortest path distances of Example 4.2 on the vertex sets, and counting measures on both vertex and edge sets. Then $X$ and $Y$ are graphs in $\mathsf{MM}$ and, for $p=1$, the Hausdorff distance between them is

[TABLE]

where the infimum is over injective maps $\phi_{E}:X(E)\to Y(E)$ of the edges and injective short maps $\phi_{V}:X(V)\to Y(V)$ of the vertices. If $X$ is monomorphic to $Y$, that is, there is an injective graph homomorphism from $X$ to $Y$, then $d_{H}(X,Y)=0$. But, unlike the previous example, we can still have $d_{H}(X,Y)<\infty$ even when $X$ is not homomorphic to $Y$, because the edge map is allowed violate the source and target constraints.

A concrete example is shown in Figure 4.1. The 2-cycle $X$ is plainly not homomorphic to the 4-cycle $Y$, but, due to the pictured transformation, the Hausdorff distance from $X$ to $Y$ is $d_{H}(X,Y)=2$. In general, if $C_{n}$ is the directed cycle of length $n$, then it can be shown that $d_{H}(C_{m},C_{n})=\min\{m,n-m\}$ for any $1\leq m\leq n$. This quantity is the length of the shortest path between the endpoints of an $m$-path on the $n$-cycle. In the other direction, $d_{H}(C_{m},C_{n})=\infty$ when $m>n$ since there is no longer any injection from $C_{m}$ to $C_{n}$.

A stream of further examples can be generated by combining the features above, namely data attributes and metric weakening of the homomorphism constraints, in graphs or in other $\mathsf{C}$-sets, such as symmetric graphs, reflexive graphs, and their higher-dimensional generalizations.

5. Wasserstein metric on Markov kernels

Our goal is now to define a Wasserstein-style metric on metric measure $\mathsf{C}$-spaces, thus bringing together the threads of the two preceding sections. As a first step, we define a metric on Markov kernels, to serve the same role for the Wasserstein metric as the supremum or $L^{p}$ metrics do for the Hausdorff metric. Defining a metric on Markov kernels is more subtle than defining a metric on functions, and will be the subject of this section.

The Wasserstein metric on Markov kernels generalizes the Wasserstein metric on probability distributions. Our development parallels that of the classical metric theory for optimal transport, to be found, for instance, in [51, Chapter 7] or [52, Chapter 6]. In this spirit, we begin with two notions of coupling for Markov kernels.141414What we call “products” of Markov kernels are called “couplings” in the literature on Markov chains [15, Section 19.1.2], while our notion of “coupling” seems to have no established name.

Definition 5.1** (Couplings and products).**

A coupling of Markov kernels $M:X\to Y$ and $N:X\to Z$ is any Markov kernel $\Pi:X\to Y\times Z$ with marginal $M$ along $Y$ and marginal $N$ along $Z$. That is, $\Pi\cdot\operatorname{proj}_{Y}=M$ and $\Pi\cdot\operatorname{proj}_{Z}=N$, where $\operatorname{proj}_{Y}:Y\times Z\to Y$ and $\operatorname{proj}_{Z}:Y\times Z\to Z$ are the canonical projection maps.

A product of Markov kernels $M:W\to Y$ and $N:X\to Z$ is any Markov kernel $\Pi:W\times X\to Y\times Z$ with marginal $M$ along $W$ and $Y$ and marginal $N$ along $X$ and $Z$. That is, $\Pi\cdot\operatorname{proj}_{Y}=\operatorname{proj}_{W}\cdot M$ and $\Pi\cdot\operatorname{proj}_{Z}=\operatorname{proj}_{X}\cdot N$, where $\operatorname{proj}_{W}:W\times X\to W$ and $\operatorname{proj}_{X}:W\times X\to X$ are the evident projections.

The set of all couplings of Markov kernels $M$ and $N$ is denoted by $\operatorname{Coup}(M,N)$ and the set of all products by $\operatorname{Prod}(M,N)$.

To phrase it differently, a Markov kernel $\Pi:X\to Y\times Z$ is a coupling of $M:X\to Y$ and $N:X\to Z$ if for every $x\in X$, the probability distribution $\Pi(x)$ is a coupling of $M(x)$ and $N(x)$. Similarly, a Markov kernel $\Pi:W\times X\to Y\times Z$ is a product of $M:W\to Y$ and $N:X\to Z$ if for every $w\in W$ and $x\in X$, the distribution $\Pi(w,x)$ is a coupling of $M(w)$ and $N(x)$. In the special case where $W$ and $X$ are singleton sets, couplings and products of Markov kernels coincide and amount to couplings of probability measures.

The set of products of Markov kernels $M:W\to Y$ and $N:X\to Z$ is never empty, because one can always take the independent product,

[TABLE]

given by the pointwise independent product of probability measures. When the kernels $M$ and $N$ share a common domain $X$, products are extensions of couplings, because any product $\Pi\in\operatorname{Prod}(M,N)$ gives rise to a coupling $\Delta_{X}\cdot\Pi\in\operatorname{Coup}(M,N)$ by pre-composing with the diagonal map $\Delta_{X}:x\mapsto(x,x)$. In particular, the set of couplings is never empty either.

Definition 5.2** (Wasserstein metrics on Markov kernels).**

Let $X$ and $Y$ be metric measure spaces. For any exponent $1\leq p<\infty$, the Wasserstein metric of order $p$ on Markov kernels $M,N:X\to Y$ is

[TABLE]

This metric generalizes two famous constructions in analysis. When the kernels are deterministic, we recover the $L^{p}$ metric on functions between metric measure spaces, reviewed in Section 4. When $X=\{*\}$ is a singleton set, the kernels $M$ and $N$ can be identified with probability measures $\mu=M(*)$ and $\nu=N(*)$, and we recover the classical Wasserstein metric on probability measures,

[TABLE]

The relationships between the base metric and its derived metrics are summarized in the diagram of Figure 5.1.

It is possible to define a Wasserstein metric on Markov kernels in the case $p=\infty$, generalizing the $L^{\infty}$ metric on functions and the $W_{\infty}$ metric on probability measures [9], [41, Section 3.2]. We do not pursue this case here, as the optimization problem ceases to be linear in the coupling $\Pi$.

We need to verify that the proposed metric on Markov kernels is actually a metric. As in the proof for the classical Wasserstein metric, the main property to verify is the triangle inequality, and crucial step in doing so is establishing a gluing lemma. Loosely speaking, the gluing lemma says that Markov kernels into $X\times Y$ and $Y\times Z$ that share a common marginal along $Y$ can be glued along $Y$ to form a Markov kernel into $X\times Y\times Z$.

Lemma 5.3** (Gluing lemma for Markov kernels).**

Let $W$ be a measurable space and $X,Y,Z$ be Polish measurable spaces. Suppose $M_{X}:W\to X$, $M_{Y}:W\to Y$, and $M_{Z}:W\to Z$ are Markov kernels, and $\Pi_{XY}\in\operatorname{Coup}(M_{X},M_{Y})$ and $\Pi_{YZ}\in\operatorname{Coup}(M_{Y},M_{Z})$ are couplings thereof. Then there exists a Markov kernel $\Pi_{XYZ}:W\to X\times Y\times Z$ with marginals $\Pi_{XY}$ along $X\times Y$ and $\Pi_{YZ}$ along $Y\times Z$.

Proof.

By the disintegration theorem for Markov kernels (Theorem A.5), there exist Markov kernels $\Pi_{X\,|\,Y}:W\times Y\to X$ and $\Pi_{Z\,|\,Y}:W\times Y\to Z$ such that, using a mild abuse of notation,

[TABLE]

and

[TABLE]

Then the Markov kernel $\Pi_{XYZ}:W\to X\times Y\times Z$ defined by

[TABLE]

satisfies the desired properties. ∎

By a variant of the usual gluing argument, we show that the Wasserstein metric on Markov kernels is indeed a metric.

Theorem 5.4**.**

Let $X$ and $Y$ be metric measure spaces. For any $1\leq p<\infty$, the Wasserstein metric of order $p$ on Markov kernels $X\to Y$ is a metric.

Moreover, if $Y$ is a classical metric space, then the Wasserstein metric is also classical when restricted to equivalence classes of Markov kernels with finite moments of order $p$, i.e., to Markov kernels $M:X\to Y$ such that

[TABLE]

for some (and hence any) $y_{0}\in Y$, where we regard Markov kernels $M$ and $M^{\prime}$ as equivalent if $M(x)=M^{\prime}(x)$ for $\mu_{X}$-almost every $x\in X$.

Proof.

We prove the triangle inequality first. Let $M_{1},M_{2},M_{3}:X\to Y$ be Markov kernels and let $\Pi_{12}\in\operatorname{Coup}(M_{1},M_{2})$ and $\Pi_{23}\in\operatorname{Coup}(M_{2},M_{3})$ be couplings thereof. By the gluing lemma, there exists a Markov kernel $\Pi:X\to Y\times Y\times Y$ such that $\Pi\cdot\operatorname{proj}_{12}=\Pi_{12}$ and $\Pi\cdot\operatorname{proj}_{23}=\Pi_{23}$, where $\operatorname{proj}_{ij}:Y^{3}\to Y^{2}$, $i<j$, are the evident projections. Forming the coupling $\Pi_{13}:=\Pi\cdot\operatorname{proj}_{13}\in\operatorname{Coup}(M_{1},M_{3})$, we estimate

[TABLE]

where we apply the triangle inequality in the second inequality and Minkowski’s inequality on $L^{p}(X\times Y^{3},\mu_{X}\otimes\Pi)$ in the third inequality. Since the resulting inequality holds for any couplings $\Pi_{12}$ and $\Pi_{23}$, we conclude that

[TABLE]

Moreover, for any Markov kernel $M:X\to Y$, the deterministic coupling $M\cdot\Delta_{Y}$, where $\Delta_{Y}:Y\to Y\times Y$ is the diagonal map, yields

[TABLE]

proving that $W_{p}(M,M)=0$. Thus $W_{p}$ is a metric in generalized sense.

For the second part of the theorem, suppose $Y$ is a classical metric space. If Markov kernels $M$ and $M^{\prime}$ satisfy $W_{p}(M,M^{\prime})=0$, then, assuming for the moment that the infimum is achieved, there exists a coupling $\Pi\in\operatorname{Coup}(M,M^{\prime})$ such that

[TABLE]

Thus, for $\mu_{X}$-almost every $x\in X$,

[TABLE]

For each such $x$, since the metric $d_{Y}$ is positive definite, $\Pi(x)$ is concentrated on the diagonal. Thus $\Pi(x)=\nu\cdot\Delta_{Y}$ for some probability measure $\nu$ on $Y$ and hence $\Pi(x)\in\operatorname{Coup}(\nu,\nu)$. But, by assumption, $\Pi(x)\in\operatorname{Coup}(M(x),M^{\prime}(x))$, and so $M(x)=\nu=M^{\prime}(x)$. Thus $M(x)=M^{\prime}(x)$ for $\mu_{X}$-a.e. $x\in X$, and we conclude that $W_{p}$ is positive definite. Next, $W_{p}$ is symmetric since the base metric $d_{Y}$ is. Finally, by taking the independent coupling and using Minkowski’s inequality again, it is easy to show that $W_{p}$ is finite on Markov kernels with finite moments of order $p$. ∎

In the second half of the proof, we assumed the following result.

Proposition 5.5**.**

The infimum in the Wasserstein metric on Markov kernels is attained. That is, for any Markov kernels $M,N:X\to Y$ on mm spaces $X,Y$, there exists a coupling $\Pi:X\to Y\times Y$ of $M$ and $N$ such that

[TABLE]

Proof.

According to a known existence theorem for optimal couplings (Theorem A.6), the infimum can even be achieved simultaneously at every point $x$. That is, there exists a coupling $\Pi\in\operatorname{Coup}(M,N)$ such that for every $x\in X$,

[TABLE]

The proof even shows that the Wasserstein metric on Markov kernels can be written in terms of the Wasserstein metric on probability measures as

[TABLE]

Thus, if we view a Markov kernel $X\to Y$ as a function $X\to\operatorname{Prob}(Y)$ of metric spaces, the Wasserstein metric on Markov kernels reduces to the familiar $L^{p}$ metric. However, this result depends on the existence theorem for optimal couplings (Theorem A.6), the proof of which is non-trivial. Wherever possible we prefer to work with the original Definition 5.2 in terms of couplings of Markov kernels.

When at least one of the kernels is deterministic, the Wasserstein metric has a simple, closed-form expression.

Proposition 5.6** (Wasserstein metric on deterministic kernels).**

Let $X,Y,Z$ be metric measure spaces. For any Markov kernel $M:X\to Y$ and measurable functions $f:X\to Z$ and $g:Y\to Z$,

[TABLE]

In particular, for any measurable functions $f,g:X\to Y$,

[TABLE]

Proof.

To prove the first statement, notice that $f$ and $M\cdot g$ have a single coupling, at once deterministic and independent. By Tonelli’s theorem,

[TABLE]

The second statement follows from the first by taking $X=Y$ and $M=1_{X}$. ∎

6. Wasserstein metric on metric measure $\mathsf{C}$-spaces

We are at last ready to construct a Wasserstein-style metric on metric measure $\mathsf{C}$-spaces, combining the general metric theory for $\mathsf{C}$-sets with the Wasserstein metric on Markov kernels.

Let $\mathsf{MMarkov}$ be the category with metric measures spaces as objects and Markov kernels as morphisms. In Section 3, we identified measurable functions with deterministic Markov kernels, obtaining an embedding functor $\mathcal{M}:\mathsf{Meas}\to\mathsf{Markov}$ and thus a relaxation of the $\mathsf{C}$-set homomorphism problem. In exactly the same way, the category $\mathsf{MM}$ of metric measure spaces is functorially embedded inside $\mathsf{MMarkov}$. We denote this embedding also by $\mathcal{M}:\mathsf{MM}\to\mathsf{MMarkov}$. Just as we relaxed the $\mathsf{C}$-set homomorphism problem, so will we relax the Hausdorff metric on mm $\mathsf{C}$-spaces.

As a first step, we make $\mathsf{MMarkov}$ into a metric category compatible with the $L^{p}$ metrics. By Theorem 5.4, $\mathsf{MMarkov}$ is a metric category under the Wasserstein metric of order $p$, for any $1\leq p<\infty$. Furthermore, by Proposition 5.6, this metric agrees with the corresponding $L^{p}$ metric on deterministic Markov kernels. Thus, the embedding $\mathcal{M}:\mathsf{MM}\to\mathsf{MMarkov}$ is an isometry of metric categories.

In Section 4, we characterized the short morphisms of $\mathsf{MM}$ under its $L^{p}$ metric. The next proposition extends this characterization to $\mathsf{MMarkov}$, formally reducing to Proposition 4.9 when all the morphisms are deterministic Markov kernels. Consequently, the isometric embedding functor $\mathcal{M}:\mathsf{MM}\to\mathsf{MMarkov}$ restricts to an embedding $\operatorname{\mathsf{Short}}(\mathsf{MM})\to\operatorname{\mathsf{Short}}(\mathsf{MMarkov})$ of short morphisms.

Proposition 6.1** (Short Markov kernels).**

Let $W,X,Y,Z$ be metric measure spaces and let $M:X\to Y$ be a Markov kernel.

1. (a)

$W_{p}(MN,MN^{\prime})\leq W_{p}(N,N^{\prime})$* for all Markov kernels $N,N^{\prime}:Y\to Z$ if and only if $M$ is measure-decreasing,151515When the measure spaces coincide, that is, $X=Y$ and $\mu_{X}=\mu_{Y}$, the measure $\mu_{X}$ is also called a subinvariant measure of $M$ [15, Definition 1.4.1].***

[TABLE] 2. (b)

$W_{p}(PM,P^{\prime}M)\leq W_{p}(P,P^{\prime})$* for all Markov kernels $P,P^{\prime}:W\to X$ if and only if $M$ is distance-decreasing of order $p$, i.e., there exists a product $\Pi$ of $M$ with itself such that*

[TABLE]

Consequently, a Markov kernel is a short morphism if and only if it is distance-decreasing and measure-decreasing. $\operatorname{\mathsf{Short}}(\mathsf{MMarkov})$ is the category of mm spaces and distance- and measure-decreasing Markov kernels.

Proof.

Towards proving part (a), suppose $M:X\to Y$ is measure-decreasing. For any coupling $\Pi:Y\to Z\times Z$ of $N$ and $N^{\prime}$, the composite $M\cdot\Pi:X\to Z\times Z$ is a coupling of $M\cdot N$ and $M\cdot N^{\prime}$. Therefore,

[TABLE]

Since $\Pi\in\operatorname{Coup}(N,N^{\prime})$ is arbitrary, we have $W_{p}(MN,MN^{\prime})^{p}\leq W_{p}(N,N^{\prime})^{p}$.

For the converse direction, suppose that $M:X\to Y$ is not measure-decreasing. Choose a set $Y\in\Sigma_{Y}$ such that $\mu_{X}M(B)>\mu_{Y}(B)$. Let $Z$ be a classical metric space with at least two points $z$ and $z^{\prime}$, let $g:Y\to Z$ be the constant function at $z$, and let $g^{\prime}:Y\to Z$ be the function equal to $z^{\prime}$ on $B$ and equal to $z$ outside of $B$. The composite $Mg$ is also constant, so by Proposition 5.6 we have

[TABLE]

To prove part (b), suppose $M:X\to Y$ is distance-decreasing, and let $\Pi$ be a product of $M$ with itself that attains the bound. For any coupling $\Lambda:W\to X\times X$ of $P$ and $P^{\prime}$, the composite $\Lambda\cdot\Pi:W\to Y\times Y$ is a coupling of $P\cdot M$ and $P^{\prime}\cdot M$. Therefore,

[TABLE]

Since $\Lambda\in\operatorname{Coup}(P,P^{\prime})$ is arbitrary, we obtain $W_{p}(PM,P^{\prime}M)^{p}\leq W_{p}(P,P^{\prime})^{p}$.

Conversely, suppose this inequality holds for all Markov kernels $P,P^{\prime}$. As in the proofs of Propositions 4.7 and 4.9, let $W=I=\{*\}$ be the singleton probability space and let $e_{x}:I\to X$ denote the generalized element at $x\in X$. For any $x,x^{\prime}\in X$, take $P=e_{x}$ and $P^{\prime}=e_{x^{\prime}}$ to obtain $W_{p}(M(x^{\prime}),M(x^{\prime}))\leq d_{X}(x,x^{\prime})$. Using Theorem A.6, we can construct $\Pi\in\operatorname{Prod}(M,M)$ such that

[TABLE]

Conclude that $M$ is distance-decreasing. ∎

The Wasserstein metric on mm $\mathsf{C}$-spaces is the Hausdorff metric (Definition 4.11) on $\mathsf{C}$-sets in the metric category $\mathsf{MMarkov}$. In concrete terms, the definition is:

Definition 6.2** (Wasserstein metric on mm $\mathsf{C}$-spaces).**

Let $\mathsf{C}$ be a finitely presented category. For any $1\leq p<\infty$, the Wasserstein metric of order $p$ on metric measure $\mathsf{C}$-spaces is given by, for $X,Y\in[\mathsf{C},\mathsf{MM}]$,

[TABLE]

where the sum is over a fixed, finite generating set of morphisms in $\mathsf{C}$ and the infimum is over all Markov transformations $\Phi:X\to Y$, not necessarily natural, whose components $\Phi_{c}:X(c)\to Y(c)$ are short morphisms (so belong to $\operatorname{\mathsf{Short}}(\mathsf{MMarkov})$).

In view of the concepts combined, this metric should perhaps be called the “Hausdorff-Wasserstein metric” or even, if we are taking the history seriously [50], the “Hausdorff-Kantorovich-Rubinstein metric.” However, we find these names too cumbersome to contemplate.

The proposed metric is indeed a metric, by Theorem 4.12. Like any Hausdorff metric on $\mathsf{C}$-sets, it is not generally finite, positive definite, or symmetric, but the failure of positive definiteness is more severe than before, since $d_{W,p}(X,Y)=0$ whenever there exists a Markov morphism $\Phi:X\to Y$ with components in $\operatorname{\mathsf{Short}}(\mathsf{MMarkov})$. More generally, the Wasserstein metric is a relaxation of the Hausdorff-$L^{p}$ metric from Section 4, as clarified by the following inequality.

Proposition 6.3** (Wasserstein metric as convex relaxation).**

For any $1\leq p<\infty$, the Wasserstein metric of order $p$ on mm $\mathsf{C}$-spaces is a convex relaxation of the Hausdorff-$L^{p}$ metric on mm $\mathsf{C}$-spaces. That is, for all mm $\mathsf{C}$-spaces $X$ and $Y$,

[TABLE]

where

[TABLE]

is the Hausdorff metric on $\mathsf{C}$-sets in the metric category $\mathsf{MM}$ with its $L^{p}$ metric. Moreover, the problem of computing $d_{W,p}(X,Y)^{p}$ can be formulated as a convex optimization problem, possibly in infinite dimensions.

Proof.

We first prove the relaxation. As in the proof of Theorem 4.12, we write $|\phi|_{f}:=d_{L^{p}}(Xf\cdot\phi_{c^{\prime}},\phi_{c}\cdot Yf)$ for the weight of a transformation $\phi:X\to Y$ at a morphism $f:c\to c^{\prime}$, and, similarly, we write $|\Phi|_{f}:=W_{p}(Xf\cdot\Phi_{c^{\prime}},\Phi_{c}\cdot Yf)$ for the weight of a Markov transformation. By the discussion preceding Proposition 6.1, for any transformation $\phi:X\to Y$ with components in $\operatorname{\mathsf{Short}}(\mathsf{MM})$, there is a Markov transformation $\mathcal{M}(\phi):=(\mathcal{M}(\phi_{c}))_{c\in\mathsf{C}}$ with components in $\operatorname{\mathsf{Short}}(\mathsf{MMarkov})$ and of the same weight, $|\mathcal{M}(\phi)|_{f}=|\phi|_{f}$. Consequently,

[TABLE]

As for the convexity, since the infimum in the Wasserstein metric on Markov kernels is attained (Proposition 5.5), the value $d_{W,p}(X,Y)^{p}$ is the infimum of

[TABLE]

taken over the Markov kernels

[TABLE]

indexed by objects $c\in\mathsf{C}$ and morphisms $f:c\to c^{\prime}$ generating $\mathsf{C}$, and subject to the constraints, for all $c\in\mathsf{C}$,

[TABLE]

In this optimization problem, the optimization variables belong to convex spaces, the objective is linear, and all the constraints, including those for couplings and products, are linear equalities or inequalities. The problem is therefore convex. ∎

As in Section 3, the optimization problem is a linear program provided the $\mathsf{C}$-sets are finite. Let $X$ and $Y$ be finite metric measure $\mathsf{C}$-spaces. Identifying Markov kernels with right stochastic matrices, measures with row vectors, and exponentiated metrics $d_{X(c)}^{p}$ with column vectors $\delta_{X(c)}\in\mathbb{R}^{|Y(c)|^{2}}$, the distance $d_{W,p}(X,Y)^{p}$ is the value of the linear program:

[TABLE]

We leave implicit in the notation the dimensionalities of the projection operators $\operatorname{proj}_{i}$ and of column vectors $\mathbbm{1}$ consisting of all 1’s.

While it appears forbidding, this linear program simplifies in certain common situations. When $X(c)$ has the discrete metric, any Markov kernel $\Phi_{c}:X(c)\to Y(c)$ is distance-decreasing, so the product $\Pi_{c}$ and associated constraints can be eliminated from the program. When $X(c^{\prime})=Y(c^{\prime})$ and $\Phi_{c^{\prime}}:X(c^{\prime})\to Y(c^{\prime})$ is fixed to be the identity, as happens for fixed attribute sets, then for any morphism $f:c\to c^{\prime}$, the Wasserstein distance between $Xf$ and $\Phi_{c}\cdot Yf$ has a closed form (Proposition 5.6). The coupling $\Pi_{f}$ and associated constraints can thus be eliminated.

Both kinds of simplification occur in the next two examples.

Example 6.4* (Classical Wasserstein metric).*

Continuing Examples 2.11 and 4.15, let $X$ and $Y$ be attributed sets, equipped with discrete metrics and any probability measures, and taking attributes in a fixed mm space $\mathcal{A}$. Then $X$ and $Y$ are attributed sets in $\mathsf{MM}$ and the Wasserstein metric of order $p$ between them is

[TABLE]

where the second equality follows by the disintegration theorem (Theorem A.4). In particular, $d_{W,p}(X,Y)^{p}$ is the value of an optimal transport problem. When $X\xrightarrow{\operatorname{attr}}\mathcal{A}$ and $Y\xrightarrow{\operatorname{attr}}\mathcal{A}$ are injective, $X$ and $Y$ can be identified with subsets of $\mathcal{A}$ and we recover the classical Wasserstein metric, namely $d_{W,p}(X,Y)=W_{p}(\mu_{X},\mu_{Y})$.

Example 6.5* (Wasserstein metric on attributed graphs).*

Continuing Examples 2.12 and 4.16, let $X$ and $Y$ be vertex-attributed graphs taking attributes in a fixed mm space $\mathcal{A}$. Equip the vertex and edge sets with discrete metrics and with any fully supported measures. Then $X$ and $Y$ are attributed graphs in $\mathsf{MM}$ and the Wasserstein metric of order $p$ between them is

[TABLE]

where the infimum is over Markov graph morphisms $\Phi:X\to Y$ with measure-decreasing components $\Phi_{V}$ and $\Phi_{E}$.

This metric takes an infinite value whenever no Markov graph morphism exists. The next example features a weaker metric, presented, for simplicity, in the case of unattributed graphs.

Example 6.6* (Weak Wasserstein metric on graphs).*

Let $X$ and $Y$ be graphs. Continuing Example 4.17, equip the edge sets with discrete metrics, the vertex sets with shortest path distances, and the vertex and edge sets with counting measures. The Wasserstein metric of order 1 is then

[TABLE]

where the infimum is over measure-decreasing Markov kernels $\Phi_{E}$ and distance- and measure-decreasing Markov kernels $\Phi_{V}$. By Proposition 6.3, this metric relaxes the Hausdorff metric of Example 4.17. It is genuinely weaker: on directed cycles, we have $d_{W,1}(C_{m},C_{n})=0$ when $1\leq m\leq n$, as witnessed by the uniform Markov graph morphisms of Example 3.8. (They do not increase distances, like any Markov kernels equal everywhere to a constant distribution.) We still have $d_{W,1}(C_{m},C_{n})=\infty$ when $m>n$ due to the measure-decreasing constraint.

7. Conclusion

We have introduced Hausdorff and Wasserstein metrics on graphs and other $\mathsf{C}$-sets and illustrated them through a variety of examples. That being said, we have established only the most basic properties of the concepts involved. Possibilities abound for extending this work, in both theoretical and practical directions. Let us mention a few of them.

Although encompassing graphs, simplicial sets, dynamical systems, and other structures, the formalism of $\mathsf{C}$-sets remains possibly the simplest equational logic, sitting at the bottom of a hierarchy of increasingly expressive systems [29, 12]. By admitting categories $\mathsf{C}$ with extra structure, such as sums, products, or exponentials, more complicated structures can be realized as structure-preserving functors from $\mathsf{C}$ into $\mathsf{Set}$ or some other category $\mathsf{S}$. For example, categories with finite products describe monoids, groups, rings, modules, and other familiar algebraic structures. This is the original setting of categorical logic [30].

Many of the ideas developed here extend to categories $\mathsf{C}$ with extra structure. The pertinent questions are whether the extra structure can be accommodated in the category $\mathsf{Markov}$ and its variants and how this affects the computation. Sums (coproducts) and units (terminal objects) are easily handled. $\mathsf{Markov}$ has finite sums and a unit, and since the direct sum of Markov kernels is a linear operation, it preserves the class of linear optimization problems. Products are more important and more delicate. $\mathsf{Markov}$ does not have categorical products, and its natural tensor product, the independent product, is not a linear operation. In keeping with the spirit of this paper, products in $\mathsf{C}$ should be translated into optimal couplings in $\mathsf{Markov}$, resulting in a larger optimization problem. We leave a proper development of this idea to future work.

A linear program is solvable in polynomial time and therefore improves dramatically in tractability over an NP-hard combinatorial problem. Nevertheless, solving a generic linear program is not always practical. Indeed, the recent surge in popularity of optimal transport is due partly to the introduction, by Cuturi and others, of specialized algorithms for solving the optimal transport program, which far outperform generic interior-point solvers [13, 37]. It would be useful to know whether and how these fast algorithms for optimal transport can be adapted to the linear programs in this paper.

In the new algorithms for optimal transport, the optimization objective is augmented by a term proportional to the negative entropy of the coupling, a technique known as entropic regularization. With this addition, the optimal transport problem improves from merely convex to strongly convex and, in particular, has a unique solution. Besides being useful for optimization, entropic regularization has a statistical interpretation as Gaussian deconvolution [40].

For the Markov morphism feasibility problem of Section 3, adding entropic regularization yields an optimization problem whose solution is the Markov morphism of maximum entropy. For instance, in Figure 3.1, the maximum entropy Markov morphism is the uniform mixture $\Phi=\tfrac{1}{2}\phi_{1}+\tfrac{1}{2}\phi_{2}$ of the two graph homomorphisms $\phi_{1}$ and $\phi_{2}$. In Figure 3.3, the unique Markov morphism has the maximum possible entropy, with all vertex and edge distributions being uniform. As these examples show, entropic regularization is antithetically opposed to the recovery of deterministic solutions. Entropic regularization should be investigated more systematically in this context, for algorithmic reasons and for its intrinsic interest.

Appendix A Markov kernels

A Markov kernel is the probabilistic analogue of a function, assigning to every point in its domain not a single point in its codomain but a whole probability distribution over its codomain. Markov kernels are fundamental objects in probability theory and statistics [8, 24, 25, 28]. In this appendix, we recall the definition and basic properties of Markov kernels, as well as a few more obscure results from the literature.

Definition A.1** (Markov kernels).**

Let $(X,\Sigma_{X})$ and $(Y,\Sigma_{Y})$ be measurable spaces. A Markov kernel from $X$ to $Y$, also known as a probability kernel or stochastic kernel, is a function $M:X\times\Sigma_{Y}\to[0,1]$ such that

1. (i)

for all points $x\in X$, the map $M(x;{-}):\Sigma_{Y}\to[0,1]$ is a probability measure on $Y$, and 2. (ii)

for all sets $B\in\Sigma_{Y}$, the map $M({-};B):X\to[0,1]$ is measurable.

We usually write $M(dy\,|\,x)$ instead of $M(x;dy)$, in agreement with the standard notation for conditional probability.

Equivalently, a Markov kernel from $X$ to $Y$ is a measurable map $X\to\operatorname{Prob}(Y)$, where $\operatorname{Prob}(Y)$ is the space of all probability measures on $Y$ under the $\sigma$-algebra generated by the evaluation maps $\mu\mapsto\mu(B)$, $B\in\Sigma_{Y}$ [24, Lemma 1.40]. With this perspective, it is natural to denote a Markov kernel simply as $M:X\to Y$ and to write $M(x)$ for the distribution $M(x;{-})$ of $x$ under $M$.

If Markov kernels are probabilistic functions, then they ought to be composable. They are, according to a standard definition [28, Definition 14.25].

Definition A.2** (Composition of Markov kernels).**

The composition of a Markov kernel $M:X\to Y$ with another Markov kernel $N:Y\to Z$ is the Markov kernel $M\cdot N:X\to Z$ defined by

[TABLE]

The identity $1_{X}:X\to X$ with respect to this composition law is the usual identity function regarded as a Markov kernel, $1_{X}:x\mapsto\delta_{x}$.

A third perspective on Markov kernels is that they are linear operators on spaces of measures or probability measures (see, for instance, [8, Theorem 5.2 and Lemma 5.10] or [54, Section 3.3]).

Definition A.3** (Markov operators).**

Let $M:X\to Y$ be a Markov kernel. Given a measure $\mu$ on $X$, define the measure $\mu M$ on $Y$ by

[TABLE]

With this definition, $M$ is a Markov operator: writing $\operatorname{Meas}(X)$ for the space of all finite nonnegative measures on $X$, the Markov kernel $M$ acts as linear map $\operatorname{Meas}(X)\to\operatorname{Meas}(Y)$ that preserves the total mass, $\mu M(Y)=\mu(X)$. In particular, $M$ acts as a linear map $\operatorname{Prob}(X)\to\operatorname{Prob}(Y)$ on spaces of probability measures.

Let $\mu$ be a measure on $X$ and let $M:X\to Y$ be a Markov kernel. Besides applying $M$ to $\mu$, yielding the measure $\mu M$ on $Y$, we can also take the product of $\mu$ and $M$, yielding a measure $\mu\otimes M$ on $X\times Y$ defined on measurable rectangles by

[TABLE]

where $A\in\Sigma_{X}$ and $B\in\Sigma_{Y}$. The product measure $\mu\otimes M$ has marginal $\mu$ along $X$ and marginal $\mu M$ along $Y$. In the special case where $M(x)$ equals a fixed measure $\nu$ for all $x$, we recover the usual product measure $\mu\otimes\nu$.

It is often useful to know that the product operation is invertible. The inverse operation is called disintegration.

Theorem A.4** **(Disintegration of measures

[25, Theorem 1.23]).

Let $X$ be a measurable space and $Y$ be a Polish space. For any $\sigma$-finite measure $\pi$ on $X\times Y$, with $\sigma$-finite marginal $\mu$ along $X$, there exists a Markov kernel $M:X\to Y$ such that $\pi=\mu\otimes M$. Moreover, if $M^{\prime}:X\to Y$ is any other Markov kernel with this property, then $M(x)=M^{\prime}(x)$ for $\mu$-almost every $x\in X$.

What is less well known is that Markov kernels into product spaces can also be disintegrated. We use this result in Section 5 to prove a gluing lemma for Markov kernels.

Theorem A.5** **(Disintegration of Markov kernels

[25, Theorem 1.25]).

Let $X$ be a measurable space and let $Y$ and $Z$ be Polish spaces. For any Markov kernel $\Pi:X\to Y\times Z$ with marginal $M:X\to Y$ along $Z$, there exists a Markov kernel $\Lambda:X\times Y\to Z$ such that $\Pi=M\otimes\Lambda$, where the product $M\otimes\Lambda$ is defined on measurable rectangles by

[TABLE]

where $x\in X$, $B\in\Sigma_{Y}$, and $C\in\Sigma_{Z}$.

In the special case where $X=\{*\}$ is a singleton, a Markov kernel $\Pi:X\to Y\times Z$ is a probability distribution on $Y\times Z$ and we recover a version of the previous theorem.

Under mild assumptions, optimal couplings of probability measures exist, according to a standard result in optimal transport [52, Theorem 4.1]. Markov kernels have optimal couplings under the same assumptions, though proving it is more involved. Versions of the following theorem appear in the literature as [15, Theorem 20.1.3], [52, Corollary 5.22], and [56, Theorem 1.1].

Theorem A.6** (Optimal coupling of Markov kernels).**

Let $X$ be a measurable space, $Y$ be a Polish space, and $c:Y\times Y\to\mathbb{R}$ be a nonnegative, lower semi-continuous cost function. For any Markov kernels $M,N:X\to Y$, there exists a Markov kernel $\Pi:X\times X\to Y\times Y$ such that, for every $x,x^{\prime}\in X$,

1. (i)

$\Pi(x,x^{\prime})$* is a coupling of $M(x)$ and $N(x^{\prime})$, and* 2. (ii)

$\Pi(x,x^{\prime})$* is optimal with respect to the cost function $c$, i.e.,*

[TABLE]

We invoke this theorem in Section 5 to show that the infimum defining the Wasserstein metric on Markov kernels is attained.