The cut metric for probability distributions

TL;DR

This paper explores a variant of the cut metric for probability distributions, introducing a pinning operation and demonstrating its properties and applications, including a regularity lemma analogue and continuity of operations.

Contribution

It introduces a natural pinning operation on limit objects and establishes a canonical cut metric approximation for probability distributions, extending graph limit concepts.

Findings

Pinning operation yields a canonical cut metric approximation.

Cut metric continuity shown for product measures.

Establishes basic properties of the cut metric for probability distributions.

Abstract

Guided by the theory of graph limits, we investigate a variant of the cut metric for limit objects of sequences of discrete probability distributions. Apart from establishing basic results, we introduce a natural operation called {\em pinning} on the space of limit objects and show how this operation yields a canonical cut metric approximation to a given probability distribution akin to the weak regularity lemma for graphons. We also establish the cut metric continuity of basic operations such as taking product measures.

Figures

Equations

$${\Delta }_{⊠}(\mu ,\nu )$$

$$D_1(\sigma ,\tau )$$

$$D_{⊠}\left(\mu ,\nu \right)$$

$$\dot{\mu }$$

$$D_{⊠}(\kappa ,{\kappa }^{\prime })$$

$${\mu }^{(n)}(\sigma )$$

$${\Delta }_{⊠}({\mu }^{(n)},{\nu }^{(n)})$$

$${\mu }^{(n)}(\sigma )={\int }_0^1\prod _{i=1}^n{\left(\frac{is}{n}\right)}^{{\sigma }_i}{\left(1-\frac{is}{n}\right)}^{1-{\sigma }_i}\mathrm{d}s.$$

$${\mu }_T^{(n)}(\sigma )$$

$${\kappa }_T$$

$$({\mathbf{X}}^{\Xi }(i,j){)}_{i,j\in [n]}\text{mbox}and({\mathbf{X}}^{\Xi }(\phi (i),\psi (j)){)}_{i,j\in [n]}$$

$$\underset{N\to \infty }{\lim }\mathbb{P}\left[\forall i,j\in [n]:{\mathbf{X}}^{{\mu }_N}(i,j)={\tau }_{i,j}\right]$$

$${\mu }_n(\sigma )$$

$${\bar{\mu }}_i(\sigma )$$

$$[0,1]$$

$$D_{⊠}(\bar{\mu },\bar{\nu })$$

$$z_{\mu }(\tau ,x_1,\dots ,x_{\theta })$$

$$\mathrm{d}{\mu }_{\tau \downarrow x_1,\dots ,x_{\theta }}(\sigma )$$

$$\mathbb{P}\left[{\hat{\mathbf{\tau }}}^{\mu }=\tau \mid {\hat{\mathbf{x}}}_1,\dots ,{\hat{\mathbf{x}}}_{\theta }\right]$$

$${\mu }_{\downarrow \theta }$$

$$\mu \otimes \nu \left(\left(\genfrac{}{}{0ex}{}{{\sigma }_1}{{\tau }_1}\right),\dots ,\left(\genfrac{}{}{0ex}{}{{\sigma }_n}{{\tau }_n}\right)\right)=\mu ({\sigma }_1,\dots ,{\sigma }_n)\nu ({\tau }_1,\dots ,{\tau }_n).$$

$$\kappa \otimes {\kappa }^{\prime }$$

$$\kappa \oplus {\kappa }^{\prime }$$

$$R_{{\omega }_1,\dots ,{\omega }_n}\left({\sigma }_1,\dots ,{\sigma }_n\right)$$

$$R_{\ell ,{\omega }_1,\dots ,{\omega }_n}(\mu )$$

$$\mathcal{D}(\mu ,\nu )$$

$$\mathcal{D}(\mathbf{X},\mathbf{Y})$$

$$D_{\text{boxslash}}(\mu ,\nu )$$

$${\Delta }_{\text{boxslash}}(\mu ,\nu )$$

Full text

Supported by Stiftung Polytechnische Gesellschaft Frankfurt am Main.

1. Introduction and results

1.1. Background and motivation

The theory of graph limits clearly qualifies as one of the great recent success of modern combinatorics [6, 7, 32, 34]. Exhibiting a complete metric space of limit objects of sequences of finite graphs, the theory strikes a link between combinatorics and analysis. In fact, the notion of graphon convergence unifies several combinatorially meaningful concepts, such as convergence of subgraph counts or with respect to the cut metric. In effect, combinatorial ideas admit neat analytic interpretations. For instance, the Szemerédi regularity lemma yields the compactness of the graphon space [35].

While sequences of graphs occur frequently in combinatorics (e.g., in the theory of random graphs), sequences of probability distributions on increasingly large discrete domains play no less prominent a role in the mathematical sciences. For instance, such sequences are the bread and butter of mathematical physics. A classical example is the Ising model on a $d$-dimensional integer lattice of side length $n$, a model of ferromagnetism. The Ising model renders a probability measure, the so-called Boltzmann distribution, on the space $\{-1,+1\}^{[n]^{d}}$ that captures the distribution of the magnetic spins of the $n^{d}$ vertices. The objective is to extract properties of this probability distribution in the limit of large $n$ such as the nature of correlations. While mathematical physics has a purpose-built theory of limits of probability measures on lattices [23], this theory fails to cover other classes of important statistical mechanics models, such as mean-field models that ‘live’ on random graphs [38]. Additionally, in statistics and data science sequences of discrete probability distributions arise naturally, e.g., as the empirical distributions of samples as more data are acquired.

The purpose of this paper is to show how the theory of graph limits can be adapted and extended to obtain a coherent theory of limits of probability distributions on discrete cubes. First cursory steps were already taken in an earlier contribution [14]. For instance, a probabilistic version of the cut metric was defined in that paper. Moreover, Austin [4], Diaconis and Janson [19] and (later) Panchenko [43] pointed out the connection between the theory of graph limits and the Aldous-Hoover representation [1, 25, 29]. But thus far a complete and concise disquisition has been lacking. We therefore develop the basics of a cut-norm based limiting theory for probability measures, including the completeness and compactness of the space of limiting objects, a kernel representation, a sampling theorem and a discussion of the connection with exchangeable arrays. Some of the proofs rely on arguments similar to the ones used in the theory of graph limits, and none of them will come as a gross surprise to experts. In fact, a few statements (such as the compactness of the space of limiting objects) already appeared in [14], albeit without detailed proofs, and a few others (such as the characterisation of exchangeable arrays) are generalisations of results from [19]. But here we present unified proofs of these basic results in full generality to provide a coherent and mostly self-contained treatment that, we hope, will facilitate applications.

Additionally, and this constitutes the main technical novelty of the paper, we present a new construction of regular partitions for limit objects of discrete probability distributions that constitutes a continuous generalisation of the pinning operation for discrete probability distributions from [11, 40, 45]. The result provides an approximation akin to the graphon version of the Frieze-Kannan regularity lemma [21]. The pinning operation merely involves a purely mechanical reweighting of the probability distribution. The ‘obliviousness’ of the operation was critical to work on spin glass models on random graphs and on inference problems [11, 12, 13, 14]. We show that a similarly oblivious procedure carries over naturally to the space of limit objects. The proof, which hinges on a delicate analysis of cut norm approximations, constitutes the main technical achievement of the paper.

1.2. Results

We proceed to set out the main concepts and to state the main results of the paper. A detailed account of related work follows in Section 1.3. The cut metric is a mainstay of the theory of graph limits. An adaptation for probability measures was suggested in [13, 14]. Let us thus begin by recalling this construction.

1.2.1. The cut metric

Let $\Omega\neq\emptyset$ be a finite set and let $n\geq 1$ be an integer. Further, for probability distributions $\mu,\nu$ on the discrete cube $\Omega^{n}$ let $\Gamma(\mu,\nu)$ be the set of all couplings of $\mu,\nu$, i.e., all probability distributions $\gamma$ on the product space $\Omega^{n}\times\Omega^{n}$ with marginal distributions $\mu,\nu$. Additionally, let $\SS_{n}$ be the set of all permutations $[n]\to[n]$. Following [13], we define the (weak) cut distance of $\mu,\nu$ as

[TABLE]

The idea is that we first get to align $\mu,\nu$ as best as possible by choosing a suitable coupling $\gamma$ along with a permutation $\varphi$ of the $n$ coordinates. Then an adversary comes along and points out the largest remaining discrepancy. Specifically, the adversary picks an event $S\subset\Omega^{n}\times\Omega^{n}$ under the coupling, a set $X\subset[n]$ of coordinates and an element $\omega\in\Omega$ and reads off the discrepancy of the frequency of $\omega$ on $S,X$. It is easily verified that (1.1) defines a pre-metric on the space $\mathcal{L}_{n}=\mathcal{L}_{n}(\Omega)$ of probability distribution on $\Omega^{n}$. Thus, $\Delta_{\boxtimes}(\,\cdot\,,\,\cdot\,)$ is symmetric and satisfies the triangle inequality. But distinct $\mu,\nu$ need not satisfy $\Delta_{\boxtimes}(\mu,\nu)>0$. Hence, to obtain a metric space $\mathfrak{L}_{n}=\mathfrak{L}_{n}(\Omega)$ we identify any $\mu,\nu\in\mathcal{L}_{n}$ with $\Delta_{\boxtimes}(\mu,\nu)=0$.

Following [14], we embed the spaces $\mathcal{L}_{n}$ into a joint space $\mathcal{L}$. Specifically, let $\mathcal{P}(\Omega)$ be the space of all probability distributions on $\Omega$. We identify $\mathcal{P}(\Omega)$ with the standard simplex in $\mathbb{R}^{n}$ and thus endow $\mathcal{P}(\Omega)$ with the Euclidean topology and the corresponding Borel algebra. Further, let $\mathcal{S}$ be the space of all measurable maps $\sigma:[0,1]\to\mathcal{P}(\Omega)$, $\sigma\mapsto\sigma_{x}$, up to equality (Lebesgue-)almost everywhere. We equip $\mathcal{S}$ with the $L_{1}$-metric

[TABLE]

and the corresponding Borel algebra, thus obtaining a complete, separable metric space. The space $\mathcal{L}$ is defined as the space of all probability measures on $\mathcal{S}$.

Much as in the discrete case, for probability distributions $\mu,\nu$ on $\mathcal{S}$ we let $\Gamma(\mu,\nu)$ be the space of all couplings of $\mu,\nu$, i.e., probability distributions $\gamma$ on $\mathcal{S}\times\mathcal{S}$ with marginals $\mu,\nu$. Moreover, let $\SS$ be the space of all measurable bijections $\varphi:[0,1]\to[0,1]$ such that both $\varphi$ and its inverse $\varphi^{-1}$ map the Lebesgue measure to itself.111We recall that on a standard Borel space the inverse map $\varphi^{-1}$ is measurable as well, see Lemma 2.2. Then the cut distance of $\mu,\nu$ is defined by the expression

[TABLE]

where, of course, $S,X$ range over measurable sets. Thus, as in the discrete case we first align $\mu,\nu$ as best as possible by choosing a coupling and a suitable ‘permutation’ $\varphi$. Then the adversary puts their finger on the largest remaining discrepancy. One easily verifies that (1.2) defines a pre-metric on $\mathcal{L}$. Thus, identifying any $\mu,\nu$ with $D_{\boxtimes}\left({\mu,\nu}\right)=0$, we obtain a metric space $\mathfrak{L}$. The points of this space we call $\Omega$-laws.

Theorem 1.1**.**

The metric space $\mathfrak{L}$ is compact.

Theorem 1.1 was already stated in [14], but no detailed proof was included. We will give a full proof based on a novel analytic argument in Section 3.

What is the connection between the spaces $\mathfrak{L}_{n}$ and the ‘limiting space’ $\mathfrak{L}$? As pointed out in [14], a probability distribution $\mu$ on $\Omega^{n}$ naturally induces an $\Omega$-law. Indeed, we represent each $\sigma\in\Omega^{n}$ by a step function $\dot{\sigma}:[0,1]\to\mathcal{P}(\Omega)$ whose value on the interval $[(i-1)/n,i/n)$ is just the atom $\delta_{\sigma_{i}}\in\mathcal{P}(\Omega)$ for each $i\in[n]$. (This construction is somewhat similar to the one proposed for ‘decorated graphs’ in [33].) Then we let $\dot{\mu}\in\mathcal{L}$ be the distribution of $\dot{\sigma}\in\mathcal{S}$ for $\sigma$ chosen from $\mu$; in symbols,

[TABLE]

Thus, we obtain a map $\mathcal{L}_{n}\to\mathcal{L}$, $\mu\mapsto\dot{\mu}$. The definition of the cut metric guarantees that $D_{\boxtimes}(\dot{\mu},\dot{\nu})=0$ if $\Delta_{\boxtimes}(\mu,\nu)=0$. Consequently, the map $\mu\mapsto\dot{\mu}$ induces a map $\mathfrak{L}_{n}\to\mathfrak{L}$. The following statement shows that this map is in fact an embedding, and that therefore the space $\mathfrak{L}$ unifies all the spaces $\mathfrak{L}_{n}$, $n\geq 1$.

Theorem 1.2**.**

There exists a function $\mathfrak{d}:[0,1]\to[0,1]$ with $\mathfrak{d}^{-1}(0)=\{0\}$ such that for all $n\geq 1$ and all $\mu,\nu\in\mathfrak{L}_{n}$ we have $\mathfrak{d}(\Delta_{\boxtimes}(\mu,\nu))\leq D_{\boxtimes}(\dot{\mu},\dot{\nu})\leq\Delta_{\boxtimes}(\mu,\nu).$

We will see a few examples of convergence in the cut metric momentarily. But let us first explore a convenient representation of the space $\mathfrak{L}$.

Remark 1.3**.**

The definition of the space $\mathfrak{L}$ is based on measurable functions $\sigma:[0,1]\to\mathcal{P}(\Omega)$. Of course, one could replace the unit interval by another atomless probability space, and this may be natural/convenient in some situations. (The definition of the product and direct sum in Section 1.2.6 below could be quoted as a case a point.) But the use of the unit interval is without loss of generality (see Lemma 2.3 below).

1.2.2. The kernel representation

As in the case of graph limits, $\Omega$-laws can naturally be represented by functions on the unit square that we call kernels. To be precise, let $\mathcal{K}{}$ be the set of all measurable maps $\kappa:[0,1]^{2}\to\mathcal{P}(\Omega)$, $(s,x)\mapsto\kappa_{s,x}$, up to equality almost everywhere. For $\kappa,\kappa^{\prime}\in\mathcal{K}{}$ we define, with $S,X$ ranging over measurable sets,

[TABLE]

As before (1.3) defines a pre-metric on $\mathcal{K}{}$. We obtain a metric space $\mathfrak{K}{}$ by identifying $\kappa,\kappa^{\prime}\in\mathcal{K}{}$ with $D_{\boxtimes}(\kappa,\kappa^{\prime})=0$.

There is a natural map $\mathcal{K}{}\to\mathcal{L}{}$. Namely, for a kernel $\kappa$ and $s\in[0,1]$ let $\kappa_{s}:[0,1]\to\mathcal{P}(\Omega)$ be the measurable map $x\mapsto\kappa_{s,x}$. This map belongs to the space $\mathcal{S}$. Thus, $\kappa$ induces a probability distribution $\mu^{\kappa}$ on $\mathcal{S}$, namely the distribution of $\kappa_{\bm{s}}$ for a uniformly random $\bm{s}\in[0,1]$. The definition of the cut distance guarantees that $D_{\boxtimes}(\mu^{\kappa},\mu^{\kappa^{\prime}})=0$ if $D_{\boxtimes}(\kappa,\kappa^{\prime})=0$. Therefore, as pointed out in [14], the map $\kappa\mapsto\mu^{\kappa}$ induces a map $\mathfrak{K}{}\to\mathfrak{L}{}$.

Theorem 1.4**.**

The map $\mathfrak{K}\to\mathfrak{L}$ induced by $\kappa\mapsto\mu^{\kappa}$ is an isometric bijection.

Thus, any $\Omega$-law $\mu$ can be represented by an $\Omega$-kernel, which we denote by $\kappa^{\mu}$.

Example 1.5**.**

With $\Omega=\left\{{0,1}\right\}$ let $\mu^{(n)}\in\mathcal{L}_{n}$ be uniformly distributed over all $\sigma\in\left\{{0,1}\right\}^{n}$ with even parity. In symbols,

[TABLE]

Similarly, let $\nu^{(n)}$ be uniformly distributed on the set of $\sigma\in\left\{{0,1}\right\}^{n}$ with odd parity. Then $\mu^{(n)},\nu^{(n)}$ have total variation distance one for all $n$ because they are supported on disjoint subsets of $\{0,1\}^{n}$. Nevertheless, in the cut distance both sequences $(\mu^{(n)})_{n},(\nu^{(n)})_{n}$ converge to the common limit $\mu=\delta_{u}\in\mathcal{L}$ supported on $u:[0,1]\to\mathcal{P}(\{0,1\})$, $x\mapsto(1/2,1/2)$. Specifically, we claim that

[TABLE]

To verify the first bound, consider the following coupling $\gamma^{(n)}$: choose the first $n-1$ bits $\bm{\sigma}_{1},\ldots,\bm{\sigma}_{n-1}\in\left\{{0,1}\right\}$ uniformly and independently and choose $\bm{\sigma}_{n}\in\left\{{0,1}\right\}$ so that $\sum_{i=1}^{n}\bm{\sigma}_{i}\equiv 0\mod 2$. Then $\gamma^{(n)}\in\mathcal{P}(\Omega^{n}\times\Omega^{n})$ is the distribution of $((\bm{\sigma}_{1},\ldots,\bm{\sigma}_{n}),(\bm{\sigma}_{1},\ldots,1-\bm{\sigma}_{n}))$. In effect, under $\gamma^{(n)}$ the two $n$-bit vectors differ in exactly one position, whence the first part of (1.4) follows from (1.1). The second bound in (1.4) follows from the central limit theorem.

Example 1.6**.**

Let $\mu^{(n)}$ be the probability distribution on $\{0,1\}^{n}$ induced by the following experiment. First, pick $\bm{s}\in[0,1]$ uniformly at random. Then, given $\bm{s}$, obtain $\bm{\sigma}\in\left\{{0,1}\right\}^{n}$ by letting $\bm{\sigma}_{i}=1$ with probability $i\bm{s}/n$ independently for each $i\in[n]$. In formulas,

[TABLE]

Kernel representations $\kappa^{(n)}$ of $\mu^{(n)}$ are displayed in Figure 1 for some values of $n$. The sequence $\kappa^{(n)}$ converges to the kernel $\kappa:[0,1]^{2}\to\mathcal{P}(\left\{{0,1}\right\})$ defined by $\kappa_{s,x}(1)=sx$, $\kappa_{s,x}(0)=1-sx$.

Example 1.7**.**

The Curie-Weiss model is an (extremely) simple model of ferromagnetism. The vertices of a complete graph of order $n$ correspond to iron atoms that can take one of two possible magnetic spins $\pm 1$. Energetically it is beneficial for atoms to be aligned and the impact of the energetic term is governed by a temperature parameter $T>0$. To be precise, the Boltzmann distribution $\mu^{(n)}$ on $\{\pm 1\}^{n}$ defined by

[TABLE]

captures the distribution of spin configurations at a given temperature. The Curie-Weiss model is completely understood mathematically and it is well known that a phase transition occurs at $T=1$. In the framework of the cut distance, this phase transition manifests itself in the different limits that the sequence $(\mu_{T}^{(n)})_{n}$ converges to. Specifically, the kernel $\kappa_{T}$ representing the limit reads

[TABLE]

where $0<m_{T}<1$ is the unique zero of $m_{T}/T-\ln(1+m_{T})/2+\ln(1-m_{T})/2$ for $T>1$.

1.2.3. Counting and sampling

In the theory of graph limits convergence with respect to the cut metric is equivalent to convergence of subgraph counts. We are going to derive a similar equivalence for $\Omega$-laws. In fact, we are going to derive an extension of this result that links the cut metric to the theory of exchangeable arrays. We recall that a probability distribution $\Xi$ on the space $\Omega^{\mathbb{N}\times\mathbb{N}}$ of infinite $\Omega$-valued arrays is exchangeable if the following is true. If $\bm{X}^{\Xi}=(\bm{X}^{\Xi}(i,j))_{i,j\geq 1}\in\Omega^{\mathbb{N}\times\mathbb{N}}$ is drawn randomly from $\Xi$, then for any integer $n$ and for any permutations $\varphi,\psi:[n]\to[n]$ the random $n\times n$-arrays

[TABLE]

are identically distributed. Let $\mathfrak{X}=\mathfrak{X}(\Omega)$ denote the set of all exchangeable distributions. Since the product space $\Omega^{\mathbb{N}\times\mathbb{N}}$ is compact by Tychonoff’s theorem, endowed with the weak topology $\mathfrak{X}$ is a compact, separable space.

A kernel $\kappa\in\mathfrak{K}$ naturally induces an exchangeable distribution. Specifically, let $\bm{s}_{1},\bm{x}_{1},\bm{s}_{2},\bm{x}_{2},\ldots\in[0,1]$ be mutually independent uniformly distributed random variables. We obtain a random array $\bm{X}^{\kappa}\in\Omega^{\mathbb{N}\times\mathbb{N}}$ by drawing independently for any $i,j\in\mathbb{N}$ an element $\bm{X}^{\kappa}(i,j)\in\Omega$ from the distribution $\kappa_{\bm{s}_{i},\bm{x}_{j}}\in\mathcal{P}(\Omega)$. Clearly, the distribution $\Xi^{\kappa}$ of $\bm{X}^{\kappa}$ is exchangeable. By extension, a probability distribution $\pi$ on $\mathfrak{K}$ induces an exchangeable distribution as well. Indeed, with $\kappa^{\pi}\in\mathfrak{K}$ drawn from $\pi$, we let $\Xi^{\pi}\in\mathfrak{X}$ be the distribution of the random array $\bm{X}^{\pi}\in\Omega^{\mathbb{N}\times\mathbb{N}}$ obtained by first drawing $\kappa^{\pi}$ independently of the $(\bm{s}_{k},\bm{x}_{\ell})_{k,l\geq 1}$ and then drawing each entry $\bm{X}^{\pi}(i,j)$ from $\kappa^{\pi}_{\bm{s}_{i},\bm{x}_{j}}$. We equip the space $\mathcal{P}(\mathfrak{K})$ of probability measures on $\mathfrak{K}$ with the weak topology.

Theorem 1.8**.**

The map $\mathcal{P}(\mathfrak{K})\to\mathfrak{X}$, $\pi\mapsto\Xi^{\pi}$ is a homeomorphism.

For the special case $\Omega=\left\{{0,1}\right\}$ Theorem 1.8 is the directed graph version of  [19, Theorem 5.3].

For $\mu\in\mathfrak{L}$ let us write $\bm{X}^{\mu}$ for the exchangeable array $\bm{X}^{\kappa^{\mu}}$ induced by a kernel representation of $\mu$. Suppose that $(\mu_{N})_{N\geq 1}$ is a sequence of $\Omega$-laws that converges to $\mu\in\mathfrak{L}$. Then Theorem 1.8 shows that for any $n\geq 1$ and for any $\tau=(\tau_{i,j})_{i,j\in[n]}\in\Omega^{n\times n}$,

[TABLE]

Conversely, if $\mu_{N},\mu\in\mathfrak{L}$ are such that (1.5) holds for all $n,\tau$, then Theorem 1.8 implies that $\lim_{N\to\infty}D_{\boxtimes}(\mu_{N},\mu)=0$. Thus, with $\Omega^{n\times n}$-matrices replacing subgraphs, Theorem 1.8, provides the probabilistic counterpart of the equivalence of subgraph counting and graphon convergence [32, Theorem 11.5].

Additionally, the theory of graph limits shows that a large enough random graph obtained from a graphon by sampling is close to the original graphon in the cut metric. There is a corresponding statement in the realm of probability distributions as well. Specifically, for an integer $n\geq 1$ let $\mu_{n}\in\mathcal{P}(\Omega^{n})$ be the discrete probability distribution defined by

[TABLE]

In words, $\mu_{n}$ is the empirical distribution of the rows of $(\bm{X}^{\mu}(i,j))_{i,j\in[n]}$. Strictly speaking, being dependent on the random coordinates $(\bm{s}_{i},\bm{x}_{j})_{i,j\geq 1}$, $\mu_{n}$ is a random probability distribution on $\Omega^{n}$. The following theorem supplies a probabilistic version of the sampling theorem for graphons [6, Lemma 4.4].

Theorem 1.9**.**

There exists $c=c(\Omega)>0$ such that for all $n>1$ and all $\mu\in\mathfrak{L}$ we have $\mathbb{E}\left[{D_{\boxtimes}(\mu,\mu_{n})}\right]\leq c/\sqrt{\log n}.$

The following theorem implies that the dependence on $n$ in Theorem 1.9 is best possible, apart from the value of the constant $c$.

Theorem 1.10**.**

There is a constant $c>0$ such that for any $\varepsilon>0$ there exists $\mu\in\mathfrak{L}$ such that $D_{\boxtimes}(\mu,\nu)\geq\varepsilon$ for all $\nu\in\mathcal{L}$ whose support contains at most $\exp(c/\varepsilon^{2})$ configurations.

1.2.4. Extremality

Among all the probability measures on the discrete domain $\Omega^{n}$, the product measures are clearly the simplest. We will therefore be particularly interested in distributions that are close to product measures in the cut metric. To this end, for a probability measure $\mu$ on $\Omega^{n}$ we let

[TABLE]

Thus, $\bar{\mu}_{i}\in\mathcal{P}(\Omega)$ is the marginal distribution of the $i$th coordinate under the measure $\mu$, and $\bar{\mu}$ is the product measure with the same marginals as $\mu$. Then $\Delta_{\boxtimes}(\mu,\bar{\mu})$ gauges how ‘similar’ $\mu$ is to a product measure. To be precise, since the cut metric is quite weak, a ‘small’ value of $\Delta_{\boxtimes}(\mu,\bar{\mu})$ need not imply that $\mu$ behaves like a product measure in every respect. For instance, the entropy of $\mu$ might be much smaller than that of $\bar{\mu}$. But if $\Delta_{\boxtimes}(\mu,\bar{\mu})$ is small, then (1.5) implies that the joint distribution of a bounded number of randomly chosen coordinates of $\mu$ is typically close to a product measure in total variation distance.

A similar measure of proximity to a product distribution is meaningful on the space of $\Omega$-laws as well. Formally, for $\mu\in\mathcal{L}$ define $\bar{\mu}\in\mathcal{L}$ as the atom concentrated on the single function

[TABLE]

Since $D_{\boxtimes}(\bar{\mu},\bar{\nu})=0$ whenever $D_{\boxtimes}(\mu,\nu)=0$, (1.6) induces a map $\mu\in\mathfrak{L}\mapsto\bar{\mu}\in\mathfrak{L}$. The laws $\bar{\mu}$ with $\mu\in\mathfrak{L}$ represent the generalisation of discrete product measures. Since each $\mu\in\mathfrak{L}$ is represented by a distribution on $\mathcal{S}$ that places all the probability mass on a single point, we call the laws $\bar{\mu}$ extremal. Moreover, $\mu\in\mathfrak{L}$ is called $\varepsilon$-extremal if $D_{\boxtimes}(\mu,\bar{\mu})<\varepsilon$. The following result summarises basic properties of extremal laws and of the map $\mu\mapsto\bar{\mu}$.

Theorem 1.11**.**

For all $\mu,\nu\in\mathcal{L}$ we have

[TABLE]

Furthermore, the set of extremal laws is a closed subset of $\mathfrak{L}$.

1.2.5. Pinning

The regularity lemma constitutes one of the most powerful tools of modern combinatorics. In a nutshell, the lemma shows that any graph can be approximated by a mixture of a bounded number of ‘simple’ graphs, namely quasi-random bipartite graphs. We will present a corresponding result for probability measures, respectively laws. Specifically, we will show that any law can be approximated by a mixture of a small number of extremal laws. Indeed, we will show that actually this approximation can be obtained by a simple, mechanical procedure called ‘pinning’. This is in contrast to the proof of the graphon regularity lemma, where the regular partition results from a delicate construction that involves tracking a potential function.

To describe the pinning procedure, consider $\mu\in\mathcal{L}$, $\theta\geq 1$, $x_{1},\ldots,x_{\theta}\in[0,1]$ and $\tau\in\Omega^{\theta}$. Then we define

[TABLE]

Further, assuming that $z_{\mu}(\tau,x_{1},\ldots,x_{\theta})>0$, we define a reweighted probability distribution $\mu_{\tau\downarrow x_{1},\ldots,x_{\theta}}$ by

[TABLE]

Thus, $\mu_{\tau\downarrow x_{1},\ldots,x_{\theta}}$ is obtained by reweighting $\mu$ according to the ‘reference configuration’ $\tau$, evaluated at the coordinates $x_{1},\ldots,x_{\theta}$. For completeness we also let $\mu_{\tau\downarrow x_{1},\ldots,x_{\theta}}=\mu$ if $z_{\mu}(\tau,x_{1},\ldots,x_{\theta})=0$.

The effect of this reweighting procedure becomes particularly interesting if the reference configuration and the coordinates are chosen randomly. Specifically, let $\hat{\bm{x}}_{1},\hat{\bm{x}}_{2},\ldots\in[0,1]$ be uniform and mutually independent. Further, for an integer $\theta\geq 1$ draw $\hat{\bm{\tau}}=\hat{\bm{\tau}}^{\mu}\in\Omega^{\theta}$ from the distribution

[TABLE]

Equivalently, and perhaps more intuitively, we can describe the choice of $\hat{\bm{\tau}}$ as follows. First, draw $\bm{\tau}\in\mathcal{S}$ from the distribution $\mu$; then pick $\hat{\bm{\tau}}$ from the product measure $\bm{\tau}_{\bm{x}_{1}}\otimes\cdots\otimes\bm{\tau}_{\bm{x}_{\theta}}\in\mathcal{P}(\Omega^{\theta})$. Now, having drawn the ‘reference vector’ $\hat{\bm{\tau}}$, we obtain the reweighted distribution $\mu_{\hat{\bm{\tau}}\downarrow\theta}=\mu_{\hat{\bm{\tau}}\downarrow\hat{\bm{x}}_{1},\ldots,\hat{\bm{x}}_{\theta}}$ as defined in (1.9). Clearly, (1.10) guarantees that $\bm{z}_{\mu}(\hat{\bm{\tau}})>0$ almost surely. Finally, we define

[TABLE]

Hence, $\mu_{\downarrow\theta}$ weights each possible outcome according to the probability of its reference configuration $\hat{\bm{\tau}}$. The discrete version of the operation $\mu\mapsto\mu_{\downarrow\theta}$ for $\mu\in\mathcal{L}_{n}$ was introduced in [13]. Following the terminology from that paper, we refer to the map $\mu\mapsto\mu_{\hat{\bm{\tau}}\downarrow\theta}$ as the pinning operation. The term is explained by the fact that in the discrete case, each of the products on the r.h.s. of (1.9) is either one or zero.

The next theorem shows that pinning furnishes a probabilistic equivalent of weak regular graphon partitions. To state this result, we observe that the pinning construction is well-defined on the space $\mathfrak{L}$ as well. To be precise, if $\mu,\nu\in\mathfrak{L}$ have cut distance zero, then $\mu_{\hat{\bm{\sigma}}\downarrow\theta},\nu_{\hat{\bm{\sigma}}\downarrow\theta}$ are identically distributed, and so are $\mu_{\downarrow\theta}$ and $\nu_{\downarrow\theta}$. Consequently, we can apply the pinning operation directly to elements of the space $\mathfrak{L}$.

Theorem 1.12**.**

Let $0<\varepsilon<1$, let $\mu\in\mathfrak{L}$ and draw $0\leq\bm{\theta}=\bm{\theta}(\varepsilon)\leq 64\varepsilon^{-8}\log|\Omega|$ uniformly and independently of everything else. Then \mathbb{P}\left[{\mu_{\hat{\bm{\tau}}\downarrow\bm{\theta}}\mbox{ is \varepsilon-extremal}}\right]\geq 1-\varepsilon and $\mathbb{E}[D_{\boxtimes}(\mu,\mu_{\downarrow\bm{\theta}})]<\varepsilon$.

Hence, the law $\mu_{\downarrow\bm{\theta}}$, a mixture of no more than $|\Omega|^{\bm{\theta}}$ extremal laws, likely provides an $\varepsilon$-approximation to $\mu$.

1.2.6. Continuity and overlaps

There are certain natural operations on probability measures and, by extension, laws that turn out to be continuous with respect to the cut metric. First, we consider the construction of the product measure. For discrete measures $\mu,\nu\in\mathcal{L}_{n}(\Omega)$ we can view their product $\mu\otimes\nu$ as a probability distribution on $(\Omega\times\Omega)^{n}$ such that for any $\sigma_{1},\tau_{1},\ldots,\sigma_{n},\tau_{n}\in\Omega$,

[TABLE]

We extend this construction to laws by way of the kernel representation. To this end, let $\Lambda:[0,1]\to[0,1]\times[0,1]$, $x\mapsto(\Lambda_{1}(x),\Lambda_{2}(x))$ be a measurable bijection that maps the Lebesgue measure on $[0,1]$ to the Lebesgue measure on $[0,1]^{2}$ such that, conversely, $\Lambda^{-1}$ maps the Lebesgue measure on $[0,1]^{2}$ to the Lebesgue measure on $[0,1]$.222The existence of such a $\Lambda$ follows from Lemma 2.3 below. Following [14], for measurable maps $\kappa,\kappa^{\prime}:[0,1]^{2}\to\mathcal{P}(\Omega)$ we introduce

[TABLE]

For any kernels $\kappa,\kappa^{\prime},\kappa^{\prime\prime},\kappa^{\prime\prime\prime}$ such that $D_{\boxtimes}(\kappa,\kappa^{\prime\prime})=D_{\boxtimes}(\kappa^{\prime},\kappa^{\prime\prime\prime})=0$ we clearly have $D_{\boxtimes}(\kappa\otimes{}\kappa^{\prime},\kappa^{\prime\prime}\otimes{}\kappa^{\prime\prime\prime})=0$. Thus, the $\otimes{}$-operation is well defined on the kernel space $\mathfrak{K}{}$. Hence, due to Theorem 1.4 the construction extends to laws, i.e., given $\Omega$-laws $\mu,\nu$ we obtain an $\Omega^{2}$-law $\mu\otimes{}\nu$. Furthermore, it is easy to see that for any $\mu,\nu\in\mathcal{L}_{n}(\Omega)$ the $\Omega^{2}$-law representing the product measure $\mu\otimes{}\nu$ is precisely the $\otimes{}$-product of the laws $\dot{\mu},\dot{\nu}$ representing $\mu,\nu$.

Theorem 1.13**.**

The map $(\mu,\nu)\in\mathfrak{L}(\Omega)\mapsto\mu\otimes\nu\in\mathfrak{L}(\Omega^{2})$ is continuous.

There is a second fundamental operation on distributions/laws that resembles the operation of obtaining a $n\times n$-rank one matrix from two vectors of length $n$. Specifically, for vectors $\sigma,\tau\in\Omega^{[n]}$ let $\sigma\oplus\tau\in(\Omega^{2})^{[n]\times[n]}$ be the vector with entries $(\sigma\oplus\tau)_{ij}=(\sigma_{i},\tau_{j})$ for all $i,j\in[n]$. Additionally, for distributions $\mu,\nu\in\mathcal{L}_{n}(\Omega)$ let $\mu\oplus\nu$ be the distribution of the pair $\bm{\sigma}^{\mu}\oplus\bm{\tau}^{\nu}$ with $\bm{\sigma}^{\mu},\bm{\tau}^{\nu}\in\Omega^{n}$ chosen from $\mu,\nu$, respectively.

We extend the $\oplus$-operation to kernels as follows. For $\kappa,\kappa^{\prime}:[0,1]^{2}\to\mathcal{P}(\Omega)$ let

[TABLE]

It is easy to see that for $\kappa,\kappa^{\prime},\kappa^{\prime\prime},\kappa^{\prime\prime\prime}$ with $D_{\boxtimes}(\kappa,\kappa^{\prime\prime})=D_{\boxtimes}(\kappa^{\prime},\kappa^{\prime\prime\prime})=0$ we have $D_{\boxtimes}(\kappa\oplus\kappa^{\prime},\kappa^{\prime\prime}\oplus\kappa^{\prime\prime\prime})=0$. Hence, the $\oplus$-operation is well-defined on the space $\mathfrak{K}{}$ and thus, due to Theorem 1.4, on the space $\mathfrak{L}{}$ as well. Moreover, for discrete measures $\mu,\nu\in\mathcal{P}(\Omega^{n})$ one verifies immediately that the law representing $\mu\oplus\nu$ coincides with $\dot{\mu}\oplus\dot{\nu}$.

Theorem 1.14**.**

The map $\mathfrak{L}(\Omega)\to\mathfrak{L}(\Omega^{2})$, $(\mu,\nu)\mapsto\mu\oplus\nu$ is continuous.

Theorems 1.13 and 1.14 immediately imply the continuity of further functionals that play a fundamental role in mathematical physics. Specifically, let $\sigma_{1},\ldots,\sigma_{n}\in\mathcal{S}$. For $\sigma_{1},\ldots,\sigma_{n}\in\mathcal{S}$ and $\omega_{1},\ldots,\omega_{n}\in\Omega$ we define

[TABLE]

Furthermore, for $\mu\in\mathcal{L}$ and $\ell\geq 1$ we define

[TABLE]

Additionally, let $R_{\ell,n}(\mu)=(R_{\ell,\omega_{1},\ldots,\omega_{n}}(\mu))_{\omega_{1},\ldots,\omega_{n}\in\Omega}$. In physics jargon, the arrays $R_{\ell,n}(\mu)$ are known as multi-overlaps of $\mu$. Since $R_{\ell,n}(\mu)=R_{\ell,n}(\nu)$ if $D_{\boxtimes}(\mu,\nu)=0$, the multi-overlaps are well-defined on the space $\mathfrak{L}$ of laws.

Corollary 1.15**.**

The functions $\mu\in\mathfrak{L}\mapsto R_{\ell,n}(\mu)$ with $\ell,n\geq 1$ are continuous.

1.3. Discussion and related work

Borgs, Chayes, Lovász, Sós, Szegedy and Vesztergombi launched the theory of (dense) graph limits in a series of important and influential articles [6, 7, 33, 34, 35]. Lovász [32] provides a unified account of the state of the art up to about 2012. Moreover, Janson [28] gives an excellent account of the measure-theoretic foundations of the theory of graph limits and some of its generalisation.

Given the many areas of application where sequences of probability measures on increasingly large discrete cubes appear, the most prominent example being perhaps the study of Boltzmann distributions in mathematical physics, it is unsurprising that attempts have been made to construct limiting objects for such sequences. The theory of Gibbs measures embodies the classical, physics-inspired approach to this task [23]. Here the aim is to construct and classify all possible ‘infinite-volume’ limits of Boltzmann distributions defined on spatial structures such as trees or lattices. The limiting objects are called Gibbs measures. A fundamental question, whose ramifications extend from the study of phase transitions in physics to the computational complexity of counting and sampling, is whether there is a unique Gibbs measure that satisfies all the finite-volume conditional equations (e.g,  [22, 46, 47]). However, since the theory of Gibbs measures is confined to systems with an underlying lattice-like geometry, numerous applications are beyond its reach. For instance, Marinari et al. [38] argued that the classical theory of Gibbs measures does not provide an appropriate framework for the study of (diluted) mean-field models such as the Sherrington-Kirkpatrick model, the Viana-Bray model or the hardcore model on a sparse random graph. Further examples of ‘non-spatial’ sequences of distributions abound in computer science, statistics and data science.

Panchenko [43, 44] employed the more abstract Aldous-Hoover representation of exhangeable arrays in his work on mean-field models [1, 25]. Kallenberg’s monograph [29] provides the definite treatment of this abstract theory. Furthermore, Austin [4] extends and generalises the concept of exchangeable arrays and discusses applications to the Viana-Bray spin glass model. The close relationship between the theory of graph limits and exchangeable arrays was first noticed by Diaconis and Janson [19]. Their [19, Theorem 9.1] is essentially a directed graph version of Theorem 1.8 in the special case $\Omega=\left\{{0,1}\right\}$. Moreover, the appendix of Panchenko’s monograph [43] also contains a proof of the Aldous-Hoover representation theorem via graph limits.

Although the connection between genuinely probabilistic constructions such as the Aldous-Hoover representation and graph limits was noticed in prior work [4, 19, 43], those contributions stopped short of working out a fully-fledged adaptation of the theory of graph limits to a limit theory for probability measures on discrete cubes. A prior article by Coja-Oghlan, Perkins and Skubch [14] made a first cursory attempt at filling this gap and already contained the definition (1.2) of the cut metric and of the space $\mathfrak{L}$ of laws. Additionally, the compactness of the space $\mathfrak{L}$ (Theorem 1.1) and a weaker version of the kernel representation (Theorem 1.4) were stated in [14], although no detailed proofs were given. Furthermore, a definition similar to the discrete cut metric (1.1) was devised in [13] and a statement similar to Theorem 1.14 was previously proved by Coja-Oghlan and Perkins [12, Proposition A.2]. Finally, versions of the pinning operation for discrete probability measures appeared in [11, 40, 45] and recently Eldad [20] devised an extension to subspaces of $\mathbb{R}^{n}$, i.e., to the case of spins that need not take discrete values.

The contribution of the present paper is that we expressly and explicitly adapt and extend the concepts of the theory of graph limits to the context of probability distributions on increasing sequences of discrete cubes. We present in a unified way the proofs of the most important basic facts such as the relationship between the discrete and the continuous cut metric (Theorem 1.2), the kernel representation (Theorem 1.4), the sampling theorem (Theorem 1.9) and the continuity of product measures (Theorems 1.13 and 1.14). The proofs of these results are based on extensions and adaptations of techniques from the theory of graph limits. Moreover, we present a self-contained derivation of the representation theorem for exchangeable arrays (Theorem 1.8). The added value by comparison to prior work [14, 19] is that here we present detailed, unified proofs that operate directly in the probabilistic setting, rather than by extensive allusion to the graphon space. Additionally, we present a self-contained proof of the compactness result (Theorem 1.1). While the argument set out in, e.g., [32, Chapter 9] could be adapted to the probabilistic setting, we present a different argument based on analytic techniques that might be of independent interest. But the main technical novelty is certainly the pinning theorem (Theorem 1.12) that generalises the discrete version from [11]. The proof is delicate and uses many of the other, more basic results.

The pinning operation from Theorem 1.12 is somewhat reminiscent of Tao’s construction of regular partitions [49] and of the construction of Lovász and Szegedy [35]. For example, Tao’s construciton of a regular partition is based on sampling a number $\theta$ of vertices of a graph $G$ and then partitioning the remaining vertices into $2^{\theta}$ classes according to their adjacencies with the reference vertices. The discrete version pinning operation from [11, 45] proceeds similarly; see Theorem 4.1 below, except that the number of pinned coordinates $\bm{\theta}$ is chosen randomly, rather than deligently given $G$. The same is true of the number of pinned coordinates in Theorem 1.12, which additionally yields a continuous version applicable to general $\Omega$-laws.

Finally, there have been several further related contributions that extend the classical (dense) theory of graph limits as set out in [32]. Just as the classical theory, these extensions partly have a probabilistic component as they incorporate random graphs. For example, the important $L^{p}$-theory of sparse graph convergence covers limit objects of exchangeable sparse graphs [9]. A further contribution pertinent to sparse random graphs is the work of Crane and Dempsey [17] and Cai, Campbell and Broderick [15] on edge-exchangeability. Moreover, the articles [8, 50] deal with graphexes, which are limit objects of random geometric graphs. Further important extensions of the theory of graph limits include the work of Nešetřil and Ossona de Mendez [41] on convergence of sparse graphs that satisfy first order formulas, the paper of Hoppen, Kohayakawa, Moreira, Rath and Sampaio [26] on sequences of permutations (permutons), the article by Coregliano and Razborov [16] on limits on dense combinatorial objects, and Janson’s work [27] on limits of posets. Some of these contributions, as well as Austin’s work [3, 4] involve stronger versions of exchangeability than the classical de Finetti or Aldous-Hoover notions of exchangeability.

1.4. Outline

After presenting the necessary background and notation in Section 2, in Section 3 we will prove the basic facts about laws and the cut metric stated above. Specifically, Section 3 contains the proofs of Theorems 1.1, 1.4, 1.8, 1.9, 1.11, 1.13 and 1.14. Subsequently, in Section 4 we prove Theorem 1.12, which constitutes the main technical contribution of the paper. Finally, in Section 4.4 we establish Theorem 1.2.

2. Preliminaries

2.1. Measure theory

Throughout the paper we continue to denote by $\lambda$ the Lebesgue measure on the Euclidean space $\mathbb{R}^{k}$; the reference to $k$ will always be clear from the context. For the convenience of the reader we collect a few basic facts from measure theory that we will need. The first lemma follows from the Isomorphism Theorem, see e.g. [30, Sec. 15.B].

Lemma 2.1**.**

Suppose that ${\mathcal{E}}=(X,\mathcal{A},\mu)$ is a standard Borel space equipped with a probability measure $\mu$. Then there exists a measurable map $f:[0,1]\to X$ that maps the Lebesgue measure to $\mu$.

Lemma 2.2** (Theorem 3.2 of [37]).**

Suppose that ${\mathcal{E}}=(X,\mathcal{A})$, ${\mathcal{E}}^{\prime}=(X^{\prime},\mathcal{A}^{\prime})$ are standard Borel spaces and that $f:X\to X^{\prime}$ is a measurable bijection. Then its inverse $f^{-1}$ is measurable.

Lemma 2.3** (Theorem A.7 of [28]).**

If $(\mathcal{X},\mu)$ is an atomless Borel probability space and $\lambda$ is the Lebesgue measure, then there is a measure preserving bijection of $(\mathcal{X},\mu)$ to $([0,1],\lambda)$.

The following is the Riesz-Markov-Kakutani representation theorem [24].

Lemma 2.4**.**

Suppose that ${\mathcal{E}}_{0}$ is a compact metric space and that $\varphi:C({\mathcal{E}}_{0})\to\mathbb{R}$ is a positive linear functional on the space of continuous functions $C({\mathcal{E}}_{0})$ on ${\mathcal{E}}_{0}$. Moreover, assume that $\varphi(\bm{1})=1$. Then there exists a unique probability measure $\mu$ on ${\mathcal{E}}_{0}$ such that $\varphi(f)=\int_{{\mathcal{E}}_{0}}f{\mathrm{d}}\mu$ for all $f\in C({\mathcal{E}}_{0})$.

We will need Lemma 2.4 in Section 3.4 to prove the completeness of the space of laws with respect to the cut metric.

Additionally, in several places throughout the paper we will need the following metric on probability measures. Suppose that $({\mathcal{E}},D)$ is a complete separable metric space and that $D$ is bounded. Then the space $\mathcal{P}({\mathcal{E}})$ of probability measures on ${\mathcal{E}}$ equipped with the Wasserstein metric

[TABLE]

where we recall that $\Gamma(\mu,\nu)$ is the set of all couplings of $\mu,\nu$, also is complete and separable. The Wasserstein metric induces the weak topology on $\mathcal{P}({\mathcal{E}})$ [51, Theorem  6.9]. The definition (2.1) extends to ${\mathcal{E}}$-valued random variables $\bm{X},\bm{Y}$, for which we define

[TABLE]

with $\Gamma(\bm{X},\bm{Y})$ denoting the set of all couplings of $\bm{X},\bm{Y}$. We will frequently be working with the Wasserstein metric $\mathcal{D}_{\boxtimes}(\,\cdot\,,\,\cdot\,)$ induced by the cut metric on $\mathfrak{L}$ or $\mathfrak{K}{}$.

2.2. Variations on the cut metric

When we defined the cut metric $D_{\boxtimes}(\mu,\nu)$ in (1.2) we allowed for a coupling of $\mu,\nu$ as well as a ‘coordinate permutation’ $\varphi\in\SS$. Sometimes the latter is not desirable. Therefore, for $\mu,\nu\in\mathcal{L}$ we define the strong cut distance as

[TABLE]

with $S,X$ ranging over measurable sets. It is easily verified that $D_{\boxslash}(\,\cdot\,,\,\cdot\,)$ is a pre-metric on $\mathcal{L}$. Analogously, for $\mu,\nu\in\mathcal{P}\left(\Omega^{n}\right)$ let

[TABLE]

Similarly, we will be led to consider several variants of the kernel cut metric from (1.3). Specifically, let $\mathcal{K}_{\mathbb{R}}=\mathcal{K}_{\mathbb{R}}(\Omega)$ be the set of all maps $\kappa,\kappa^{\prime}:[0,1]^{2}\to\mathbb{R}^{\Omega}$ such that the functions $(s,x)\in[0,1]^{2}\mapsto\kappa_{s,x}(\omega)$ belong to $L^{1}([0,1]^{2},\mathbb{R})$ for all $\omega\in\Omega$, up to equality almost everywhere. Then for $\kappa,\kappa^{\prime}\in\mathcal{K}_{\mathbb{R}}$ we define

[TABLE]

Thus, $D_{\boxtimes}(\,\cdot\,,\,\cdot\,)$ is the natural extension of (1.3) to $\mathcal{K}_{\mathbb{R}}$, $D_{\boxslash}(\,\cdot\,,\,\cdot\,)$ is the kernel version of (2.2), $D_{\oblong}(\,\cdot\,,\,\cdot\,)$ represents the strongest variant of the cut metric that does not allow for any measure-preserving transformations, and $D_{\blacksquare}(\,\cdot\,,\,\cdot\,)$ is the graphon cut metric as studied in [33]. We also recall the graphon cut (pre-)metric for $L^{1}$-functions $\kappa,\kappa^{\prime}:[0,1]^{2}\to\mathbb{R}$ from [32], which is defined as

[TABLE]

The different variants of the cut metric are related as follows. For a measurable map $\varphi:[0,1]\to[0,1]$ and $\kappa\in\mathcal{K}_{\mathbb{R}}$ define $\kappa_{\varphi},\kappa^{\varphi}\in\mathcal{K}_{\mathbb{R}}$ by letting $\kappa_{\varphi\,s,x}=\kappa_{s,\varphi(x)}$ and $\kappa^{\varphi}_{s,x}=\kappa_{\varphi(s),x}$, respectively. Then

[TABLE]

As a consequence, for all $\kappa,\kappa^{\prime}\in\mathcal{K}_{\mathbb{R}}$ we have

[TABLE]

For a function $W:(s,x)\mapsto W_{s,x}$ defined on $[0,1]^{2}$ we define the transpose $W^{\dagger}:(s,x)\mapsto W_{x,s}$. We call $W$ symmetric if $W=W^{\dagger}$. For $\kappa\in\mathcal{K}_{\mathbb{R}}$ we define a family $(\kappa^{(\omega)})_{\omega\in\Omega}$ of symmetric functions defined by

[TABLE]

We can interpret $\kappa_{s,x}^{(\omega)}$ as the edge weight in a bipartite graph with vertex set $[0,1]$. We stress, that in (2.6) the ordering of $s$ and $x$ is quite delicate.

Lemma 2.5**.**

For all $\kappa\in\mathcal{K}_{\mathbb{R}}$ we have $D_{\oblong}(\kappa,\kappa^{\prime})=2\max_{\omega\in\Omega}D_{\oblong}(\kappa^{(\omega)},{\kappa^{\prime}}^{(\omega)}).$

Proof.

Given $\omega\in\Omega$ and $S,X\subset[0,1]$ let $T=\left\{{(1+s)/2:s\in S}\right\}\cup\left\{{x/2:x\in X}\right\},\ Y=\left\{{(1+x)/2:x\in X}\right\}\cup\left\{{s/2:s\in S}\right\}.$ Then by construction

[TABLE]

Hence, $D_{\oblong}(\kappa,\kappa^{\prime})\leq 2\max_{\omega\in\Omega}D_{\oblong}(\kappa^{(\omega)},{\kappa^{\prime}}^{(\omega)})$. Regarding the converse bound, we may assume by symmetry that $T,Y\subset[0,1]$ satisfy $T=1-Y$. Indeed, the choice $T=1-Y$ incorporates that the upper right part of the kernel is the transposed lower left part. Therefore, letting $S=\left\{{2t-1:t\in T\cap[1/2,1]}\right\},\ X=\left\{{2t:t\in T\cap[0,1/2]}\right\}$ we again obtain (2.7), and thus $D_{\oblong}(\kappa,\kappa^{\prime})\geq 2\max_{\omega\in\Omega}D_{\oblong}(\kappa^{(\omega)},{\kappa^{\prime}}^{(\omega)})$. ∎

Remark 2.6**.**

Clearly, the cut metric from the theory of graph limits $D_{\blacksquare}(\cdot,\cdot)$ can be bounded from below by the present definition $D_{\boxtimes}(\cdot,\cdot)$, as one must apply the same measure-preserving transformation on both axes. On the other hand, for $\Omega=\left\{{0,1}\right\}$, once we turn kernels $\kappa,\kappa^{\prime}\in\mathfrak{K}$ into ‘bipartite graphons’ via (2.6), we find directly

[TABLE]

The converse bound does not hold for any constant as can be seen as follows. Let $\kappa_{s,x}=\bm{1}\left\{{s<1/2}\right\}$ and $\kappa^{\prime}_{s,x}=\kappa_{x,s}$. By choosing the measure preserving map $\varphi(x)=1-x$, we get

[TABLE]

But as $\kappa^{\prime}$ represents the law $\nu$ supported only on $\delta_{\sigma}$ with $\sigma_{x}=\mathbf{1}\left\{{x\leq 1/2}\right\}$, whilst $\kappa^{\prime}$ is the uniform distribution over the two configurations $\sigma_{1}=1$ and $\sigma_{0}=0$, we can bound $D_{\boxtimes}(\kappa,\kappa^{\prime})\geq 1/4$.

For $\kappa,\kappa^{\prime}\in L^{1}([0,1]^{2},\mathbb{R})$ we define

[TABLE]

Then $\left\|{\,\cdot\,}\right\|_{\Box}$ is a norm on $L^{1}([0,1]^{2},\mathbb{R})$. Analogously, for a matrix $A\in\mathbb{R}^{n\times n}$ we define

[TABLE]

We need the following ‘sampling lemma’ for the cut norm.

Lemma 2.7** ([32, Lemma 10.6]).**

Suppose that $\kappa:[0,1]^{2}\to[-1,1]$ is symmetric. Let $\bm{x}_{1},\ldots,\bm{x}_{k}$ be independently and uniformly chosen from $[0,1]$. Denote by $\kappa[k]\in[-1,1]^{k\times k}$ the matrix with entries $\kappa_{i,j}[k]=\kappa_{\bm{x}_{i},\bm{x}_{j}}$. Then

[TABLE]

2.3. The $L_{1}$-metric

We define a subspance of $\mathcal{K}_{\mathbb{R}}$ by letting

[TABLE]

Similarly, we let $\mathcal{S}_{1}$ be the space of all measurable functions $\sigma:[0,1]\to[0,1]^{\Omega}$. Further, we denote the $L_{1}$-metric on $\mathcal{K}_{1}$ and $\mathcal{S}_{1}$ by $D_{1}(\,\cdot\,,\,\cdot\,)$. Thus,

[TABLE]

and similarly for $\mathcal{S}_{1}$.

2.4. Regularity

For a kernel $\kappa\in\mathcal{K}_{\mathbb{R}}{}$ and partitions $S=(S_{1},\ldots,S_{k})$, $X=(X_{1},\ldots,X_{\ell})$ of the unit interval into pairwise disjoint measurable subsets define $\kappa^{S,X}\in\mathcal{K}{}$ by

[TABLE]

In words, $\kappa^{S,X}_{s,x}$ is the conditional expectation of $\kappa_{\bm{s},\bm{x}}$ given the $\sigma$-algebra generated by the rectangles $S_{i}\times X_{j}$. If the two partitions $S,X$ are identical, we write $\kappa^{S}$ instead of $\kappa^{S,X}$. We use similar notation for maps $\kappa:[0,1]^{2}\to\mathbb{R}$. The following fact is a kernel variant of the well-known Frieze-Kannan regularity lemma.

Lemma 2.8** ([32, Corollary 9.13]).**

For every symmetric $\kappa:[0,1]^{2}\to[0,1]$ and every $k\geq 1$ there exists a partition $S=(S_{1},\ldots,S_{k})$ of $[0,1]$ into pairwise disjoint measurable sets such that $D_{\oblong}(\kappa,\kappa^{S})\leq 2/\sqrt{\log k}.$

This notion of regularity is robust with respect to refining the partition.

Lemma 2.9** ([32, Lemma 9.12]).**

Let $\kappa:[0,1]^{2}\to[0,1]$ be symmetric and $\kappa^{\prime}:[0,1]^{2}\to[0,1]$ be a symmetric step function and denote by $S$ a partition of $[0,1]$ into a finite number of meausrable sets on which $\kappa^{\prime}$ is constant. Then $D_{\oblong}(\kappa,\kappa^{S})\leq 2D_{\oblong}(\kappa,\kappa^{\prime}).$

Applying Lemma 2.9 to the step function $\kappa^{R}$ for a partition $R$ that refines a partition $S$ of $[0,1]$, we obtain the following corollary.

Corollary 2.10**.**

Let $R,S$ be partitions of $[0,1]$ such that $R$ refines $S$. Then $D_{\oblong}(\kappa,\kappa^{R})\leq 2D_{\oblong}(\kappa,\kappa^{S}).$

Proof.

This follows from Lemma 2.9 because $\kappa^{S}$ is constant on the partition classes of $R$. ∎

3. Fundamentals

This section contains the proofs of the basic facts, namely the compactness of the space of $\Omega$-laws (Theorem 1.1), the isometric property of the kernel representation (Theorem 1.4), the sampling theorem (Theorem 1.9), the comparison of the discrete and the continuous cut metric (Theorem 1.2), the continuity statements from Theorems 1.13 and 1.14 and the connection to exchangeable arrays (Theorem 1.8). We begin with the proof of Theorem 1.4.

3.1. Proof of Theorem 1.4

Any measurable map $f:[0,1]\to\mathcal{S}$, $s\mapsto f_{s}$ induces a kernel $\kappa^{f}:[0,1]^{2}\to\mathcal{P}(\Omega)$, $(s,x)\mapsto f_{s,x}\in\mathcal{P}(\Omega)$. Moreover, $f$ maps the Lebesgue measure on $[0,1]$ to a probability distribution $\mu^{f}\in\mathcal{L}$.

Lemma 3.1**.**

Suppose that $f,g:[0,1]\to\mathcal{S}$ are measurable. Then $D_{\boxslash}(\mu^{f},\mu^{g})\leq D_{\boxslash}(\kappa^{f},\kappa^{g})$.

Proof.

Fix $\omega\in\Omega$ and $\varphi\in\SS$. The construction of $\kappa^{f},\kappa^{g}$ guarantees that with $\bm{s}\in[0,1]$ chosen uniformly at random, the distribution $\gamma$ of the pair $(\kappa^{f}_{\bm{s}},\kappa^{g}_{\varphi(\bm{s})})\in\mathcal{S}\times\mathcal{S}$ is a coupling of $\mu^{f},\mu^{g}$. We now claim that

[TABLE]

Indeed, fix measurable $T\subset\mathcal{S}^{2}$ and $X\subset[0,1]$ and let $S=\left\{{s\in[0,1]:(\kappa^{f}_{s},\kappa^{g}_{\varphi(s)})\in T}\right\}$. Then by the construction of $\gamma$,

[TABLE]

whence (3.1) follows. Finally, since (3.1) holds for all $\varphi,\omega$, we conclude that $D_{\boxslash}(\mu^{f},\mu^{g})\leq D_{\boxslash}(\kappa^{f},\kappa^{g})$. ∎

The following lemma establishes the converse of Lemma 3.1 for functions that take only finitely many values.

Lemma 3.2**.**

Suppose that $f,g:[0,1]\to\mathcal{S}$ are measurable maps whose images $f([0,1]),g([0,1])\subset\mathcal{S}$ are finite sets. Then $D_{\boxslash}(\kappa^{f},\kappa^{g})\leq D_{\boxslash}(\mu^{f},\mu^{g})$.

Proof.

Suppose that $f([0,1])=\{\sigma_{1},\ldots,\sigma_{k}\}$ and $g([0,1])=\{\tau_{1},\ldots,\tau_{\ell}\}$. Moreover, let $V_{i}$ be the set of all $s\in[0,1]$ such that $f(s)=\sigma_{i}$ and let $W_{j}$ be the set of all $s\in[0,1]$ such that $g(s)=\tau_{j}$. In addition, let $v_{i}=\lambda(V_{i})$, $w_{j}=\lambda(W_{j})$. Then

[TABLE]

Consequently, any coupling $\gamma$ of $\mu^{f},\mu^{g}$ induces a coupling $\Gamma\in\mathcal{P}([k]\times[\ell])$ of the probability distributions $(v_{1},\ldots,v_{k})$ and $(w_{1},\ldots,w_{\ell})$. To turn $\Gamma$ into a measure-preserving map $[0,1]\to[0,1]$ we partition any sets $V_{i},W_{j}$ into pairwise disjoint measurable subsets $(V_{i,h})_{h\in[\ell]}$ and $(W_{h,j})_{h\in[k]}$, respectively, such that for all $i,j,h$,

[TABLE]

Then by Lemma 2.3 for any $i,j$ there exists a bijection $\varphi_{i,j}:V_{i,j}\to W_{i,j}$ such that both $\varphi_{i,j}$ and $\varphi_{i,j}^{-1}$ are measurable and preserve the Lebesgue measure. Piecing these maps together, we obtain the bijection

[TABLE]

Both $\varphi$ and $\varphi^{-1}$ are measurable and preserve the Lebesgue measure, i.e., $\varphi\in\SS$. Moreover, for any sets $S,X\subset[0,1]$ and any $\omega\in\Omega$ we have

[TABLE]

Hence, (3.2) is extremised by sets $S$ such that for all $i,j$ either $V_{ij}\subset S$ or $S\cap V_{ij}=\emptyset$. For such a set $S$ let $T=T(S)$ contain all pairs $(i,j)$ such that $V_{ij}\subset S$. Then (3.2) yields

[TABLE]

Since (3.3) holds for all $S,X,\omega,\gamma$, the assertion follows. ∎

Corollary 3.3**.**

Let $f,g:[0,1]\to\mathcal{S}$ be measurable. Then $D_{\boxslash}(\kappa^{f},\kappa^{g})\leq D_{\boxslash}(\mu^{f},\mu^{g})$.

Proof.

Because $\mathcal{S}$ is a convex subset of the separable Banach space $L^{1}([0,1],\mathbb{R}^{\Omega})$, the measurable maps $f,g$ are pointwise limits of sequences $(f_{n})_{n\geq 1}$, $(g_{n})_{n\geq 1}$ of measurable functions $f_{n},g_{n}:[0,1]\to\mathcal{S}$ whose images are finite sets. Moreover, Lemma 3.2 implies that

[TABLE]

Further, for all $\omega\in\Omega$ and $S,X\subset[0,1]$ we have

[TABLE]

Because $f_{n}\to f$ pointwise, the r.h.s. of (3.5) vanishes as $n\to\infty$. Consequently,

[TABLE]

Combining (3.6) with Lemma 3.1, we conclude that

[TABLE]

Finally, the assertion follows from (3.4), (3.6), (3.7) and the triangle inequality. ∎

Corollary 3.4**.**

For all $\kappa,\kappa^{\prime}\in\mathcal{K}{}$ we have $D_{\boxslash}(\mu^{\kappa},\mu^{\kappa^{\prime}})=D_{\boxslash}(\kappa,\kappa^{\prime})$.

Proof.

This is an immediate consequence of Lemma 3.1 and Corollary 3.3. ∎

Proof of Theorem 1.4.

Corollary 3.4 and (2.4) show that the map $\mathfrak{K}\to\mathfrak{L}$, $\kappa\mapsto\mu^{\kappa}$ is an isometry. Moreover, Lemma 2.1 implies that this map is surjective. Thus, because $\mathfrak{K},\mathfrak{L}$ are metric spaces, $\kappa\mapsto\mu^{\kappa}$ is an isometric bijection. ∎

3.2. Proof of Theorem 1.9

We begin by extending Lemma 2.8 to (not necessarily symmetric) kernels $\kappa\in\mathcal{K}{}$.

Lemma 3.5**.**

There is $c=c(\Omega)>0$ such that for any {\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\varepsilon\in(0,1)}, $\kappa\in\mathcal{K}$ there exist partitions $S=(S_{1},\ldots,S_{k})$, $X=(X_{1},\ldots,X_{\ell})$ of the unit interval into measurable subsets such that $k+\ell\leq\exp(c/\varepsilon^{2})$ and $D_{\oblong}(\kappa,\kappa^{S,X})<\varepsilon$.

Proof.

Let $\ell=\lceil\exp(c^{\prime}/\varepsilon^{2})\rceil$ for a large enough $c^{\prime}=c^{\prime}(\Omega)$. Applying Lemma 2.8 to the kernels $\kappa^{\left({\omega}\right)}$ from (2.6), we obtain partitions $T^{\left({\omega}\right)}=(T^{\left({\omega}\right)}_{1},\ldots,T^{\left({\omega}\right)}_{\ell})$ of $[0,1]$ such that

[TABLE]

Let $T=(T_{1},\ldots,T_{k})$ be the coarsest common refinement of all the partitions $T^{\left({\omega}\right)}$ and of the partition $\{[0,1/2),[1/2,1]\}$. Then

[TABLE]

Moreover, (3.8) and Corollary 2.10 imply that

[TABLE]

Further, let $S^{\prime}=(S_{1}^{\prime},\ldots,S_{K}^{\prime})$ comprise all partition classes $T_{i}\subset[0,1/2]$ and let $X^{\prime}=(X_{1}^{\prime},\ldots,X_{L}^{\prime})$ be the partition of $[1/2,1]$ consisting of all the classes $T_{i}\subset[1/2,1]$. Finally, let $S_{i}=\{2s:s\in S_{i}^{\prime}\}$ and $X_{i}=\{2x-1:x\in X_{i}^{\prime}\}$. Then the partitions $S=(S_{1},\ldots,S_{K})$ and $X=(X_{1},\ldots,X_{L})$ satisfy $D_{\oblong}(\kappa,\kappa^{S,X})<\varepsilon$ by Lemma 2.5. The desired bound on the total number $K+L$ of classes of $S,X$ follows from (3.9). ∎

For a kernel $\kappa$ and an integer $n$ obtain $\kappa_{n}$ as follows. Draw $\bm{x}_{1},\bm{s}_{1},\ldots,\bm{x}_{n},\bm{s}_{n}\in[0,1]$ uniformly and independently and let $\kappa_{n}$ be the kernel representing the matrix $(\kappa_{\bm{s}_{i},\bm{x}_{j}})_{i,j}$. Additionally, obtain $\hat{\kappa}_{n}\in\Omega^{n\times n}$ by letting $\hat{\kappa}_{n,i,j}=\omega$ with probability $\kappa_{\bm{s}_{i},\bm{x}_{j}}(\omega)$ independently for all $i,j$. We identify $\hat{\kappa}_{n}$ with its kernel representation. Moreover, we notice that $\hat{\kappa}_{n}$ coincides with the $n\times n$ upper left sub-matrix of $\bm{X}^{\kappa}$ from Section 1.2.3.

Lemma 3.6**.**

Let $\kappa,\kappa^{\prime}\in\mathcal{K}{}$. With probability $1-\exp(-\Omega(\sqrt{n}))$ we have $D_{\oblong}\left({\kappa_{n},\kappa_{n}^{\prime}}\right)=O(D_{\oblong}(\kappa,\kappa^{\prime})+n^{-1/4})$.

Proof.

Let $\bar{\kappa},\bar{\kappa}^{\prime}$ be the symmetric kernel representations of $\kappa$ and $\kappa^{\prime}$ respectively given via (2.6). Sample $\bm{y}_{1},\ldots,\bm{y}_{2n}$ points in $[0,1]$ uniformly and independently at random. Denote by $\mathcal{B}$ the event that $\left|{i:\bm{y}_{i}\leq\frac{1}{2}}\right|=n$ and assume, given $\mathcal{B}$, that without loss $\bm{y}_{1},\ldots,\bm{y}_{n}\leq 1/2$ and $\bm{y}_{n+1},\ldots,\bm{y}_{2n}\geq 1/2$. Denote by $\bm{x}_{1},\ldots,\bm{x}_{n}=2\bm{y}_{1},\ldots,2\bm{y}_{n}$ and by $\bm{s}_{1},\ldots,\bm{s}_{n}=2\bm{y}_{n+1}-1,\ldots,2\bm{y}_{2n}-1$. Clearly, $(\bm{x}_{1},\bm{s}_{1}),\ldots,(\bm{x}_{n},\bm{s}_{n})$ are independent uniform samples from $[0,1]^{2}$.

Now, let $\tilde{\kappa}:[0,1]^{2}\to[-1,1]$ be the kernel representing the matrix $(\bar{\kappa}_{\bm{y}_{i},\bm{y}_{j}}-\bar{\kappa}_{\bm{y}_{i},\bm{y}_{j}}^{\prime})_{i,j\in[2n]}$. Applying Lemma 2.7 to $\tilde{\kappa}^{\left({\omega}\right)}$, we obtain

[TABLE]

Given $\mathcal{B}$, we translate $\tilde{\kappa},\bar{\kappa},\bar{\kappa}^{\prime}$ back into kernels via (2.6) and apply Lemma 2.5, thus

[TABLE]

Hence, the assertion follows from (3.11) and the fact that $\mathbb{P}\left({\mathcal{B}}\right)=\Omega\left({n^{-1/2}}\right)$. ∎

Lemma 3.7**.**

We have $\mathbb{E}[D_{\oblong}(\kappa_{n},\hat{\kappa}_{n})]=O(n^{-1/2})$.

Proof.

We adapt the simple argument from the proof of [32, Lemma 10.11] for our purposes. Letting $X_{i,j,\omega}=\mathbf{1}\left\{{\hat{\kappa}_{n,i,j}=\omega}\right\}$, we have $\mathbb{E}[X_{i,j,\omega}]=\kappa_{n,i,j}(\omega)$. Furthermore, because both $\kappa_{n},\hat{\kappa}_{n}$ are kernel representations of $n\times n$ matrices, the supremum

[TABLE]

is attained at sets $S,X$ that are unions of intervals $[(i-1)/n,i/n)$ with $i\in[n]$. Hence,

[TABLE]

Now, for any $\omega,I,J$ the random variable $\sum_{i\in I}\sum_{j\in J}X_{i,j,\omega}$ is a sum of $|I\times J|$ independent Bernoulli variables. Therefore, Azuma’s inequality yields

[TABLE]

Since (3.13) holds for any specific $I,J,\omega$, the assertion follows from the union bound and (3.12). ∎

Proof of Theorem 1.9.

Lemma 3.5 yields partitions $X=(X_{1},\ldots,X_{\ell})$, $S=(S_{1},\ldots,S_{\ell})$ of $[0,1]$ with $\ell\leq n^{1/4}$ such that

[TABLE]

Applying Lemma 3.6 to $\kappa$ and $\kappa^{S,X}$, we obtain

[TABLE]

In addition, we claim that

[TABLE]

To see this, let

[TABLE]

Since $N_{h},M_{h}$ are binomial variables, the Chernoff bound shows that with probability $1-o(1/n)$,

[TABLE]

Let

[TABLE]

Providing that the bounds (3.17) hold, we can construct $\varphi,\psi\in\SS$ such that for all $h\in[n]$,

[TABLE]

Furthermore, by construction we have $\kappa^{S,X}_{\varphi(s),\psi(x)}=\kappa^{S,X}_{s,x,n}$ if there exist $h,h^{\prime}\in[\ell]$ such that $x\in\mathcal{N}_{h}$, $\varphi(x)\in X_{h}$ and $s\in\mathcal{M}_{h}$, $\psi(x)\in S_{h}$. Therefore, (3.18) implies that for all $T,Y\subset[0,1]$, $\omega\in\Omega$,

[TABLE]

whence (3.16) follows. Combining (3.14), (3.15) and (3.16), we see that

[TABLE]

Finally, (3.19), Lemma 3.7 and Theorem 1.4 imply the assertion. ∎

3.3. Proof of Theorems 1.13 and 1.14

For measurable $k,k^{\prime}:[0,1]^{3}\to[0,1]^{\Omega}$ we let

[TABLE]

Then $D_{\oblong}(\,\cdot\,,\,\cdot\,)$ defines a pre-metric. Further, for measurable $\kappa,\kappa^{\prime}:[0,1]^{2}\to[0,1]^{\Omega}$ we define

[TABLE]

We will derive Theorem 1.14 from the following statement.

Proposition 3.8**.**

The map $(\kappa,\kappa^{\prime})\mapsto\kappa\oplus^{\prime}\kappa^{\prime}$ is $D_{\oblong}$-continuous.

Proof.

Given $\varepsilon>0$ choose a small $\delta=\delta(\varepsilon)>0$. Suppose that $D_{\oblong}(\kappa,\kappa^{\prime})<\delta$. Due to the triangle inequality, to establish continuity it suffices to show that for every $\kappa^{\prime\prime}:[0,1]^{2}\to[0,1]^{\Omega}$,

[TABLE]

Thus, consider measurable $X,S$ and fix $\omega,\omega^{\prime}\in\Omega$. To estimate the last integral consider $y\in[0,1]$ and let $X_{y}=\left\{{x\in[0,1]:(x,y)\in X}\right\}\subset[0,1]$. Moreover, let $T_{1},\ldots,T_{\ell}$ be a decomposition of $S$ into pairwise disjoint measurable sets such that for all $j\in[\ell]$ we have

[TABLE]

Since $\kappa^{\prime\prime}_{s,y}(\omega^{\prime})\in[0,1]$, we may assume that $\ell\leq 4/\varepsilon$. Furthermore,

[TABLE]

Since this estimate holds for all $y\in[0,1]$, we obtain

[TABLE]

for all $S,X,\omega,\omega^{\prime}$. Thus, we obtain (3.20). ∎

Proof of Theorem 1.14.

Theorem 1.14 follows from Proposition 3.8 and (2.4). ∎

We use a similar argument to prove Theorem 1.13. Specifically, for $\kappa,\kappa^{\prime}:[0,1]^{2}\to[0,1]^{\Omega}$ define

[TABLE]

Proposition 3.9**.**

The map $(\kappa,\kappa^{\prime})\mapsto\kappa\otimes^{\prime}\kappa^{\prime}$ is $D_{\oblong}$-continuous.

Proof.

The definition of $D_{\oblong}(\,\cdot\,,\,\cdot\,)$ ensures that the map $\kappa\mapsto\kappa^{\dagger}$, where $\kappa^{\dagger}_{s,x}=\kappa_{x,s}$, is continuous. Therefore, the assertion follows from Proposition 3.8. ∎

Proof of Theorem 1.13.

Theorem 1.13 follows immediately from Proposition 3.9 and (2.4). ∎

3.4. Proof of Theorem1.1

We begin by proving that the space $\mathcal{K}{}$ is complete with respect to $D_{\oblong}(\,\cdot\,,\,\cdot\,)$, the strongest version of the cut metric.

Lemma 3.10**.**

The space $\mathcal{K}$ equipped with the $D_{\oblong}(\,\cdot\,,\,\cdot\,)$ metric is complete.

Proof.

Suppose that $(\kappa_{n})_{n\geq 1}$ is a Cauchy sequence. Then for any measurable $S,X\subset[0,1]$ and any $\omega\in\Omega$ the sequence $\int_{S}\int_{X}\kappa_{n,s,x}(\omega){\mathrm{d}}x{\mathrm{d}}s$ is Cauchy as well. Therefore, because any continuous function $f:[0,1]^{2}\to\mathbb{R}$, $(s,x)\mapsto f_{s,x}$ is uniformly continous, the limit

[TABLE]

exists for every $\omega\in\Omega$. Indeed, the map

[TABLE]

defines a positive linear functional on the space of all continuous functions $[0,1]^{2}\to\mathbb{R}$. Hence, by the Riesz representation theorem (Lemma 2.4) there exists a unique measure $\mu_{\omega}$ on $[0,1]^{2}$ such that

[TABLE]

Indeed, the condition (3.21) ensures that $\mu_{\omega}$ is absolutely continuous with respect to the Lebesgue measure. Therefore, the Radon-Nikodym theorem yields an $L^{1}$-function $(s,x)\in[0,1]^{2}\mapsto\kappa_{s,x}(\omega)\in\mathbb{R}_{\geq 0}$ such that

[TABLE]

We claim that $\kappa$ is a kernel, i.e., that $\sum_{\omega\in\Omega}\kappa_{s,x}(\omega)=1$ for almost all $s,x$. Indeed, combining (3.21) and (3.22) yields

[TABLE]

Since the rectangles $S\times X$ generate the Borel algebra on $[0,1]^{2}$, (3.23) implies that $\sum_{\omega\in\Omega}\kappa_{s,x}(\omega)=1$ almost everywhere.

Finally, (3.21) and (3.22) show that $\lim_{n\to\infty}D_{\oblong}(\kappa_{n},\kappa)=0$. Indeed, given $\varepsilon>0$ consider a large enough $n$ and let $S,X,\omega$ be such that

[TABLE]

Equations (3.21) and (3.22) show that for large enough $N>n$,

[TABLE]

Combining (3.24) and (3.25) and recalling that $(\kappa_{n})_{n}$ is $D_{\oblong}$-Cauchy, for large enough $n$ we obtain

[TABLE]

Hence, $\kappa_{n}$ converges to $\kappa$. ∎

Corollary 3.11**.**

The space $\mathcal{K}$ equipped with the $D_{\boxslash}(\,\cdot\,,\,\cdot\,)$ metric is complete.

Proof.

We adapt a well know proof that a quotient of a Banach space with respect to a linear subspace is complete [10, Theorem 1.12.14]. Thus, suppose that $(\kappa_{n})_{n}$ is a $D_{\boxslash}(\,\cdot\,,\,\cdot\,)$-Cauchy sequence. There exists a subsequence $(\kappa_{n_{\ell}})_{\ell}$ such that $D_{\boxslash}(\kappa_{n_{\ell}},\kappa_{n_{\ell+1}})<2^{-\ell}$ for all $\ell$. Hence, passing to this subsequence, we may assume that $(\kappa_{n})_{n}$ satisfies

[TABLE]

We are now going to construct a sequence $(k_{n})_{n}$ of maps $[0,1]^{2}\to\mathcal{P}(\Omega)$ such that $D_{\boxslash}(\kappa_{n},k_{n})=0$ for all $n$ and

[TABLE]

We let $k_{1}$ be any kernel such that $D_{\boxslash}(k_{1},\kappa_{1})=0$ and proceed by induction. Having constructed $k_{1},\ldots,k_{n}$ already, we observe that the definition of $D_{\boxslash}(\,\cdot\,,\,\cdot\,)$ ensures that

[TABLE]

Therefore, (3.26) implies that there is $k_{n+1}:[0,1]^{2}\to\mathcal{P}(\Omega)$ with $D_{\boxslash}(\kappa_{n+1},k)=0$ such that $D_{\oblong}(k_{n},k_{n+1})<2^{-n}$. Thus, we obtain a sequence $(k_{n})_{n}$ satisfying (3.27). Finally, any sequence $(k_{n})_{n}$ that satisfies (3.27) is $D_{\oblong}(\,\cdot\,,\,\cdot\,)$-Cauchy. Therefore, Lemma 3.10 shows that $(k_{n})_{n}$ has a limit $k$. Since $D_{\boxslash}(k_{n},\kappa_{n})=0$, we conclude that

[TABLE]

i.e., $(\kappa_{n})_{n}$ converges to $k$. ∎

Corollary 3.12**.**

The spaces $\mathfrak{K}$ and $\mathfrak{L}$ equipped with the $D_{\boxtimes}(\,\cdot\,,\,\cdot\,)$ metric are complete and separable.

Proof.

To establish the completeness of $\mathfrak{K}{}$ we repeat the same argument as in the proof of Corollary 3.11. The completeness of $\mathfrak{L}$ then follows from Theorem 1.4. Moreover, Theorem 3.5 shows that the set of laws with finite support is dense in $\mathfrak{L}$. Hence, to prove the separability of $\mathfrak{L}$ it suffices to observe that the space $\mathcal{S}$ is separable, which it is because the set of all finite linear combinations of indicator functions $x\mapsto\bm{1}\{a<x<b\}$ with $a,b\in\mathbb{Q}$ is dense in $L^{1}([0,1],\mathbb{R})$. Finally, Theorem 1.4 implies that $\mathfrak{K}{}$ is separable as well. ∎

We denote by $\mathcal{P}(\mathfrak{L})$ the space of probability distributions on the Polish space $\mathfrak{L}$, endowed with the topology of weak convergence. As we saw in Section 2.1, this topology is metrised by the Wasserstein metric

[TABLE]

We begin by proving that $\mathcal{P}(\mathfrak{L})$ is compact. To this end we will construct a continuous map from another compact space onto $\mathcal{P}(\mathfrak{L})$. Specifically, recall that $\Omega^{\mathbb{N}\times\mathbb{N}}$ is a compact Polish space with respect to the product topology. The space $\mathcal{P}(\Omega^{\mathbb{N}\times\mathbb{N}})$ equipped with the weak topology is therefore compact as well. Further, the space $\mathfrak{X}\subset\mathcal{P}(\Omega^{\mathbb{N}\times\mathbb{N}})$ of exchangeable distributions is closed with respect to the weak topology, and therefore compact.

To construct a continuous map $\mathfrak{X}\to\mathcal{P}(\mathfrak{L})$, $\xi\mapsto\rho^{\xi}$ we are going to take a pointwise limit of maps $\mathfrak{X}\to\mathcal{P}(\mathfrak{L})$, $\xi\mapsto\rho^{\xi,n}$. Given $\xi\in\mathfrak{X}$ and $n\geq 1$ we define $\rho^{\xi,n}$ as follows. Draw $\bm{X}^{\xi}=(\bm{X}_{i,j}^{\xi})_{i,j\geq 1}\in\Omega^{\mathbb{N}\times\mathbb{N}}$ from $\xi$. Then define a probability distribution on $\Omega^{n}$ by letting

[TABLE]

Thus, $\bm{\mu}_{*}^{\xi,n}$ is the empirical distribution of the rows of the top-left $n\times n$ minor of $\bm{X}^{\xi}$. Finally, let $\bm{\mu}^{\xi,n}=\dot{\bm{\mu}}_{*}^{\xi,n}\in\mathfrak{L}$ be the law induced by this discrete distribution and let $\rho^{\xi,n}\in\mathcal{P}(\mathfrak{L})$ be the distribution of $\bm{\mu}^{\xi,n}$ (with respect to the choice of $\bm{X}^{\xi}$).

Lemma 3.13**.**

For every $\xi\in\mathfrak{X}$ the limit $\rho^{\xi}=\lim_{n\to\infty}\rho^{\xi,n}$ exists and the map $\xi\mapsto\rho^{\xi}$ is continuous.

Proof.

Let $\xi\in\mathfrak{X}$. Since $\mathcal{P}(\mathfrak{L})$ is complete, to establish the existence of the limit we just need to prove that the sequence $(\rho^{\xi,n})_{n}$ is Cauchy. To this end it suffices to verify the following condition:

[TABLE]

Indeed, if (3.29) is satisfied, then there exists a subsequence $(\rho^{\xi,n_{M}})_{M}$ such that $\mathcal{D}_{\boxtimes}(\rho^{\xi,n_{M}},\rho^{\xi,n_{M+1}})<2^{-M}$ for all $M$. In particular, the subsequence is Cauchy. Because $\mathcal{P}(\mathfrak{L})$ is complete, this subsequence thus has a limit $\rho^{\star}$, and (3.29) ensures that the entire sequence $(\rho^{\xi,n})_{n}$ converges to $\rho^{\star}$ as well.

To verify (3.29) let $\varepsilon>0$, pick a large $n=n(\varepsilon)>0$ and choose $N=N(n)$ large enough. We aim to prove that

[TABLE]

To this end, we couple $\rho^{\xi,n},\rho^{\xi,N}$ by drawing $\bm{X}^{\xi}\in\Omega^{\mathbb{N}\times\mathbb{N}}$ from $\xi$ and letting $g$ be the distribution of the pair $(\bm{\mu}^{\xi,n},\bm{\mu}^{\xi,N})$. By definition of the Wasserstein metric, to establish (3.30) it suffices to show that

[TABLE]

But (3.31) follows from Theorem 1.9. Indeed, the construction (3.28) ensures that $\bm{\mu}^{\xi,n}$ is the empirical distribution of rows of the upper left $n\times n$-block of $\bm{X}^{\xi}$, while $\bm{\mu}^{\xi,N}$ is the empirical distribution of the rows of the $N\times N$-upper left block. Due to the exchangeability of $\xi$, the distribution of the upper left $n\times n$-block is identical to the distribution of a random $n\times n$-minor of the matrix $\bm{X}^{\xi}$. Therefore, assuming that $N\gg n^{2}$ so that upon sub-sampling $n$ out of $N$ indices no index is chosen twice, in the notation of Theorem 1.9 we have $d_{\mathrm{TV}}(\bm{\mu}^{\xi,n},\bm{\mu}^{\xi,N}_{n,n})<\varepsilon/2$, whence we obtain (3.31) and thus (3.30). Hence, the limit $\rho^{\xi}=\lim_{n\to\infty}\rho^{\xi,n}$ exists for all $\xi$.

To show continuity fix $\varepsilon>0$ and let $\xi,\eta\in\mathfrak{X}$. Due to (3.31) there exists $n=n(\varepsilon)>0$ independent of $\xi,\eta$ such that

[TABLE]

Since $\mathfrak{X}$ is equipped with the weak topology, any $\xi>0$ admits a neighbourhood $U$ such that for all $\eta\in U$,

[TABLE]

Hence, the upper left $n\times n$-corners of $\bm{X}^{\xi},\bm{X}^{\eta}$ have total variation distance at most $\varepsilon/4$. In effect, there is a coupling of $(\bm{X}_{i,j}^{\xi})_{i,j\in[n]},(\bm{X}_{i,j}^{\eta})_{i,j\in[n]}$ under which both these random $n\times n$-matrices coincide with probability at least $1-\varepsilon/4$. Clearly, this coupling extends to a coupling of the measures $\bm{\mu}^{\xi,n}$, $\bm{\mu}^{\eta,n}$ such that $\mathbb{E}[D_{\boxtimes}(\bm{\mu}^{\xi,n},\bm{\mu}^{\eta,n})]\leq\varepsilon/4$. Consequently, $\mathcal{D}_{\boxtimes}(\rho^{\xi,n},\rho^{\eta,n})\leq\varepsilon/4$. Combining this bound with (3.32), we conclude that $\mathcal{D}_{\boxtimes}(\rho^{\xi},\rho^{\eta})<\varepsilon$ for all $\eta\in U$, whence $\xi\mapsto\rho^{\xi}$ is continuous. ∎

As a next step we are going to embed the space $\mathfrak{L}$ into $\mathfrak{X}{}$. For a given law $\mu\in\mathfrak{L}$ let $\xi^{\mu}$ be the distribution of $\bm{X}^{\mu}$.

Lemma 3.14**.**

The map $\mu\mapsto\xi^{\mu}$ is continuous and $\rho^{\xi^{\mu}}=\delta_{\mu}$.

Proof.

Due to Theorem 1.4 it suffices to show that the map $\kappa\in\mathfrak{K}\mapsto\xi^{\kappa}$, where $\xi^{\kappa}$ is the distribution of $\bm{X}^{\kappa}$, is continuous. Combining Theorems 1.14 and 1.13, we conclude that the map $\mathfrak{K}_{\Omega}\to\mathfrak{K}_{\Omega^{[n]\times[n]}}$, $\kappa\mapsto(\kappa^{\oplus n})^{\otimes n}$ is continuous, where we iterate the $\oplus$ and the $\otimes$ operations $n$ times. For the sake of clarity, let us spell out the precise meaning of iterating these operations. The definition of the $\oplus$-operation extends to kernels $\kappa,\kappa^{\prime}$ that take values in $\mathcal{P}(\Omega)$, $\mathcal{P}(\Omega^{\prime})$ for different sets $\Omega,\Omega^{\prime}$ by simply viewing $\kappa,\kappa^{\prime}$ as $\Omega\cup\Omega^{\prime}$-kernels. With this extension it makes sense to iterate the $\oplus$-operation; notice that $\kappa^{\oplus h}$ takes values in $\Omega^{h}$. We define $\kappa^{\otimes h}$ analogously. Finally, combining these two operations we obtain $\kappa^{\oplus n\,\otimes n}=(\kappa^{\oplus n})^{\otimes n}$, which is an $\Omega^{n\times n}$-kernel. Furthermore, for any $\sigma\in\Omega^{[n]\times[n]}$ the map

[TABLE]

is continuous by the definition of the cut metric. Therefore, being a concatenations of continuous maps, the functions

[TABLE]

are continuous as well. Since $\mathfrak{X}$ carries the weak toplogy, the continuity of the maps $\mathcal{T}_{\sigma}$ implies the continuity of the map $\kappa\mapsto\xi^{\kappa}$. ∎

Corollary 3.15**.**

The map $\mathfrak{X}\to\mathcal{P}(\mathfrak{L})$, $\xi\mapsto\rho^{\xi}$ is surjective.

Proof.

Suppose that $\mathfrak{p}\in\mathcal{P}(\mathfrak{L})$. With $\nu\mapsto\xi^{\nu}$ the measurable map from Lemma 3.14, we define $\xi^{\mathfrak{p}}=\int_{\mathfrak{L}}\delta_{\xi^{\mu}}{\mathrm{d}}\mathfrak{p}(\mu)$. Then $\rho^{\xi^{\mathfrak{p}}}=\mathfrak{p}$. ∎

Corollary 3.16**.**

The space $\mathfrak{L}$ is compact.

Proof.

The space $\mathfrak{X}$ is compact as it is the space of probability measures on the compact Polish space $\Omega^{\mathbb{N}\times\mathbb{N}}$. Since Lemma 3.13 and Corollary 3.15 render a continuous surjective map $\mathfrak{X}\to\mathcal{P}(\mathfrak{L})$ and a continuous image of a compact space is compact, the space $\mathcal{P}(\mathfrak{L})$ is compact. To finally conclude that $\mathfrak{L}$ is compact as well, consider a sequence $(\mu_{n})_{n\geq 1}$ in $\mathfrak{L}$. Because $\mathcal{P}(\mathfrak{L})$ is compact, the sequence $(\delta_{\mu_{n}})_{n\geq 1}$ possesses a convergent subsequence $(n_{\ell})_{\ell\geq 1}$. Let $\pi$ be the limit of that subsequence. Consider a point $\nu$ in the support of $\pi$ and let $(U_{k})_{k\geq 0}$ be a sequence of open neighbourhoods of $\nu$ such that $U_{k+1}\subset U_{k}$ for all $k$ and $\bigcap_{k\geq 1}U_{k}=\{\nu\}$. By Urysohn’s lemma there are continuous functions $f_{k}:\mathfrak{L}\to[0,1]$ such that $f_{k}$ takes the value one on $U_{k}$ and the value [math] outside $U_{k-1}$. Now, for all $k\geq 1$ we have

[TABLE]

Hence, $\mu_{n_{\ell}}\in U_{k-1}$ for almost all $\ell$. Consequently, $\nu=\lim_{\ell\to\infty}\mu_{n_{\ell}}$. Thus, the metric space $\mathfrak{L}$ is sequentially compact and therefore compact. ∎

Of course the second part of the proof above merely establishes the well known fact that the mapping $\mathfrak{L}\to\mathcal{P}(\mathfrak{L})$, $\mu\mapsto\delta_{\mu}$ is a homeomorphic embedding onto a closed subspace. We included the brief argument for the sake of completeness.

Proof of Theorem1.1.

The theorem follows from Corollaries 3.12 and 3.16. ∎

3.5. Proof of Theorem 1.11

Let $\mu,\nu\in\mathcal{L}$. Toward the proof of (1.7) let

[TABLE]

Since $\mu,\nu$ are atoms concentrated on the pure state (1.6), respectively, we obtain

[TABLE]

whence (1.7) is immediate. Moreover, the first part of (1.8) follows from (3.33), while the second part is immediate from the triangle inequality.

3.6. Proof of Theorem 1.8

The product topology on $\Omega^{\mathbb{N}\times\mathbb{N}}$ is the weakest topology under which all the functions

[TABLE]

are continuous. Equivalently, the product topology is induced by the metric

[TABLE]

Hence, the weak topology on $\mathfrak{X}{}\subset\mathcal{P}(\Omega^{\mathbb{N}\times\mathbb{N}})$ is induced by the corresponding Wasserstein metric $\mathcal{D}_{\max}(\,\cdot\,,\,\cdot\,)$.

As a first step we are going to show that the map $\pi\mapsto\Xi^{\pi}$ is $(\mathcal{D}_{\boxtimes},\mathcal{D}_{\max})$-continuous. Indeed, assume that $\pi,\pi^{\prime}\in\mathcal{P}(\mathfrak{K})$ satisfy $\mathcal{D}_{\boxtimes}(\pi,\pi^{\prime})<\delta$ for a small enough $\delta=\delta(\varepsilon)>0$. Then the corresponding coupling shows together with Lemma 3.14 and Theorem 1.4 that $\mathcal{D}_{\max}(\,\cdot\,,\,\cdot\,)<\varepsilon$. Furthermore, $\pi\mapsto\Xi^{\pi}$ is one-to-one because it can be inverted via Corollary 3.15. Moreover, Lemma 3.13 implies that the map $\pi\mapsto\Xi^{\pi}$ is surjective, as the inverse image of $\xi\in\mathfrak{X}$ is just $\rho^{\xi}\in\mathcal{P}(\mathfrak{L})\cong\mathcal{P}(\mathfrak{K})$. Thus, we know that $\mathcal{P}(\mathfrak{K})\to\mathfrak{X}$, $\pi\mapsto\Xi^{\pi}$ is a continuous bijection. Finally, since $\mathcal{P}(\mathfrak{K})$ is compact and the continuous image of a compact set is compact, the map $\pi\mapsto\Xi^{\pi}$ is open and thus a homeomorphism. ∎

3.7. Proof of Theorem 1.10

For a bipartite graph $G=(U,V,E)$ with $|U|=|V|=n$, and a partition $P=(S_{1}...S_{l},V_{1}...V_{k})$, denote by $G^{P}$ the weighted bipartite graph on vertex set $([l],[k])$ s.t. the weight of edge $ij$ is given by ${\mathrm{d}}(S_{i},V_{j})$.

Theorem 3.17** ([18], Theorem 7.1).**

There exists $\varepsilon>0,n\in\mathbb{N}$ and a bipartite graph $G=(U,V,E)$ with $|U|=|V|=n$ s.t. every partition $P=(S_{1}...S_{l},V_{1}...V_{k})$ of $(U,V)$ satisfying $D_{\blacksquare}(G,G^{P})\leq\varepsilon$ requires at least $l=\exp\left(\Theta(\varepsilon^{-2})\right)$ parts, independently of $k$.

Proof of Theorem 1.10.

Let $G$ be a graph given by the previous theorem and let $\kappa_{G}$ be the corresponding graphon. Denote by $\kappa$ a kernel consisting of $\kappa_{G}$ and its transposed graphon given by (2.6) in the special case $\Omega=\{0,1\}$. Denote by $\mu=\mu(G)\in\mathcal{L}$ the corresponding law given by Theorem 1.4. Assume there is $\nu\in\mathcal{L}$ with support of size less then $l=\exp\left(\Theta(\varepsilon^{-2})\right)$ and $D_{\boxtimes}(\mu,\nu)<\frac{\varepsilon}{2}.$ Then $\nu$ induces a partition $K$ of $[0,1]$ into at most $l$ parts s.t. $D_{\boxtimes}(\kappa,\kappa^{\nu})=D_{\boxtimes}(\kappa,\kappa^{K})\leq\frac{\varepsilon}{2}$ which implies that there is a partition $S$ and a graphon $\kappa^{S}$ s.t. $D_{\blacksquare}(\kappa_{G},\kappa^{S})\leq\varepsilon$. As $\kappa_{G}$ and $\kappa_{S}$ are by definition embeddings of (finite) graphs into the space of graphons, this is a contradiction to Theorem 3.17. ∎

4. The pinning operation

In this section we prove Theorem 1.12. We begin by investigating a discrete version of the pinning operation, which played a key role in recent work on random factor graphs [11]. The discrete version of the pinning theorem, Theorem 4.1 below, was already established as [11, Lemma 3.5]. In Section 4.1 we give a shorter proof, based on an argument form [45]. Moreover, in Section 4.2 we show by a somewhat delicate argument that the pinning operation is continuous with respect to the cut metric. Finally, in Section 4.3 we complete the proof of Theorem 1.12.

4.1. Discrete pinning

For a probability measure $\mu\in\mathcal{L}_{n}$ and a set $I\subset[n]$ we denote by $\mu_{I}$ the joint distribution of the coordinates $i\in I$. Thus, $\mu_{I}$ is the probability distribution on $\Omega^{I}$ defined by

[TABLE]

Where $I=\{i_{1},\ldots,i_{t}\}$ is given explicitly, we use the shorthand $\mu_{I}=\mu_{i_{1},\ldots,i_{t}}$.

Theorem 4.1**.**

For every $\varepsilon>0$ for all large enough $n$ and all $\mu\in\mathcal{L}_{n}$ the following is true. Draw and integer $0\leq\bm{\theta}\leq\lceil\log|\Omega|/\varepsilon^{2}\rceil$ uniformly random and let $\bm{I}\subset[n]$ be a random set of size $\bm{\theta}$. Additionally, draw $\hat{\bm{\sigma}}$ from $\mu$ independently of $\bm{\theta},\bm{I}$. Let

[TABLE]

Then

[TABLE]

Apart from [11, Lemma 3.5], statements related to Theorem 4.1 were previously obtained by Montanari [40] and Raghavendra and Tan [45]. To be precise, [40, Theorem 2.2] deals with the special case of the discrete pinning operation for graphical channels and the number $\bm{\theta}$ of pinned coordinates scales linearly with the dimension $n$. The original proof of Theorem 4.1 in [11] was based on a generalisation of Montanari’s argument. Moreover, [45, Lemma 4.5] asserted the existence of $T=T(\mu,\varepsilon)>0$ such that

[TABLE]

rather than showing that a random $\bm{\theta}$ does the trick. But at second glance the proof given in [45], which is significantly simpler than the one from [11], actually implies Theorem 4.1.

For completeness we include the short proof of Theorem 4.1 via the argument from [45]. We need a few concepts from information theory. Let $X,Y,Z$ be random variables that take values in finite domains. We recall that the conditional mutual information of $X,Y$ given $Z$ is defined as

[TABLE]

with the conventions $0\log 0=0$, $0\log\frac{0}{0}=0$ and with the sum ranging over all possible values $x,y,z$ of $X,Y,Z$, respectively. Moreover, the conditional entropy of $X$ given $Y$ reads

[TABLE]

We also recall the basic identity

[TABLE]

Finally, Pinsker’s inequality provides that for any two probability distribution $\mu,\nu$ on a finite set $\mathcal{X}$,

[TABLE]

signifies the Kullback-Leibler divergence. The proof of the following lemma is essentially identical to the proof of [45, Lemma 4.5].

Lemma 4.2**.**

Let $\mu\in\mathcal{P}(\Omega^{n})$ and let $\bm{\sigma}\in\Omega^{n}$ be a sample drawn from $\mu$. Let $\bm{i},\bm{i}^{\prime},\bm{i}_{1},\ldots\in[n]$ be uniformly distributed and mutually independent as well as independent of $\bm{\sigma}$. Then for any integer $T$ we have

[TABLE]

Proof.

Due to (4.3), for every $\theta\geq 0$,

[TABLE]

Summing on $\theta=1,\ldots,T$, we obtain

[TABLE]

The desired bound follows because $\mathcal{H}(\bm{\sigma}_{i})\leq\log|\Omega|$ and $\mathcal{H}(\bm{\sigma}_{i}\mid\bm{\sigma}_{\bm{i}_{1}},\ldots,\bm{\sigma}_{\bm{i}_{T+1}})\geq 0$. ∎

Now let $T>0$ be an integer and draw $0\leq\bm{\theta}\leq T$ uniformly at random and construct $\hat{\mu}$ as in Theorem 4.1. Then as an immediate consequence of Lemma 4.2 we obtain the following bound, where, of course, the expectation refers to the choice of $\hat{\mu}$ and the independently chosen and uniform $\bm{i},\bm{i}^{\prime}$.

Corollary 4.3**.**

We have $\mathbb{E}\left[{D_{\mathrm{KL}}\left({{{\hat{\mu}_{\bm{i},\bm{i}^{\prime}}}\|{\hat{\mu}_{\bm{i}}\otimes\hat{\mu}_{\bm{i}^{\prime}}}}}\right)}\right]\leq(\log\left|{\Omega}\right|)/T.$

Proof.

Keeping the notation from Lemma 4.2, we let $\bm{I}=(\bm{i},\bm{i}^{\prime},\bm{i}_{1},\ldots,\bm{i}_{\theta})$ and $\bm{\Sigma}=(\bm{\sigma}_{\bm{i}_{1}},\ldots,\bm{\sigma}_{\bm{i}_{\theta}})$. Recalling the definition (4.1) of $\hat{\mu}$, we find

[TABLE]

Hence, the assertion follows from Lemma 4.2. ∎

Proof of Theorem 4.1.

Applying Pinsker’s inequality (4.4), Jensen’s inequality and Corollary 4.3, we find

[TABLE]

whence the desired bound follows if $T\geq(\log\left|{\Omega}\right|)/(2\varepsilon^{2})$. ∎

Finally, the following lemma clarifies the bearing that the bound (4.2) has on the cut metric. The lemma is an improved version of [13, Lemma 2.9]. Following [5] we say that $\mu\in\mathcal{L}_{n}$ is $\varepsilon$-symmetric if

[TABLE]

Lemma 4.4**.**

For any $\varepsilon>0$ and every finite set $\Omega$ there exists $n_{0}>0$ s.t. for every $n\geq n_{0}$ every $\varepsilon^{2}/4$-symmetric $\mu\in\mathcal{L}_{n}$ satisfies $\Delta_{\boxslash}(\mu,\bigotimes_{i=1}^{n}\mu_{i})<\varepsilon$.

Proof.

Let $\delta=\varepsilon^{2}/4$. Since $\mu\otimes\bar{\mu}$ is a coupling of $\mu$ and $\bar{\mu}$ it suffices to show that for any set $I\subset[n]$ and every $\omega\in\Omega$,

[TABLE]

Let $X(\sigma)=X(\sigma,I,\omega)=\sum_{i\in I}\mathbf{1}\left\{{\sigma_{i}=\omega}\right\}$ and denote by $\bar{X}$ its expectation with respect to $\mu$, that is $\bar{X}=\langle X(\sigma),\mu\rangle$. Because $\mu$ is $\delta$-symmetric we can bound the second moment of $X$ as follows:

[TABLE]

Hence,

[TABLE]

Let $\alpha\in(0,1)$ and $P(\alpha)=\mu\left\{{\left|{X(\sigma)-\bar{X}}\right|\geq\alpha n}\right\}$. Then Chebyshev’s inequality and (4.6) yield $P(\alpha)\leq(|I|+\delta n^{2})/(\alpha n)^{2}.$ Hence, for events $S_{h}=\left\{\left|{X(\sigma)-\bar{X}}\right|\geq 2^{h}\varepsilon n\right\}$ we obtain $\mu(S_{h})\leq P(2^{h}\varepsilon)\leq 4^{-h}\delta/\varepsilon^{2}+O(1/n).$ Therefore,

[TABLE]

Thus, (4.5) follows from (4.7) and the choice of $\delta$. ∎

4.2. Continuity

Recall that for a given $\mu\in\mathfrak{L}$ the pinned $\mu_{\hat{\bm{\sigma}}^{\mu}\downarrow n}\in\mathfrak{L}$ is random. Thus, for the pinned laws we consider the $D_{\boxtimes}$-Wasserstein metric. The aim in this paragraph is to establish the following key statement.

Proposition 4.5**.**

The operator $\mu\mapsto\mu_{\hat{\bm{\sigma}}^{\mu}\downarrow n}$ is $(D_{\boxtimes}(\,\cdot\,,\,\cdot\,),\mathcal{D}_{\boxtimes}(\,\cdot\,,\,\cdot\,))$-continuous for any $n\geq 1$.

Toward the proof of Proposition 4.5 we need to consider a slightly generalised version of the pinning operation. Specifically, for a measurable map $\kappa:[0,1]^{2}\to[0,1]^{\Omega}$ and $\tau\in\Omega^{n}$ let

[TABLE]

Thus, $\bm{z}_{\tau}$ is a random variable, dependent on the uniformly and independently chosen $\hat{\bm{x}}_{1},\ldots,\hat{\bm{x}}_{n}\in[0,1]$. Also let $\bm{z}(\kappa)=\sum_{\tau\in\Omega^{n}}\bm{z}_{\tau}\left({\kappa}\right)$. Further, define $\kappa_{\tau\downarrow n}\in\mathcal{K}_{1}$ as follows. If $\bm{z}_{\tau}(\kappa)=0$, then we let $\kappa_{\tau\downarrow n}=\kappa$. But if $\bm{z}_{\tau}(\kappa)>0$, then we let $\kappa_{\tau\downarrow n}$ be a kernel representation of the probability distribution

[TABLE]

Additionally, let $\hat{\bm{\sigma}}^{\kappa}\in\Omega^{n}$ denote a vector drawn from the distribution $(\bm{z}_{\tau}(\kappa)/\bm{z}(\kappa))_{\tau\in\Omega^{n}}$ if $\bm{z}(\kappa)>0$, and let $\hat{\bm{\sigma}}^{\kappa}\in\Omega^{n}$ be uniformly distributed otherwise.

Lemma 4.6**.**

For any $n\geq 1$, $\varepsilon>0$ there is $\delta>0$ such that for all $\kappa\in\mathcal{K}{}$ and all $\kappa^{\prime}\in\mathcal{K}_{1}$ with $D_{1}(\kappa,\kappa^{\prime})<\delta$ we have

[TABLE]

Toward the proof of Lemma 4.6 we require the following statement.

Lemma 4.7**.**

For any $n\geq 1$, $\varepsilon>0$ and $\kappa\in\mathcal{K}{}$ we have $\mathbb{P}\left[{\bm{z}_{\hat{\bm{\sigma}}^{\kappa}}(\kappa)<\varepsilon\mid\hat{\bm{x}}_{1},\ldots,\hat{\bm{x}}_{n}}\right]<\varepsilon\left|{\Omega}\right|^{n}.$

Proof.

We have $\mathbb{P}\left[{\bm{z}_{\hat{\bm{\sigma}}}(\kappa)<\varepsilon\mid\hat{\bm{x}}_{1},\ldots,\hat{\bm{x}}_{n}}\right]=\sum_{\tau\in\Omega^{n}}\bm{1}\left\{{\bm{z}_{\tau}(\kappa)<\varepsilon}\right\}\bm{z}_{\tau}(\kappa)<\varepsilon|\Omega|^{n}.$ ∎

Proof of Lemma 4.6.

Given $\varepsilon>0$ pick small enough $\eta=\eta(\varepsilon,n)>0$, $\delta=\delta(\eta)>0$. Consider $\kappa\in\mathcal{K},\kappa^{\prime}\in\mathcal{K}_{1}$ such that $D_{1}(\kappa,\kappa^{\prime})<\delta$ and let $\mu=\mu^{\kappa}$, $\mu^{\prime}=\mu^{\kappa^{\prime}}$. Then we see that

[TABLE]

Hence, in the following we may condition on the event that $1-\eta<\bm{z}(\kappa^{\prime})<1+\eta$. Given that this is so, choose $\hat{\bm{\sigma}},\hat{\bm{\sigma}}^{\prime}\in\Omega^{n}$ from the distributions

[TABLE]

Further, define the probability density functions

[TABLE]

so that

[TABLE]

To couple $\mu_{\hat{\bm{\sigma}}\downarrow n},\mu_{\hat{\bm{\sigma}}^{\prime}\downarrow n}^{\prime}$ draw a pair $(\bm{t},\bm{t}^{\prime})\in[0,1]^{2}$ from the following distribution: with probability $\int_{0}^{1}\hat{p}(s){\mathrm{d}}s$, we draw $\bm{t}=\bm{t}^{\prime}$ from the distribution $(\int_{0}^{1}\hat{p}(s){\mathrm{d}}s)^{-1}\hat{p}(s){\mathrm{d}}s$, and with probability $1-\int_{0}^{1}\hat{p}(s){\mathrm{d}}s$ we draw $\bm{t},\bm{t}^{\prime}$ independently from the distributions

[TABLE]

respectively. Then $(\kappa_{\bm{t}},\kappa_{\bm{t}^{\prime}}^{\prime})$ provides a coupling of $\mu_{\hat{\bm{\sigma}}\downarrow n},\mu^{\prime}_{\hat{\bm{\sigma}}^{\prime}\downarrow n}$. Consequently,

[TABLE]

To estimate $\mathbb{P}\left[{\bm{t}\neq\bm{t}^{\prime}}\right]$ let

[TABLE]

Picking $\delta$ sufficiently small ensures that

[TABLE]

and on the event ${\mathcal{E}}$ we have

[TABLE]

Hence, on ${\mathcal{E}}$ we can couple $\hat{\bm{\sigma}},\hat{\bm{\sigma}}^{\prime}$ such that

[TABLE]

Additionally, let ${\mathcal{E}}^{\prime}=\left\{{\hat{\bm{\sigma}}=\hat{\bm{\sigma}}^{\prime},\,\bm{z}_{\hat{\bm{\sigma}}}(\kappa)\geq\eta^{1/3}}\right\}$. Then Lemma 4.7, (4.10) and (4.11) imply that

[TABLE]

Moreover, on ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}$ we have

[TABLE]

and consequently

[TABLE]

Finally, the assertion follows from (4.9), (4.10), (4.12) and (4.13). ∎

Lemma 4.8**.**

For any $\varepsilon>0$, $\ell\geq 1$ there is $\delta>0$ such that for all $\kappa\in\mathcal{K}{}$ such that $\mu^{\kappa}\in\mathcal{L}$ is supported on a set of size at most $\ell$ and all $\iota\in\mathcal{K}_{1}$ with $D_{\oblong}(\kappa,\iota)<\delta$ we have $\mathcal{D}_{\boxtimes}(\kappa_{\hat{\bm{\sigma}}^{\kappa}\downarrow n},\iota_{\hat{\bm{\sigma}}^{\iota}\downarrow n})<\varepsilon$.

Proof.

Pick $\alpha=\alpha(\varepsilon,\ell,n)$, $\beta=\beta(\alpha)$, $\xi=\xi(\beta)$, $\zeta=\zeta(\xi)$, $\eta=\eta(\zeta)>0$ and $\delta=\delta(\eta)>0$ sufficiently small. To summarise,

[TABLE]

We may assume that there is a partition $S_{1},\ldots,S_{\ell}$ of $[0,1]$ such that $\kappa$ is constant on $S_{i}\times\left\{{x}\right\}$ for all $x\in\left[{0,1}\right]$. Moreover, we may assume without loss that there is $k\in[\ell]$ such that $\lambda(S_{i})>\eta$ for all $i\leq k$, while $\lambda(S_{i})<\eta$ for all $i>k$. Let $t_{i}:[0,1]\to S_{i}$ be a measurable bijection that maps the Lebesgue measure on $[0,1]$ to the probability measure $\lambda(S_{i})^{-1}{\mathrm{d}}s$ on $S_{i}$ for $i\leq k$ (see Lemma 2.3). Assuming that $\delta$ is small enough, we see that the kernels

[TABLE]

have cut distance

[TABLE]

Combining Proposition 3.8 and (4.15), we conclude that after an $n$-fold application of the $\oplus^{\prime}$-operation we have $D_{\oblong}(\kappa^{(i)\oplus^{\prime}n},\iota^{(i)\oplus^{\prime}n})<\xi$. Since for every $x\in[0,1]$ the map $s\mapsto\kappa^{(i)\oplus^{\prime}n}_{s,x}$ is constant, we therefore find that

[TABLE]

Because $\lambda(S_{i})<\eta$ for all $i>k$ and $\ell\eta<\beta$ for small enough $\eta$, (4.16) implies that

[TABLE]

Combining (4.17) with Markov’s inequality, we conclude that

[TABLE]

Consequently, on ${\mathcal{E}}$ we have

[TABLE]

In particular, there exists a coupling of the reference configurations $\hat{\bm{\sigma}}^{\kappa},\hat{\bm{\sigma}}^{\iota}\in\Omega^{n}$ such that $\mathbb{P}\left[{\hat{\bm{\sigma}}^{\kappa}=\hat{\bm{\sigma}}^{\iota}}\right]\geq 1-\beta^{1/4}$. Hence, Lemma 4.7 implies that the event ${\mathcal{E}}^{\prime}=\left\{{\hat{\bm{\sigma}}^{\kappa}=\hat{\bm{\sigma}}^{\iota},\,\bm{z}_{\kappa}(\hat{\bm{\sigma}}^{\kappa})\geq\alpha}\right\}$ satisfies

[TABLE]

To complete the proof let

[TABLE]

Further, let ${\mathcal{E}}^{\prime\prime}=\left\{{\sum_{i=1}^{\ell}\left|{\hat{p}_{\kappa,i}-\hat{p}_{\iota,i}}\right|<\alpha^{3}}\right\}$. Then (4.16), (4.18) and (4.20) imply that

[TABLE]

Moreover, since $\bm{z}_{\kappa}(\hat{\bm{\sigma}}^{\kappa})\geq\alpha$ and $\hat{\bm{\sigma}}^{\kappa}=\hat{\bm{\sigma}}^{\iota}$, on ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}$ we have $\bm{z}_{\iota}(\hat{\bm{\sigma}}^{\iota})\geq\alpha/2$. Therefore, on ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}$ the probability distributions $(p_{\kappa,i})_{i\in[\ell]},(p_{\iota,i})_{i\in[\ell]}$ with

[TABLE]

have total variation distance $d_{\mathrm{TV}}((p_{\kappa,i})_{i\in[\ell]},(p_{\iota,i})_{i\in[\ell]})<2\alpha$. Consequently, there exists a coupling of random variables $\bm{i}_{\kappa},\bm{i}_{\iota}$ with these distributions such that

[TABLE]

We extend this coupling to a coupling $\gamma$ of $\mu_{\hat{\bm{\sigma}}^{\mu}\downarrow n},\nu_{\hat{\bm{\sigma}}^{\nu}\downarrow n}$: given $\bm{i}_{\kappa},\bm{i}_{\iota}$, pick any $\bm{s}_{\kappa}\in S_{\bm{i}_{\kappa}}$ and choose $\bm{s}_{\iota}\in S_{\bm{i}_{\iota}}$ from the distribution $p_{\iota}(s)/\hat{p}_{\iota,\bm{i}_{\iota}}{\mathrm{d}}s$. Then $\kappa_{\bm{s}_{\kappa}}$, $\iota_{\bm{s}_{\iota}}$ have distribution $\mu_{\hat{\bm{\sigma}}^{\mu}\downarrow n},\nu_{\hat{\bm{\sigma}}^{\nu}\downarrow n}$, respectively. Further, we claim that on ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}$,

[TABLE]

Indeed, thanks to (4.22), we may condition on the event $\bm{i}_{\iota}=\bm{i}_{\kappa}\leq k$. Hence, to prove (4.23) it suffices to show that for any $S\subset S_{\bm{i}_{\iota}}$, $X\subset[0,1]$, $\omega\in\Omega$,

[TABLE]

Because $\bm{z}_{\iota}(\hat{\bm{\sigma}}^{\kappa})\geq\alpha/2$ we may also assume that $\hat{p}_{\iota,\bm{i}_{\iota}}\geq\alpha^{2}/\ell$, and we observe that $p_{\iota}(s)\leq 1$. Now, assume for contradiction that there exist $S,X,\omega$ for which (4.24) is violated. Letting

[TABLE]

we conclude that

[TABLE]

But (4.25) contradicts the choice of the parameters from (4.14). Hence, we obtain (4.24) and thus (4.23). Finally, the assertion follows from (4.18), (4.20), (4.21) and (4.23). ∎

Lemma 4.9**.**

For every sequence $(k_{i})_{i}$ in $\mathcal{K}_{1}$ that converges to a kernel $k\in\mathcal{K}_{1}$ with respect to $D_{\oblong}(\cdot,\cdot)$ and for every kernel $k^{\prime}\in\mathcal{K}_{1}$ there is a sequence of kernels $(k^{\prime}_{i})_{i}$, $k_{i}^{\prime}\in\mathcal{K}_{1}$, s.t. $D_{\oblong}(k^{\prime}_{i},k^{\prime})\to 0$ and $D_{1}(k_{i},k_{i}^{\prime})\to D_{1}(k,k^{\prime})$.

Proof.

Let $(\kappa^{\omega})_{\omega},(\kappa^{\prime\omega})_{\omega},(\kappa_{i}^{\omega})_{\omega},(\kappa^{\prime\omega}_{i})_{\omega}$ be the families of bipartite graphons representing $k,k^{\prime},(k_{i})_{i},(k^{\prime}_{i})_{i}$ given by (2.6). From the definition of $D_{1}(\cdot,\cdot)$ and Lemma 2.5 we get

[TABLE]

The lemma follows from (4.26) and [32, Proposition 8.25]. ∎

The following lemma and the proof of Proposition 4.5 are adaptations of [32, Proof of Lemma 9.16].

Lemma 4.10**.**

Let $\varepsilon,\delta>0$ and let $k\in\mathcal{K}{}$. Let $U_{k}(\delta,\varepsilon)$ be the set of all $\kappa\in\mathcal{K}$ such that there exists $\kappa^{\prime}\in\mathcal{K}_{1}$ with $D_{\oblong}(k,\kappa^{\prime})<\delta$ and $D_{1}(\kappa^{\prime},\kappa)<\varepsilon$. Then $U_{k}(\delta,\varepsilon)$ is $D_{\oblong}$-open.

Proof.

Suppose that $\kappa\in U_{k}(\delta,\varepsilon)$ and that the sequence $(\kappa_{i})_{i\geq 1}$ satisfies $\lim_{i\to\infty}D_{\oblong}(\kappa,\kappa_{i})=0$. It suffices to show that $\kappa_{i}\in U_{k}(\delta,\varepsilon)$ for all large enough $i$. To this end consider $\kappa^{\prime}\in\mathcal{K}_{1}$ such that $D_{\oblong}(k,\kappa^{\prime})<\delta$ and $D_{1}(\kappa^{\prime},\kappa)<\varepsilon$. By Lemma 4.9 there exists a sequence $\kappa_{i}^{\prime}\in\mathcal{K}_{1}$ such that $\lim_{i\to\infty}D_{\oblong}(\kappa^{\prime},\kappa_{i}^{\prime})=0$ and $\lim_{i\to\infty}D_{1}(\kappa_{i}^{\prime},\kappa_{i})=D_{1}(\kappa,\kappa^{\prime})$. For this sequence we have

[TABLE]

Therefore, for large enough $i$ we have $D_{1}(\kappa_{i},\kappa_{i}^{\prime})<\varepsilon$ and $D_{\oblong}(k,\kappa_{i}^{\prime})\leq D_{\oblong}(k,\kappa^{\prime})+D_{\oblong}(\kappa_{i}^{\prime},\kappa^{\prime})<\delta$, whence $\kappa_{i}^{\prime}\in U_{k}(\delta,\varepsilon)$. ∎

Proof of Proposition 4.5.

Fix $\varepsilon>0$. Lemma 4.6 shows that there exists $\delta_{0}>0$ such that for all $\kappa\in\mathcal{K},\kappa^{\prime}\in\mathcal{K}_{1}$,

[TABLE]

Similarly, by Lemma 4.8 there exists a sequence $(\delta_{\ell})_{\ell}$ such that for all $\mu,\nu\in\mathcal{L}$ with $\mu$ supported on at most $\ell\geq 1$ configurations we have

[TABLE]

Suppose that $k:[0,1]^{2}\to\mathcal{P}(\Omega)$ is a step function that takes $\ell\geq 1$ different values and let $\mathcal{U}_{k}=\mathcal{U}_{k}(\delta_{\ell},\delta_{0})$ be as in Lemma 4.10. Then $\mathcal{U}_{k}$ is $D_{\oblong}$-open and $\bigcup_{k}\mathcal{U}_{k}=\mathcal{K}$ because $\mathcal{U}_{k}$ contains the $\delta_{0}$-ball around $k$ with respect to the $D_{1}$-metric. Further, let $\mathfrak{U}_{k}\subset\mathfrak{K}$ be the projection of $\mathcal{U}_{k}$ onto $\mathfrak{K}{}$. Then $\mathfrak{U}_{k}$ is open because the canonical map $\mathcal{K}\to\mathfrak{K}$ is open. Moreover, $\bigcup_{k}\mathfrak{U}_{k}=\mathfrak{K}$. Hence, a finite number of sets $\mathfrak{U}_{k}$ cover $\mathfrak{K}$. Thus, the assertion follows from (4.27) and (4.28). ∎

4.3. Proof of Theorem 1.12

Let $\varepsilon>0$ and pick a small enough $\delta>0$ and then a large enough $N>0$. Also let $T=T(\varepsilon)=64\varepsilon^{-8}\log|\Omega|$. Given $\mu\in\mathfrak{L}$ we apply Theorem 1.9 to obtain a probability distribution $\nu\in\mathcal{L}_{N}$ such that $D_{\boxtimes}(\mu,\dot{\nu})<\delta$. Invoking Theorem 1.1 and Proposition 4.5, we find

[TABLE]

By construction, for any $n$ the law $\dot{\nu}_{\hat{\bm{\sigma}}^{\dot{\nu}}\downarrow n}$ obtained by first embedding $\nu\in\mathcal{L}_{N}$ into $\mathcal{L}$ and then applying the pinning operation coincides with the law obtained by first applying (4.1) to $\nu$ and then embedding the resulting $\hat{\nu}$ into $\mathfrak{L}$. Hence, Theorem 4.1 and Lemma 4.4 show that for a uniform $\bm{\theta}\leq T(\varepsilon)$,

[TABLE]

Further, Theorem 1.2, Theorem 1.11 and (4.29) show that

[TABLE]

Combining (4.30) and (4.31) and applying Markov’s inequality, we obtain the first part of Theorem 1.12. The second assertion follows from a similar argument.

4.4. Proof of Theorem 1.2

We postponed the proof Theorem 1.2, because it relies on some of the prior results from this section. To finally carry the proof out we adapt the proof strategy from [32], where a statement similar to Theorem 1.2 was established for graphons, to the present setting of probability distributions. We begin with the following simple bound.

Lemma 4.11**.**

For any $\mu,\nu\in\mathcal{L}_{n}$ we have $\Delta_{\boxtimes}(\mu,\nu)\leq n^{3}D_{\boxtimes}(\dot{\mu},\dot{\nu}).$

Proof.

Let $\psi\in\SS$ and let $\gamma\in\Gamma(\dot{\mu},\dot{\nu})$. We are going to show that there exist a coupling $g\in\Gamma(\mu,\nu)$ and a permutation $\phi\in\SS_{n}$ such that

[TABLE]

The assertion is immediate from (4.32) and the definitions (1.1), (1.2).

With respect to the coupling $g$, matters are easy: the construction of $\dot{\mu},\dot{\mu}^{\prime}\in\mathcal{L}$ ensures that the coupling $\gamma$ readily induces a coupling $g$ of the original probability distributions $\mu,\nu$ such that $g(\sigma,\tau)=\gamma(\dot{\sigma},\dot{\tau})$ for all $\sigma,\tau\in\Omega^{n}$.

We are left to exhibit the permutation $\phi$. To this end let $I_{j}=[(j-1)/n,j/n)$. We construct a bipartite auxiliary graph $\mathcal{G}$ with vertex set $\{v_{1},\ldots,v_{n}\}\cup\{w_{1},\ldots,w_{n}\}$ in which $v_{i},w_{j}$ are adjacent iff $\lambda(I_{j}\cap\psi(I_{i}))\geq n^{-3}$. Then the Hall’s theorem implies that $\mathcal{G}$ possesses a perfect matching. Indeed, assume that $\emptyset\neq V\subset\{v_{1},\ldots,v_{n}\}$ satisfies $|\partial V|<|V|$. Then because $\psi$ preserves the Lebesgue measure we obtain

[TABLE]

a contradiction. Thus, let $\phi$ be the permutation of $[n]$ induced by any perfect matching of $\mathcal{G}$.

To complete the proof we claim that $g,\phi$ satisfy (4.32). Indeed, given a set $S\subset\Omega^{n}\times\Omega^{n}$ let $\dot{S}=\{(\dot{\sigma},\dot{\tau}):(\sigma,\tau)\in S\}$. Further, for $X\subset[n]$ let $\dot{X}\subset[0,1]$ be any measurable set such that $\lambda(\dot{X}\cap I_{i})=n^{-3}$ for all $i\in[n]$ and $\dot{X}\cap I_{j}\subset\psi(I_{i})$ if $\phi(i)=j$. Then we obtain (4.32). ∎

As a second step we will complement the coarse multiplicative bound from Lemma 4.11 with a somewhat more subtle additive bound. To this end, we need an enhanced version of a ’Frieze-Kannan type’ regularity lemma for probability distributions. Specifically, let $\mu\in\mathcal{L}_{n}$ and let $S=\{S_{1},\ldots,S_{k}\}$ and $X=\{X_{1},\ldots,X_{\ell}\}$ be partitions of $\Omega^{n}$ and $[n]$, respectively. We call the partition $S$ canonical if there exists a set $\mathcal{I}\subset[n]$ such that

[TABLE]

In words, $S$ partitions the discrete cube $\Omega^{n}$ into the $\Omega^{|\mathcal{I}|}$ sub-cubes defined by the entries on the set $\mathcal{I}$ of coordinates. In this case we define

[TABLE]

Thus, $\mu^{S,X}$ is a mixture of product measures, one for each class of the partition $S$.

Lemma 4.12**.**

For any $\Omega$ there exists $c=c(\Omega)>0$ such that for every $0<\varepsilon<1/2$, $n>0$ and all $\mu,\nu\in\mathcal{L}_{n}$ there exist a canonical partition $S_{1},\ldots,S_{k}$ of $\Omega^{n}$ and a partition $X_{1},\ldots,X_{\ell}$ of $[n]$ such that the following statements are satisfied.

• •

$k+\ell\leq\exp(\varepsilon^{-c})$.

• •

with $\gamma\in\Gamma(\mu,\mu^{S,X})$ and $\gamma^{\prime}\in\Gamma(\nu,\nu^{S,X})$ defined by

[TABLE]

we have

[TABLE]

Hence, $\Delta_{\boxtimes}(\mu,\mu^{S,X})<\varepsilon$, $\Delta_{\boxtimes}(\nu,\nu^{S,X})<\varepsilon$.

Proof.

Combining Theorem 4.1 and Lemma 4.4, we find a set $\mathcal{I}\subset[n]$ such that the induced canonical partition $S_{1},\ldots,S_{k}$ satisfies

[TABLE]

Moreover, the size $k$ of the partition is bounded by $\exp(\varepsilon^{-c^{\prime}})$ for some $c^{\prime}=c^{\prime}(\Omega)$. Now, for each $i\in[k]$ we can partition the set $[n]$ into at most $32/\varepsilon$ classes $X_{i,1},\ldots,X_{i,\ell_{i}}$ such that for all $x,y\in X_{i,j}$ we have $d_{\mathrm{TV}}(\mu_{x}[\,\cdot\,|S_{i}],\mu_{y}[\,\cdot\,|S_{i}])<\varepsilon/16$. A similar partition $X_{i,1}^{\prime},\ldots,X^{\prime}_{i,\ell_{i}^{\prime}}$ exists for $\nu[\,\cdot\,|S_{i}]$. Hence, the smallest common refinement $X_{1},\ldots,X_{\ell}$ of all these partitions $(X_{i,j}),(X_{i,j}^{\prime})$ has at most $\exp(\varepsilon^{-c})/2$ classes, for some suitable $c=c(\Omega)>0$. Further, by construction, letting

[TABLE]

we obtain from (4.35) that

[TABLE]

In addition, since $\mu^{(i)},\nu^{(i)}$ are product measures, the couplings $\gamma^{(i)},\gamma^{(i)\prime}$ for which the cut distance in (4.36) attained are trivial, i.e., $\gamma^{(i)}=\mu[\,\cdot\,|S_{i}]\otimes\mu^{(i)}$ and $\gamma^{(i)\prime}=\nu[\,\cdot\,|S_{i}]\otimes\nu^{(i)}$. Therefore, (4.36) implies (4.33)–(4.34). ∎

Lemma 4.13**.**

For any $\mu,\nu\in\mathcal{L}_{n}$ we have $\Delta_{\boxtimes}(\mu,\nu)\leq D_{\boxtimes}(\dot{\mu},\dot{\nu})+o(1)$ as $n\to\infty$.

Proof.

Let $0<\varepsilon=\varepsilon(n)=o(1)$ be a sequence that tends to zero sufficiently slowly. By Corollary 4.12 there exist partitions $S_{1},\ldots,S_{k}$ of $\Omega^{n}$ and $X_{1},\ldots,X_{\ell}$ of $[n]$ such that $\Delta_{\boxtimes}(\mu,\mu^{S,X})+\Delta_{\boxtimes}(\nu,\nu^{S,X})<\varepsilon$ and $k+\ell\leq\exp(\varepsilon^{-c})$. By the triangle inequality,

[TABLE]

Hence, there exist $\phi\in\SS$ and a coupling $g$ of $\mu^{S,X},\nu^{S,X}$ such that the induced coupling $\dot{g}$ of $\dot{\mu}^{S,X},\dot{\nu}^{S,X}$ satisfies

[TABLE]

Because $\phi$ preserves the Lebesgue measure, there exists a bijection $\varphi:[n]\to[n]$ such that the following is true. For a class $X_{i}\subset[n]$ let $\dot{X}_{i}=\bigcup_{x\in X_{i}}[(x-1)/n,x/n)$. Then uniformly for all $h,i\in[\ell]$ we have

[TABLE]

Further, we construct a coupling $G\in\Gamma(\mu,\nu)$ by letting

[TABLE]

and we claim that

[TABLE]

Clearly, (4.39) readily implies the assertion.

To verify (4.39) we observe that, due to symmetry and the triangle inequality, it suffices to show that

[TABLE]

for all $T,Y,\omega$. Now, invoking Lemma 4.12, we obtain

[TABLE]

As the same bound holds for the negative part $\left({\sigma_{Y}(\omega)-\sigma^{\prime}_{Y}(\omega)}\right)_{-}$, we obtain (4.40). Similarly, due to Corollary 4.12, (4.37) and (4.38),

[TABLE]

whence (4.41) follows. ∎

Proof of Theorem 1.2.

The theorem follows by combining Lemmas 4.11 and 4.13. ∎

Acknowledgment.

We thank Viresh Patel for bringing [49] to our attention, an anonymous reviewer for their careful reading, which has led to numerous corrections, and a second anonymous reviewer for pointing out several further references.