Matchability of heterogeneous networks pairs

TL;DR

This paper investigates the conditions under which heterogeneous networks can be accurately matched, showing that centering techniques like Universal Singular Value Thresholding significantly improve matchability in complex network models.

Contribution

It introduces a theoretical framework for understanding graph matchability in heterogeneous networks and demonstrates the effectiveness of centering methods like USVT for improving matching accuracy.

Findings

Direct matching often fails in heterogeneous networks

Centering with USVT restores matchability under mild conditions

Results validated on real and synthetic data

Abstract

We consider the problem of graph matchability in non-identically distributed networks. In a general class of edge-independent networks, we demonstrate that graph matchability can be lost with high probability when matching the networks directly. We further demonstrate that under mild model assumptions, matchability is almost perfectly recovered by centering the networks using Universal Singular Value Thresholding before matching. These theoretical results are then demonstrated in both real and synthetic simulation settings. We also recover analogous core-matchability results in a very general core-junk network model, wherein some vertices do not correspond between the graph pair.

Equations

$$\Vert A{\Vert }_F=\sqrt{\sum _{i=1}^n\sum _{j=1}^{n}A(i,j{)}^2},$$

$$\Vert A\Vert ={\sigma }_1(A),$$

$$\text{tr}(A)=\sum _{i=1}^{n}A(i,i).$$

$$A\oplus B=\left(\begin{array}{cc}A& [0{]}_{n,m}\\ [0{]}_{m,n}& B\end{array}\right),$$

$$f(n)=o(g(n))$$

$$f(n)=O(g(n))$$

$$f(n)=\omega (g(n))$$

$$f(n)=\Omega (g(n))$$

$$f(n)=\Theta (g(n))$$

$${\text{argmin}}_{P\in \Pi (n)}\Vert A-PB{P}^T{\Vert }_F^2={\text{argmin}}_{P\in \Pi (n)}-\mathrm{tr}(APB{P}^T),$$

$$\Vert A-PB{P}^T{\Vert }_F^2$$

$$=\text{tr}(A^{T}A)-2\text{tr}(A^{T}PB{P}^T)+\text{tr}\left((PB{P}^T{)}^T(PB{P}^T)\right)$$

$$=\text{tr}(A^{T}A)-2\text{tr}(APB{P}^T)+\text{tr}\left(B^{T}B\right)$$

$$\text{mathds}{1}_{\{u,v\}\in E(A)}\sim \text{Bernoulli}(Q_1(u,v))$$

$$\text{mathds}{1}_{\{u,v\}\in E(B)}\sim \text{Bernoulli}(Q_2(u,v))$$

$$R(u,v)=\frac{\text{Cov}(A(u,v),B(u,v))}{\sqrt{\text{Var}(A(u,v))\text{Var}(B(u,v))}}\ge 0$$

$${\text{argmin}}_{P\in \Pi (n)}\delta (A,PB{P}^T)=\{I_n\}?$$

$${\text{argmin}}_{P\in \Pi (n)}\delta (A,PB{P}^T)=\{I_n\},$$

$$\delta (A,PB{P}^T)=\Vert A-PB{P}^T{\Vert }_F.$$

$$\delta (A,PB{P}^T)=\Vert f(A)-f(PB{P}^T){\Vert }_F,$$

$$f(\cdot )$$

$${\delta }^{\text{GMP}}(A,PB{P}^T)$$

$$\hat{\delta }(A,PB{P}^T)$$

$${\text{argmin}}_{P\in \Pi (n)}\Vert A-\mathbb{E}(A)-P(B-\mathbb{E}(B))P^T{\Vert }_F=\{I_n\}.$$

$$Q_1=\left(\begin{array}{cc}p{J}_n& r{J}_n\\ r{J}_n& q{J}_n\end{array}\right)\text{ and }{Q}_2=\left(\begin{array}{cc}q{J}_n& r{J}_n\\ r{J}_n& p{J}_n\end{array}\right).$$

$$R=\left(\begin{array}{cc}{\varrho }_1{J}_n& {\varrho }_2{J}_n\\ {\varrho }_2{J}_n& {\varrho }_1{J}_n\end{array}\right).$$

$$\left\{\{u,v\}\in \left(\genfrac{}{}{0ex}{}{V_1}{2}\right)\right\}$$

$$\left\{\{u,v\}\in \left(\genfrac{}{}{0ex}{}{V_2}{2}\right)\right\}$$

$$\frac{1}{2}\mathbb{E}\text{tr}(APB{P}^T)$$

$$\frac{1}{2}\mathbb{E}\text{tr}(AB)$$

$$\frac{1}{2}\mathbb{E}[\text{tr}(APB{P}^T)-\text{tr}(AB)]\ge$$

$$-\left(\genfrac{}{}{0ex}{}{n}{2}\right)(2{\varrho }_1\sqrt{p(1-p)q(1-q)}+2pq)-n^{2}r(r+{\varrho }_2(1-r))$$

$$=$$

$$\{I_{2n}\}\notin {\text{argmax}}_{P\in \Pi (n)}\text{tr}(APB{P}^T)={\text{argmin}}_{P\in \Pi (n)}\Vert AP-PB{\Vert }_F;$$

$$\displaystyle\big{[}\mathcal{K}_{A,B.P}\big{]}(u,v)=\text{Cov}\left[A(u,v),(PBP^{T})(u,v)\right].$$

Full text

1 Introduction and Background

The graph matching problem seeks to find an alignment between the vertex sets of two graphs that best preserves common structure across graphs. At its simplest, it can be formulated as follows: Given two adjacency matrices $A$ and $B$ corresponding to $n$-vertex graphs, the graph matching problem seeks to minimize $\|A-PBP^{T}\|_{F}$ over permutation matrices $P$; i.e., the graph matching problem seeks a relabeling of the vertices of $B$ that minimizes the number of induced edge disagreements between $A$ and $PBP^{T}$. Variants and extensions of this problem have been extensively studied in the literature, with applications across areas as diverse as biology and neuroscience [58, 31, 10, 19], computer vision [47, 21, 36], pattern recognition [34, 48, 62], and social network analysis [35, 59, 26], among others. For a survey of many of the recent applications and approaches to the graph matching problem, see the sequence of survey papers [13, 24, 20]. While recent results [3] have whittled away at the complexity of the related graph isomorphism problem—determining whether a permutation matrix $P$ exists satisfying $A=PBP^{T}$— at its most general, where $A$ and $B$ are allowed to be weighted and directed, the graph matching problem is known to be NP-hard. Indeed, in this case, the graph matching problem is equivalent to the notoriously difficult quadratic assignment problem [37, 8, 7]. However, recent approaches that leverage efficient representation/learning methodologies (see, for example, [5, 59, 26]) have shown excellent empirical performance matching networks with up to millions of nodes.

In addition to algorithmic advancements in graph matching, there has been a flurry of activity studying the closely related problem of graph matchability: Given a latent alignment between the vertex sets of two graphs, can graph matching uncover this alignment in the presence of shuffled vertex labels? This problem arises in a variety of contexts, from network de-anonymization and privatization to multi-network hypothesis testing [38] to multimodality graph embedding methodologies [11]. Many existing results are concerned with recovering a latent alignment present across random graph models where each of $A$ and $B$ have identical marginal distributions, and exciting advancements on the threshold of matchable versus unmatchable graphs have been made across many random graph settings, including: the homogeneous correlated Erdős-Renyi model (see, for example, [46, 40, 4, 18]), the correlated stochastic blockmodel setting (see, for example, [45, 38]), the $\rho$-correlated heterogeneous Erdős-Renyi model (see, for example, [39, 41]), and in the correlated heterogeneous Erdős-Renyi model with varying edge correlations (see, for example, [49, 42]). In the non-identically distributed model setting, the work in [14, 15, 16] provide theoretic phase transitions on matchability in the $(A,B)\sim$Erdős-Rényi(p,q,$\varrho$) model (i.e., $A\sim$Erdős-Rényi(n,p), $B\sim$Erdős-Rényi(n,q) and the edge correlation across graphs in provided by the constant $\varrho$; see Definition 2).

The above results range from providing theoretic phase transitions on matchability [14, 15, 38] to providing nearly efficient methods for achieving matchability from an algorithmic perspective [18, 4, 16, 22]. While they have served to establish a novel theoretical understanding of the matchability problem, in each case the transition from matchable to unmatchable graphs is defined in terms of decreasing across-graph correlation and within-graph sparsity. Importantly, it is not a function of fundamentally different probabilistic structures across the graphs to be matched. As we often witness in applications, the graph topologies can differ significantly even amongst vertices that correspond to the same entity across networks. Social networks offer a compelling example of this, where matching across different social network platforms requires the understanding that not all users will be behaving homogeneously across different network platforms [35]. Both theoretically (see Example 5) and practically (see, for example, [10]), this distributional heterogeneity can have a deleterious effect on graph matchability.

Herein, we propose one possible solution for ameliorating the effect of $A$ and $B$ not being identically distributed, namely via a Universal Singular Value Thresholding [9] (USVT) centering preprocessing step. Working in a general correlated edge-independent random graph model (see Definition 1), we theoretically demonstrate that USVT centering asymptotically almost surely recovers the matchability for all but a vanishing fraction of the nodes (see Theorem 9). In addition, we recover analogous result (see Theorem 14) in the setting in which only a fraction of the vertices possess a true correspondence across networks, generalizing and extending the results of [46, 30, 56].

This centering step is practically implementable on even very large networks and is demonstrated to have a significant positive impact on graph matchability in both real and synthetic data settings (see Section 3). While the results contained herein do not guarantee that any computationally efficient algorithm will be able to perfectly (or almost perfectly) align any given networks after USVT centering, they provide a theoretical foundation for subsequently studying algorithmic effectiveness. Indeed, they ensure that with high probability the optimal alignment according to the graph matching objective function is, essentially, the true latent vertex alignment, guaranteeing that subsequent optimization procedures are, at the least, seeking the right permutation.

1.1 Notation

The following notation will be used throughout the manuscript: for $k\in\mathbb{Z}>0$, we will let $J_{k}$ denote the hollow $k\times k$ matrix with all $1$’s on its off-diagonal, $[0]_{k}$ will denote the $k\times k$ matrix of all [math]’s, and $[k]$ will denote the set $\{1,2,\ldots,k\}$. We will consider $A$ and $B$ interchangeably as adjacency matrices and as the corresponding graphs consisting of vertices and edges. For a set $S\subset V(A)$, we denote by $A[S]$ the induced subgraph of $A$ on the vertices of $S$.

For a matrix $A\in\mathbb{R}^{n\times n}$, the Froebenius norm of the matrix is defined as

[TABLE]

and the operator norm of $A$ is denoted

[TABLE]

where $\sigma_{1}(A)$ is the largest singular value of $A$. We denote the trace of $A$ via

[TABLE]

Below we will make use of the following trace form of the Frobenius norm: $\|A\|_{F}^{2}=\text{tr}(A^{T}A)$; see [29] for more on the Frobenius norm and its many uses. For matrices $A\in\mathbb{R}^{n\times n}$ and $B\in\mathbb{R}^{m\times m}$, we define $A\oplus B\in\in\mathbb{R}^{(n+m)\times(n+m)}$ via

[TABLE]

where $[0]_{n,m}$ is the $n\times m$ matrix of all [math]’s.

We will also make extensive use of modern asymptotic notation. To review, if $f$ and $g$ are nonnegative functions of $n\in\mathbb{Z}\geq 0$, then we write

[TABLE]

1.2 Correlated heterogeneous Erdős-Rényi graphs

Formally the graph matching problem (GMP) we will consider is defined as follows.

Definition 1**.**

Let $A,B\in\mathbb{R}^{n\times n}$ be the adjacency matrices of weighted, undirected graphs on $n$ vertices. The graph matching problem is to find an element of

[TABLE]

where $\Pi(n)$ is the set of $n\times n$ permutation matrices.

Eq. (1) follows here from

[TABLE]

We note here that, traditionally, the GMP formulated in Definition 1 is defined for unweighted graphs $A$ and $B$. The extension we consider to weighted graphs is commonly used in the literature (see, for example, the work in [53]) and is useful for studying situations in which edges/vertices in the network have weight features attached to them. This added flexibility will be needed for subsequent theoretical developments and data applications.

In the presence of a latent vertex alignment, $\phi:[n]\mapsto[n]$, between the vertices of $A$ and $B$, we wish to understand the extent to which graph matching $A$ and $B$ will recover $\phi$; i.e., if $P_{\phi}$ is the permutation matrix corresponding to $\phi$, will $\{P_{\phi}\}=\text{argmin}_{P\in\Pi(n)}\|A-PBP^{T}\|_{F}^{2}$? In order to study this problem from a probabilistic perspective, we introduce a bivariate random graph model with a natural vertex alignment across graphs: the bivariate, correlated, heterogeneous, Erdős-Rényi random graph.

Definition 2**.**

For $R,Q_{1},Q_{2}\in\mathbb{R}^{n\times n}$ symmetric, matrices, we say $A,B$ are instantiations of the $R$-correlated Heterogeneous Erdős-Rényi random graph model with parameters $R,Q_{1},Q_{2}$ (abbreviated as $\text{CorrER}(Q_{1},Q_{2},R)$) if:

$A\sim$ER$(Q_{1})$; i.e., $A$ is an independent edge random graph with no self-loops satisfying

[TABLE]

for each $\{u,v\}\in\binom{V}{2};$

• 2.

$B\sim$ER$(Q_{2})$; i.e., $B$ is an independent edge random graph with no self-loops satisfying

[TABLE]

for each $\{u,v\}\in\binom{V}{2}$;

• 3.

Edges across networks are collectively independent except that for each $\{u,v\}\in\binom{V}{2}$, the correlation between $A(u,v)$ and $B(u,v)$ is

[TABLE]

Before proceeding further, we will make a few remarks on the $\text{CorrER}(Q_{1},Q_{2},R)$ random graph model. In the homogeneous ER$(n,p)$ model, network growth as $n\rightarrow\infty$ is natural, and we can consider an asymptotic regime in which $p$ depends on $n$. Here, we similarly consider $Q_{1}$ and $Q_{2}$ to be dependent on $n$, but make no further assumptions on expressly how the dependence on $n$ is manifest. This allows for us to consider classical homogeneous Erdős-Rényi, stochastic blockmodels [28], random dot product graphs (conditioned on the latent positions) [57], etc., as subfamilies of our random graph model.

In addition, by allowing $Q_{1}$ and $Q_{2}$ to differ, this model allows for a latent correspondence to exist in settings where the underlying topology and degree structure of the graphs to be matched differs significantly. This distributional heterogeneity is often observed in real data settings (see, for example, the connectomes being aligned in [10] and the social networks aligned in [35]), and we seek to understand the limitations of graph matching approaches when attempting to overcome this heterogeneity. Note also that when $Q_{1}\neq Q_{2}$, there are restrictions on feasible correlations $R$: Indeed, if $X\sim$Bernoulli$(p)$ and $Y\sim$Bernoulli$(q)$ are $\varrho$-correlated with $p\geq q$, then the correlation must satisfy $\varrho\leq\sqrt{\frac{q(1-p)}{p(1-q)}}$.

Lastly, this model naturally allows us to consider a partition of $V$ into core ($\mathcal{C}$) and junk ($\mathcal{J}$) vertices $V=\mathcal{C}\cup\mathcal{J}$; core vertices are those that have a corresponding vertex (i.e., true match) across networks, while junk vertices do not. If we consider $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ with $R$ of the form $R=R_{c}\oplus 0_{n_{j}}$ where $R_{c}\in(0,1]^{n_{c}\times n_{c}}$, then it is reasonable to define $\mathcal{C}=[n_{c}]$ and $\mathcal{J}=[n]\setminus[n_{c}]$. For all $u\in\mathcal{J}$ it would then hold that $\text{Corr}(A(u,v),B(u,v))=0$ for all $v$, and $A(u,v)$ and $B(u,v)$ are independent random variables. A natural question to ask is when an optimal GM algorithm will correctly align the vertices in $\mathcal{C}$ across networks. This problem was studied in the context of homogeneous ER networks with constant correlation in [30], and the results in Section 2.4 generalize and extend those in [30] to this more adaptable network model.

Remark 3*.*

In what follows, $Q_{1}$ and $Q_{2}$ are not necessarily assumed to be hollow matrices. We do assume our graphs are loop-free, so that $\mathbb{E}(A)$ (resp., $\mathbb{E}(B)$) are necessarily hollow and need not equal $Q_{1}$ (resp., $Q_{2}$).

2 Graph matchability

In the $A,B\sim\text{CorrER}(Q_{1},Q_{2},R)$ setting, we seek to understand when a graph matching procedure could correctly align the vertices across networks; i.e., if $\text{argmin}_{P\in\Pi(n)}\|A-PBP^{T}\|_{F}=\{I_{n}\},$ where $I_{n}$ denotes the identity matrix. More generally, if $\delta(\cdot,\cdot):\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n}\mapsto\mathbb{R}$ is a dissimilarity (i.e., if it is a symmetric, non-negative function with $\delta(X,X)=0$ for all $X\in\mathbb{R}^{n\times n}$; see, for example, [51]), when is it the case that

[TABLE]

In this more general framework, we consider the following definition of graph matchability:

Definition 4**.**

Let $\delta(\cdot,\cdot):\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n}\mapsto\mathbb{R}$ be a dissimilarity. We will say that $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ are $\delta$-matchable if

[TABLE]

where $I_{n}$ denotes the identity matrix.

By considering an appropriate $\delta$ in Definition 4, we can fit the classical GMP in the formulation; indeed, the GMP of Definition 1 considers the dissimilarity defined via

[TABLE]

In this paper, we will consider, more generally, dissimilarity functions of the form

[TABLE]

for suitably defined matrix-valued function

[TABLE]

Special cases of interest in our present $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ setting are

[TABLE]

where $\widehat{Q}_{A}$ (resp., $\widehat{Q}_{B}$) is a suitable estimate of $Q_{1}$ (resp., $Q_{2}$) derived from $A$ (resp., $B$). In addition to the notion of $\delta$-matchability for a dissimilarity $\delta$, we will also define the notion of oracle-matchability. We will say that $A$ and $B$ distributed $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ are oracle-matchable if

[TABLE]

In the sequel, oracle-matchability will provide a useful theoretic bridge between $\delta^{\text{GMP}}$-matchability and $\widehat{\delta}$-matchability. Note that we will write $\delta^{\text{GMP}}$-matchability and $\widehat{\delta}$-matchability for the notions defined in Eq. (2) and (3) respectively.

A natural question to ask is why we define the GMP in terms of $\delta^{\text{GMP}}$ and not in terms of a more general dissimilarity $\delta$; indeed, alternate dissimilarities $\delta$ have been considered in the definition of the GMP in the graph matching literature (see for example [60, 61]). Moreover, we consider the GMP objective function formulation in Definition 1 even though in numerous settings the optimal solution to this GMP may not be a given latent vertex alignment, and in this section, we will see instances of when $A,B\sim\text{CorrER}(Q_{1},Q_{2},R)$ are, with high probability, not $\delta^{\text{GMP}}$-matchable. Our choice of $\delta^{\text{GMP}}$ in the GMP is motivated by two main factors. First, this is the classical definition of graph matching and ties our current work to a vast graph matching literature. Second, we seek to understand conditions for when the original $\delta^{\text{GMP}}$-matchability fails, yet there is a suitable dissimilarity $\delta$ for which $\delta$-matchability is achieved. As the formulation in Definition 1 is commonly used in practice, this could provide practical guidance for when vertex labels can be recovered via a different objective function viewpoint.

In recent work addressing the question of $\delta^{\text{GMP}}$-matchability, results have been established for the $R=\varrho J_{n}$, $Q_{1}=Q_{2}=pJ_{n}$ setting (see, for example, [46, 40, 4, 18]), in the correlated stochastic blockmodel setting (see, for example, [45, 38]), in the correlated heterogeneous Erdős-Renyi model (see, for example, [39, 41]), and in the general $R$ and general $Q_{1}=Q_{2}=Q$ setting (see, for example, [49, 42]). In the non-identically distributed model setting, the work in [14, 15, 16] considers $R=\varrho J_{n}$, $Q_{1}=pJ_{n}$, and $Q_{2}=qJ_{n}$. In each setting, the results showed that for sufficiently dense, sufficiently correlated graphs, $\delta^{\text{GMP}}$-matchability is almost surely achieved. Converse results in [14, 15, 38, 40] show that in the sufficiently sparse and/or weakly correlated setting, $\delta^{\text{GMP}}$-matchability is a.s. lost (i.e., a.s. the solution to the GMP is not the latent alignment). The work in [14, 15] deserves special mention, as the converse results therein are proven for general $\delta$-matchability; i.e., for sufficiently sparse and/or weakly correlated networks, $\delta$-matchability is a.s. lost for all dissimilarities $\delta$.

In these examples, it is sparsity and/or weak dependence that is potentially thwarting the matching in each instance and not the heterogeneity of the model itself. As the next straightforward but illustrative example demonstrates, the degree and structural heterogeneity across networks allowed for in the $\text{CorrER}(Q_{1},Q_{2},R)$ model makes the question of $\delta^{\text{GMP}}$-matchability a bit more nuanced.

Example 5*.*

Consider the following correlated heterogeneous stochastic blockmodel example. Let $p,q,r\in[0,1]$ be distinct, and define

[TABLE]

Let $V_{1}=[n]$ be the vertices in block 1 in $A$ and $B$ and let $V_{2}=[2n]\setminus[n]$ be the vertices in block 2. Assuming $p>q$, and letting $\varrho_{1}\leq\sqrt{\frac{q(1-p)}{p(1-q)}}$ and $\varrho_{2}\in[0,1]$ we consider $R$ of the form

[TABLE]

Unlike in the cases where the loss of $\delta^{\text{GMP}}$-matchability is due to network sparsity and/or weak correlation, in this example the non-identically distributed nature of $A$ and $B$ can obfuscate the true alignment from a graph matching perspective. Indeed, for many choices of the parameters above, the optimal permutation for the GMP in Definition 1 will not be the latent correspondence, and permuting blocks 1 and 2 will, with high probability, yield a better GMP objective function.

To wit, let $P\in\Pi(2n)$ be any permutation such that $PQ_{2}P^{T}=Q_{1}$ (so that $P$ aligns block 1 in $A$ to block $2$ in $B$ and vice versa) with corresponding permutation $\tau$. The number of edges $\{u,v\}$ such that $\{\tau(u),\tau(v\}=\{u,v\}$ is bounded above by $n$ and

[TABLE]

Therefore,

[TABLE]

Combined, we see that the difference in the objective function for $P$ as compared to $I_{2n}$ is

[TABLE]

Numerous choices of the parameters in this model (for example, $p=0.8$, $q=0.2$, $r=0.1$, $\varrho_{1}=0.25$, $\varrho_{2}=0.3$) yields that $\mathbb{E}[\text{tr}(APBP^{T})-\text{tr}(AB)]\geq cn^{2}+O(n)$ for a positive constant $c>0$ (in the example, $c=0.113$). As $\text{tr}(APBP^{T})-\text{tr}(AB)$ is highly concentrated about its expectation (see Appendix A.1), there is high probability that $\text{tr}(APBP^{T})-\text{tr}(AB)>0$, and

[TABLE]

i.e., $A$ and $B$ would not be $\delta^{\text{GMP}}$-matchable.

2.1 Centering to recover matching

In the previous example, we see that $\mathbb{E}(A)\neq\mathbb{E}(B)$ can effectively make $A$ and $B$ not $\delta^{\text{GMP}}$-matchable. One way to recover the latent alignment in this heterogeneous setting is to transform the problem back into the homogeneous case, and rather than matching $A$ and $B$, we would match $\widetilde{A}=A-\mathbb{E}(A)$ and $\widetilde{B}=B-\mathbb{E}(B)$; yielding once again $\mathbb{E}\widetilde{B}=\mathbb{E}\widetilde{A}$. As the next theorem demonstrates, this is sufficient to a.s. recover oracle-matchability under mild model conditions. Before stating the theorem, we first must define some additional notation. For $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$, and permutation $P\in\Pi(n)$, define the matrix $\mathcal{K}_{A,B,P}$ via

[TABLE]

For each $P$ in $\Pi(n)$, define

[TABLE]

Theorem 6**.**

Let $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ and consider $\widetilde{A}=A-\mathbb{E}(A)$ and $\widetilde{B}=B-\mathbb{E}(B)$. For each $k\in[n]$, define

[TABLE]

If for all $k\in[n]$ and all $P\in\Pi(n,k)$, we have

[TABLE]

then

[TABLE]

The proof of Theorem 6 relies on a now standard application of McDiarmid’s inequality and is similar to the proofs of analogous matchability results in [40, 39, 38]; details of the proof can be found in Appendix A.1.

Remark 7*.*

The growth condition on $\mathcal{X}_{P}$ in Theorem 6, namely $\mathcal{X}_{P}/k=\omega(\sqrt{n\log n})$, is attempting to capture the necessary degree to which the entry-wise covariance matrix $\mathcal{C}$ needs to be asymmetric. If we define \epsilon=\min_{u,v}\big{[}\mathcal{C}(A,B)\big{]}(u,v), then from Eqs. (16) and (15) we that for $P\in\Pi(n,k)$,

[TABLE]

and if $\epsilon=\omega(\sqrt{\log n/n})$ then $\mathcal{X}_{P}/k=\omega(\sqrt{n\log n})$. Constraining $\mathcal{X}_{P}$ globally and not entry-wise allows for more flexibility in applying the theorem to settings where some of the edges are very sparse or weakly correlated.

Consider the growth condition on $\mathcal{X}_{P}$, namely $\varrho p=\omega(\sqrt{\log n/n})$, in the $Q_{1}=Q_{2}=pJ_{n}$, $R=\varrho J_{n}$ homogeneous ER setting (wlog, assume $p<1/2$). In this setting, as $\mathbb{E}(A)=\mathbb{E}(B)$, $A$ and $B$ are oracle-matchable iff $A$ and $B$ are $\delta^{\text{GMP}}$-matchable. In the sparse setting of [14, Theorem 1], $\delta^{\text{GMP}}$-matchability is achieved with high probability when all of the following hold

[TABLE]

Note that these conditions cannot simultaneously hold when

[TABLE]

Indeed if $p/\varrho=\Omega(1)$, Eq. (7) implies $p=o(1)$, and hence $\varrho=o(1)$. Therefore

[TABLE]

contradicting Eq. (8). In the $p/\varrho=\omega(1)$ setting, modulo the sparsity conditions, there is a $\delta^{\text{GMP}}$-matchability phase transition at $p^{2}+p\varrho=\frac{\log n}{n}$, and the corresponding rate achieved in Theorem 6, namely $\varrho p=\omega(\sqrt{\log n/n})$, is above this phase transition threshold. We view this as the price paid in the Theorem for being able to handle both heterogeneous and homogeneous ER settings.

In this setting, the growth condition of Theorem 6, $\varrho p=\omega(\sqrt{\log n/n})$, can hold in the dense setting $(p=\Theta(1))$ as well as when $p/\varrho=\Omega(1).$ In the dense setting of $p=\Theta(1)$, $\delta^{\text{GMP}}$-matchability transitions at $\varrho^{2}=\Theta(\frac{\log n}{n})$ [38, 40], which our Theorem recovers (asymptotically).

In [14], the authors establish a phase transition at $p^{2}+p\varrho=\frac{\log n}{n}$, providing the corresponding converse result that ensures no $\delta$-matchability if $p^{2}+p\varrho\leq\frac{\log n-\omega(1)}{n}$ for all dissimilarities $\delta$. We do not derive a corresponding converse result to Theorem 6 herein (namely, a condition on $\mathcal{X}_{P}$ that ensures $A$ and $B$ are not $\delta$-matchable for any suitable $\delta$), as we are focused on how to practically recover $\delta$-matchability in the non-identically distributed setting; see Theorem 9.

2.2 Approximate centering to almost recover matching

Unfortunately, centering $A$ and $B$ by the true edge probability matrices $\mathbb{E}(A)$ and $\mathbb{E}(B)$ is impractical, as these model parameters are unknown in practice. Our solution to this hurdle is to estimate the unknown $\mathbb{E}(A)$ and $\mathbb{E}(B)$ via Universal Singular Value Thresholding (USVT) [9], and then approximately center the networks via these estimates.

Our method for estimating $\mathbb{E}(A)$ and $\mathbb{E}(B)$ is based on the Universal Singular Value Thresholding (USVT) method of [9], and USVT applied in the present setting is outlined in Algorithm 1.

If we estimate $\mathbb{E}(A)$ via $\widehat{Q}_{1}$ and $\mathbb{E}(B)$ via $\widehat{Q}_{2}$ using USVT, can we recover $\delta$-matchability using the approximately centered matrices, $\widehat{A}=A-\widehat{Q}_{1}$ and $\widehat{B}=B-\widehat{Q}_{2}$, for a suitable $\delta$? Given the error introduced in estimating the edge probability matrices, the answer is unsurprisingly no, at least for the proof techniques we employ herein. However, if we slightly weaken Definition 4 to allow for a vanishing fraction of unmatched nodes, then we can recover an analogous result to Theorem 6. This motivates the following definition:

Definition 8**.**

Let $\mathbf{\delta}(\cdot,\cdot):\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n}\mapsto\mathbb{R}$ be a dissimilarity. Consider random graphs $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$. We say that $A$ and $B$ are $f_{n},\delta$-matchable if

[TABLE]

Unwrapping Definition 8, we see that $f_{n},\delta$-matchability is equivalent to any optimal permutation (under dissimilarity $\delta$) correctly recovering the labels of at least $n-f_{n}$ vertices across $A$ and $B$. As the following theorem indicates, under mild model assumptions, $A$ and $B$ are $\Theta(\sqrt{n}),\widehat{\delta}$-matchable asymptotically almost surely, where the estimates $\widehat{Q}_{A}$ and $\widehat{Q}_{B}$ in Eq. 3 are the USVT estimates. The proof of Theorem 9 can be found in Appendix A.2.

Theorem 9**.**

Let $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ and further assume that for each $i=1,2$,

• i.

There exists $r_{i}\in[0,1]$ such that $Q_{i}\leq r_{i}$ entry-wise. Note that for each $n$, $r_{i}$ is fixed, though we allow $r_{i}$ to vary in $n$.

• ii.

We have that $\min_{j}\sum_{\ell}Q_{i}(j,\ell)=\omega(\log n)$.

• iii.

$Q_{i}$* is approximately low rank in that there exists a $d_{i}=o(n)$ such that $\sum_{j>d_{i}}s_{j}^{2}(Q_{i})=O(nr_{i}d_{i})$ where $s_{1}(Q_{i})\geq s_{2}(Q_{i})\geq\cdots\geq s_{n}(Q_{i})$ are the singular values of $Q_{i}$.*

If for all $k\in[n]$ and all $P\in\Pi(n,k)$, we have that there exists $\delta=\delta(n)\geq 0$ such that

[TABLE]

and for each $i=1,2$, there exists constants $a_{i}>0$ such that if $\widehat{Q}_{i}$ is the USVT estimate of $Q_{i}$ with threshold level $t_{i}=a_{i}\sqrt{nr_{i}}$ and if

[TABLE]

then $\widehat{A}=A-\widehat{Q}_{1}$ and $\widehat{B}=B-\widehat{Q}_{2}$ satisfy

[TABLE]

Let us take a moment to explore the assumptions in Theorem 9. Assumption $i.$ and $ii.$ control the allowable sparsity of the networks, ensuring that the minimum expected degree grows asymptotically faster than $\log n$. If the mean expected degree was $o(\log n)$, then the graphs would be a.s. disconnected [6], and our proof techniques fail as $A$ and $B$ would no longer concentrate about $Q_{1}$ and $Q_{2}$ with high probability [33]. The rank assumption in $iii.$ is needed to control the accuracy of the USVT estimates of the unknown $Q_{i}$’s. Practically, smaller $d_{i}$ allow us to use suitable low-rank estimates $\widehat{Q}_{i}$ of $Q_{i}$ that are computationally easier to implement; this is indeed the case in many common random graph models such as the Stochastic blockmodel [28] (where often $d_{i}=O(1)$), random dot product graphs [57] and latent position random graphs [27] (where $d_{i}$ is often taken to be $O(\mathrm{polylog}(n))$ [52]), among others.

If $R$ is bounded away from [math] entry-wise, and each entry of $Q_{i}$ is $\Theta(r_{i})$ (which is indeed the case in the oft adopted setting where each $Q_{i}=r_{i}M_{i}$ for a matrix $M_{i}$ with entries of order $\Theta(1)$) then $\epsilon$ as defined in Remark 7 satisfies $\epsilon=\Theta(\sqrt{r_{1}r_{2}})$. We then have $\mathcal{X}_{P}/k=\Omega(\sqrt{r_{1}r_{2}}\,n).$ From Eq. 22 in the proof of Theorem 9, we see that $k_{0},\widehat{\delta}$-matchability is achieved here for

[TABLE]

If, in addition $r_{1}/r_{2}=\Theta(1)$ and each $d_{i}=O(\text{polylog}(n))$, then up to a logarithmic factor $A$ and $B$ are $\Theta(\sqrt{n}),\widehat{\delta}$-matchable, and an oracle graph matching algorithm would properly align all but potentially a vanishing fraction of the nodes across the graphs.

Remark 10*.*

In Theorem 9, we estimate $Q_{i}$ via $\widehat{Q}_{i}$ using USVT with threshold $t\approx\sqrt{n}$. In application, often suitable estimates of $Q_{i}$ can be obtained with rank $d_{i}$ of order $\log n$ or $\Theta(1)$ [52], especially in the setting of latent space graph models. For the purposes of our proof approach, suitably good means that $\|\mathbb{E}(A)-\widehat{Q}_{1}\|_{F}=O(\sqrt{nr_{i}d_{i}})$ (similarly for $B$). We do not explore this model selection question further here (i.e., estimating a suitable rank rather than a threshold for our USVT estimates), as in applications often only a relatively small number of singular values are above the USVT threshold.

2.3 When to center?

We have seen above that in the setting where both $Q_{1}\neq Q_{2}$ and $A,B$ are not $\delta^{\text{GMP}}$-matchable, centering $A$ and $B$ via $\widetilde{A}$ and $\widetilde{B}$ can recover oracle-matchability by ameliorating the effect of the differing $Q_{i}$’s. Moreover, approximately centering by $\widehat{Q}_{1}$ and $\widehat{Q}_{2}$ theoretically recovers $\widehat{\delta}$-matchability for all but a vanishing fraction of the vertices. A natural question is in the case when $Q_{1}=Q_{2}$, does Theorem 9 imply that a.s. perfect $\delta^{\text{GMP}}$-matchability is potentially lost when USVT centering is performed unnecessarily?

Consider the following simple example, where $Q_{1}=Q_{2}=pJ_{n}$ and $R=\varrho J_{n}$, $n=100$, and we vary $p\in\{0.1,0.3,0.5\}$ and $\varrho\in\{0.1,0.3,0.5,0.7,0.9\}$. In this example, there is no need to center before matching, and the variability introduced by estimating the $Q_{i}$’s could potentially cause

[TABLE]

Fortunately, at least in this example we see this is not the case (see Figure 1). As matching these graphs exactly (i.e., finding the argmin of the GMP) is computationally challenging, we use as a surrogate for $\delta^{\text{GMP}}$-matchability and $\widehat{\delta}$-matchability whether the true alignment is a local (rather than global) minima of the GMP before and after centering. To test this, we match the graph pairs (USVT centered and uncentered) using the constrained gradient-ascent based graph matching algorithm, FAQ [54], initialized at the true correspondence $I_{n}$. While FAQ is not guaranteed to terminate at a local minima, if it terminate at $I_{n}$, then that is evidence in support of $I_{n}$’s local optimality. Moreover, if FAQ does not terminate at $I_{n}$, then $I_{n}$ is not a local minima of the GMP.

Letting FAQ$(A,B,I_{n})$ (resp., FAQ$(A-\widehat{Q}_{1},B-\widehat{Q}_{2},I_{n})$) denote the number of vertices correctly matched by FAQ initiated at $I_{n}$ when $A$ and $B$ are matched directly (resp., when $A$ and $B$ are USVT centered before they are matched). In Figure 1, we plot the mean ($\pm$ 1 s.d.) of FAQ$(A,B,I_{n})-$FAQ$(A-\widehat{Q}_{1},B-\widehat{Q}_{2},I_{n})$ over $p\in\{0.1,0.3,0.5\}$ and $\varrho\in\{0.1,0.3,0.5,0.7,0.9\}$. From the figure, we see that there is no significant performance lost by centering the graphs first. Indeed, in highly structured/low-rank settings (e.g., homogeneous ER or SBM), we can obtain high-fidelity estimates of the individual entries of the $Q_{i}$ versus the global estimates used in current proof. These local estimates (which can also be obtained by non-spectral methods, e.g., in the $\mathrm{ER}(n,p)$ case we can use $\hat{p}=\sum_{i<j}A(i,j)/\binom{n}{2}$) should allow for significantly less error to be introduced in the estimation of the $Q_{i}$’s and will allow for little–to–no theoretic degradation due to centering. As we are more focused on the general $Q_{i}$ case, we do not pursue this further here.

2.4 Core Matchability

Often in applications, only a fraction of the vertices in $A$ possess a latent matched pair in $B$. We will denote those vertices that have a latent match across graphs as the core vertices (denoted $\mathcal{C}$), and we will denote those vertices that do not have a latent match across graphs as the junk vertices (denoted $\mathcal{J}$). In this section, we seek to further understand the ability of an oracle graph matching procedure to correctly match the cores across graphs. This motivates the following definition of core-matchability.

Definition 11**.**

Let $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$, and consider a partition of the vertex sets into core and junk vertices,

[TABLE]

Define $\mathcal{P}_{n}^{\mathcal{C}}:=\{P\in\Pi(n)\text{ s.t. }\forall i\in\mathcal{C},P(i,i)=1\}$ to be the set of core matching permutations. For dissimilarity $\delta$, we say that $A$ and $B$ are core $\delta$-matchable if

[TABLE]

i.e., if any optimal permutation aligning $A$ and $B$ under $\delta$ perfectly matches the cores across networks.

If we consider $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ with $R$ of the form $R=R_{c}\oplus 0_{n_{j}}$ where $R_{c}\in(0,1]^{n_{c}\times n_{c}}$, then it is reasonable to define $\mathcal{C}=[n_{c}]$ and $\mathcal{J}=[n]\setminus[n_{c}]$. Indeed, under these model assumptions $R(u,v)=0$ if either $u$ or $v$ is in $\mathcal{J}$.

Completely analogously to the setting considered in Example 5, it is immediate that $A$ and $B$ need not be asymptotically almost surely core $\delta^{\text{GMP}}$-matchable even with non-vanishing core correlation in $R$. Indeed, as in Example 5, $Q_{1}$ and $Q_{2}$ can be chosen to effectively obfuscate the true alignment amongst the core vertices. Mimicking the results of Theorem 6, centering $\widetilde{A}=A-\mathbb{E}(A)$ and $\widetilde{B}=B-\mathbb{E}(B)$ again a.s. recovers core oracle-matchability of $A$ and $B$ under mild model assumptions. The proof of Theorem 12 is contained in Appendix A.3.

Theorem 12**.**

Let $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ and consider $\widetilde{A}=A-\mathbb{E}(A)$ and $\widetilde{B}=B-\mathbb{E}(B)$. Suppose that $R$ is of the form $R=R_{c}\oplus 0_{n_{j}}$ where $R_{c}\in(0,1]^{n_{c}\times n_{c}}$, and for each $P\in\Pi(n)$, let $\mathcal{X}_{P}$ be defined as in Eq. (5). If for all $Q\in\{P\in\Pi(n)\text{ s.t. }\sum_{i=1}^{n_{c}}P(i,i)=n_{c}-k\}$ we have that

[TABLE]

and also if $n_{c}\geq n_{j}$, then

[TABLE]

As before, if we define $\epsilon:=\min_{u,v\in[n_{c}]}Cov(A(u,v),B(u,v)),$ then for $Q\in\Pi(n,k)$, $\mathcal{X}_{Q}/k=\Omega(\epsilon n_{c})$. If, in addition,

[TABLE]

then Eqs. (10) and (11) hold and core oracle-matchability is recovered. In the event that $\epsilon$ is $\Theta(1)$ or $\Theta(1/\text{polylog}(n)),$ then Theorem 12 implies that $A$ and $B$ are core $\delta$-matchable in the presence of nearly linear junk, with arbitrary junk structure. This result extends and generalizes the results in [30] to the non-homogeneous ER setting.

As before, if the unknown $\mathbb{E}(A)$ and $\mathbb{E}(B)$ are estimated via USVT, then we recover partial core matchability. Before formalizing this, we first need the following extension of Definition 8 to the core-junk setting.

Definition 13**.**

Let $\delta:\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n}\mapsto\mathbb{R}$ be a dissimilarity. Let $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$, and consider a partition of the vertex sets into core and junk vertices,

[TABLE]

Define

[TABLE]

We say that $A$ and $B$ are core $f_{n},\delta$-matchable if

[TABLE]

The following Theorem provides the analogue of Theorem 9 in the core-junk setting. Note that the proof is completely analogous to that in Theorem 9, and so is omitted.

Theorem 14**.**

Let $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ and consider $\widehat{A}=A-\widehat{Q}_{1}$ and $\widehat{B}=B-\widehat{Q}_{2}$. Suppose that $R$ is of the form $R=R_{c}\oplus 0_{n_{j}}$ where $R_{c}\in(0,1]^{n_{c}\times n_{c}}$. With the assumptions on $Q_{1}$ and $Q_{2}$ from Theorem 9, and assume for each $Q\in\{P\in\Pi(n)\text{ s.t. }\sum_{i=1}^{n_{c}}P(i,i)=n_{c}-k\}$,

[TABLE]

and also assume $n_{c}\geq n_{j}$. For each $i=1,2$, there exists constants $a_{i}>0$ such that if $\widehat{Q}_{i}$ is the USVT estimate of $Q_{i}$ with threshold level $t_{i}=a_{i}\sqrt{n\rho_{i}}$ and if

[TABLE]

then

[TABLE]

3 Simulations and Experiments

In the following sections, we explore the impact on graph matchability of USVT centering in both simulated and real data settings. We note here that precisely determining the level of $\delta^{\text{GMP}}$, $\widehat{\delta}$ and oracle-matchability is infeasible for even modestly sized networks, as this would require exactly solving the NP-hard graph matching problem. To circumvent this, we instead match our networks using the FAQ algorithm of [54] initialized at a variety of starting points including the true correspondence. As it is a Frank-Wolfe [25] based algorithm, if FAQ terminates at the true correspondence (or at a permutation which matches a high percentage of the vertices), then the true correspondence is an estimated local minima of the graph matching problem. Moreover, if FAQ is initialized at the true correspondence and does not terminate at the true correspondence, then the true correspondence is not a local minima. Comparing objective function values across estimated local minima then allows us to approximately gauge the global optimality of the true correspondence. While this is not the same as directly finding the global minima desired in the definition of matchability, it nonetheless provides a useful, principled heuristic for empirically studying both matchability and deviations there from.

3.1 Simulation

To explore the utility of USVT centering as a graph matching preprocessing step, we consider the following experiment. We let $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ with

[TABLE]

and $R$ of the form

[TABLE]

and we use the FAQ algorithm of [54] to match i. $A$ and $B$ directly (labeled “Uncentered” in Figures 2–3); ii. $\widetilde{A}=A-Q_{1}$ and $\widetilde{B}=B-Q_{2}$ (labeled “Centered” in Figures 2–3); and $\widehat{A}=A-\widehat{Q}_{1}$ and $\widehat{B}=B-\widehat{Q}_{2}$ (labeled “Approx. Centered” in Figures 2–3). In each figure, we initialize the FAQ algorithm at $I_{2n}$—i.e., at the true latent alignment—and at $\tilde{I}_{2n}:=\begin{pmatrix}0&I_{n}\\ I_{n}&0\end{pmatrix}$—i.e., at the alignment completely confusing blocks one and two across networks. We plot the mean fraction of vertices matched correctly ($\pm 2$s.d.) by FAQ at the starting point that achieves the lowest graph matching objective function score (averaged over $100$ Monte Carlo replicates). As mentioned above, if the fraction matched correctly is less than $1$, then the true alignment is not a local minimum of the graph matching objective function and the graphs are not $\delta^{\text{GMP}}$, $\widehat{\delta}$, or oracle-matchable (depending on what input FAQ is matching).

In Figure 2, we consider $n=150$—i.e., the graphs are size $300$—and in the USVT estimates $\widehat{Q}_{i}$ we used $t=2.01\sqrt{300*0.8*0.2}$ as suggested in [9]. We plot $\alpha\in(0.75,0.85,0.95,1)$ versus the the mean fraction of vertices matched correctly ($\pm 2$s.d.) It is unsurprising in light of Example 5, that $A$ and $B$ are not only directly not $\delta^{\text{GMP}}$-matchable, but that the alignment found by FAQ matches none of the vertices correctly across graphs. Also of note in the figure is that as $\alpha$ increases, the oracle centered graphs $\widetilde{A}$ and $\widetilde{B}$ appear to be nearly oracle-matchable (in that the estimated local minimum found by FAQ is close to $I_{300}$). The steep performance drop off as $\alpha$ decreases is a consequence of the fact that in low-correlation regimes (low for a given $n$), oracle-matchability is often not recovered even through centering. Performance in the approximately centered case tracks performance in the centered case, with the USVT centering recovering the gains of the oracle centering. This empirically suggests that the $k_{0}$ lower bound in Theorem 9 is not sharp, as USVT centering recovers full $\widehat{\delta}$-matchability in the high $\alpha$ regime. We surmise that if $Q_{i}$ is truly low-rank, USVT centering and true centering will recover perfect $\widehat{\delta}$- and oracle-matchability respectively as $n$ increases.

In Figure 3 we repeat the above simulation with $\alpha=1$ fixed and $n\in(25,50,100,150)$. Using $t=2.01\sqrt{(2n*0.8*0.2)}$ as the USVT threshold, we again plot the mean fraction of vertices matched correctly ($\pm 2$ s.d.) versus $n$. As before, without centering the optimal alignment found by FAQ matches very few of the vertices correctly across graphs. The performance increase in the oracle centered setting as $n$ increases is a consequence of the oracle-matchability (in Theorem 6) being an asymptotic result; indeed, we should not expect correlated small networks to be almost surely oracle-matchable even with the oracle centering. We note, however, that $n=150$ (i.e., graph order $=300$) here is sufficient for the asymptotically perfect oracle-matchability to be recovered. Again we see that performance in the approximately centered case tracks performance in the centered case, with the USVT centering achieving almost all of the gains of the oracle centering.

3.2 Twitter data

In order to analyze the impact of USVT centering with real data, we considered two graphs derived from Twitter.111These graphs were provided as part of the DARPA XDATA project. The two graphs are based on the most active twitter users from April and May 2014. The graphs are unweighted with an edge between users if a user mentioned another user during the given month. After keeping only the largest common connected component, the number of users was 431 in each graph.

As can be seen in Figure 4, many vertices in this data set have very similar connectivity patterns. Indeed, the empirical Pearson correlation between the entries in the two adjacency matrices is $\approx 0.72$. It is not surprising that the similar graph topology across networks leads to good performance when matching the graphs without centering. Repeating the experiment in Section 3.1—i.e., matching the adjacency matrices $A$ (“April”) and $B$ (“May”) using FAQ initialized at $I_{431}$—yields $323$ vertices correctly aligned across networks. Although the true correspondence is not optimal (according to the GM objective function), the estimated local optimal correspondence does match $\approx 75\%$ of the vertices correctly across networks without the need for centering. It is worth noting that preprocessing the data via USVT centering before matching again yields $323$ vertices correctly matched by FAQ initialized at $I_{431}$ (with a suitable USVT threshold here being $2\sqrt{431}$). This suggests that the centering procedure does not hurt performance when the graph topologies are similar across networks, and as we will demonstrate below, can significantly increase performance when the graph topologies differ across networks. We do note here that we do not zero out the diagonal of $\widehat{Q}_{i}$ in the USVT step in this real data example, as here, hollow $\widehat{Q}_{i}$’s led to significantly worse performance than the non-hollow $\widehat{Q}_{i}$’s.

While both the centered and uncentered graphs are highly $\widehat{\delta}$- and $\delta^{\text{GMP}}$-matchable respectively, centering does have a very interesting algorithmic effect here; see Figure 5. In the figure, we plot the number of vertices in the April-May Twitter graphs correctly matched by FAQ versus the graph matching objective function value. In each panel (on the left matching $A$ and $B$, and on the right $\widehat{A}$ and $\widehat{B}$), we initialize FAQ at 100 different starting points: once at $I_{n}$ (labeled “P0=I” in the legend) and the rest at random permutation restarts (labeled “rand. start” in the legend). The figure suggests that centering has the effect of creating a more stable objective function gap between the estimated optimal permutation and suboptimal alternatives. In a setting where multiple random restarts are possible—and needed—to recover an unknown latent alignment, this suggests that the optimal alignment is perhaps more easily recognized in the centered graph regime, and hence online stopping criterion more easily implementable.

To explore the performance of the two approaches (centering and not centering) in the setting of different network topologies, we consider the following synthetic data experiment. We choose $100$ random users from the twitter networks and add $C$ to their induced subgraphs in the May network, where $C\sim$ER$(100,q)$ (followed by again binarizing $B$); i.e., if the set of randomly chosen vertices is $\mathcal{V}$, then $B[\mathcal{V}]\mapsfrom B[\mathcal{V}]+C$ and the subsequent $B$ is binarized before matching or centering. We consider $q\in\{0.1,0.3,0.5,0.7,0.9\}$. This experiment is simulating the setting where a fraction of the network changes its behavior from April to May; in this case by increasing their volume of mentions month to month. For each value of $q$, we repeat this experiment $25$ times and plot the mean accuracy ($\pm$1 s.d.) of graph matching using FAQ initialized at $I_{431}$ both with and without USVT centering; see Figure 6. This experiment demonstrates the capacity for USVT to maintain $\widehat{\delta}$-matchability in the face of additive deviations in the network structure. These deviations have the effect of altering the graph topology month–to–month, and with enough signal, they have a precipitously negative impact on the performance of matching sans centering. Centering ameliorates this effect, and emphasizes the common signal in the networks by removing the effect of this additive noise. It is interesting to note that for small values of $q$, centering negatively impacts algorithmic performance. We view this as potentially an artifact of the noise in these settings not being sufficient to obfuscate the true matching without centering.

In the core-junk setting, the heterogeneity amongst the junk vertices offers a further setting for demonstrating the utility of USVT-centered graph matching. To see this, we consider the following experiment: choose $n_{c}=100$ uniformly sampled core vertices from the twitter network and $n_{j}\in\{25,50,100,150\}$ uniformly sampled junk vertices, $\mathcal{J}_{1}$, for the April graph and $n_{j}$ uniformly sampled junk vertices, $\mathcal{J}_{2}$, for the May graph. As before, we match $A[\mathcal{C}\cup\mathcal{J}_{1}]$ and $B[\mathcal{C}\cup\mathcal{J}_{2}]$ using FAQ initialized at $I_{n_{c}+n_{j}}$ both with and without USVT centering; results are summarized in Figure 7. As seen previously, the ability of USVT-centering to ameliorate the degree/distributional heterogeneity (here amongst the junk vertices) leads to superior core label recovery compared to the uncentered matching setting.

3.3 Connectomes

For our next example, we consider the the test-retest diffusion MRI data from [32]. The dataset consists of test-retest pairs (used to evaluate reproducibility of the magnetization prepared rapid acquisition gradient echo (MPRAGE) image protocol). Each scan is converted into a weighted connectome by considering 70 brain regions of interest (labeled according to the Desikan brain atlas [17]) as the vertices, with edge weights counting the number of neural fiber bundles connecting the regions. As vertices correspond to canonical brain regions of interest, it is natural to consider the true correspondence across graphs as being given by the identity mapping.

To illustrate the role of USVT centering in this data set, we first consider as an example a pair of graphs generated as above from the data in [32]. The respective adjacency matrices for this graph pair are shown in Figure 8. Matching these brains directly using FAQ initialized at $I_{70}$ yields an estimated local optimum with 65 vertices correctly aligned across graphs; indeed, by permuting vertices $3,33,38,50,68$, we obtain a better objective function value than the GMP evaluate at $I_{70}$. We seek to understand the ability of USVT centering, which is global in nature, to correct these local mismatches.

To study this further, we apply a variant of the USVT procedure in which we automatically select the number of singular values to threshold by combining the ideas of USVT with the profile likelihood work of [63]; to wit, we select the threshold dimension via an elbow analysis of the SCREE plot of the singular values. We chose this automated procedure rather than setting a singular value threshold because these graphs are weighted, and the common threshold of $2.01\sqrt{n}$ from [9, 55] is presented for the unweighted setting. Centering the pair of graphs from Figure 8 recovers the identity $I_{70}$ as an estimated local minima of the GMP, and the global centering corrects the localized mismatch.

Extending this to a 41 scan sample from [32] (hence, we consider 41 graphs each with 70 vertices), we run FAQ initialized at $I_{70}$ for the $\binom{41}{2}$ pairs of distinct graphs with both USVT centering and no centering. When matching graph $i$ and graph $j$ for $\{i,j\}\in\binom{41}{2}$, we let

[TABLE]

In Figure 9, we plot a heatmap of the $\binom{41}{2}$ differences $D_{c}(i,j)-D(i,j)$, so that the $i,j$-th entry in the heatmap corresponds to the excess number of correct matches achieved by USVT centering. Red values indicate more correctly matched via centering and blue values indicate more correctly matched via no centering. The color intensity indicates the value of $D_{c}(i,j)-D(i,j)$ achieved, with darker colors indicating more (in the red case) or less (in the blue case) vertices correctly matched after centering. The figure demonstrates that the phenomena observed in the graphs in Figure 8 was not an anomaly. Only $2$ pairs see an improvement in matching accuracy when not centering, while $284$ pairs see an improvement in matching accuracy when USVT centering. Moreover, while many of the mismatches are local in nature, they are nonetheless ameliorated by the global USVT centering procedure.

If we consider running the same experiment on the unweighted brain graphs (using USVT centering with threshold $t$), we see the delicate nature of the USVT threshold in data applications. We note here that, again, in this unweighted case, we do not zero out the diagonal of $\widehat{Q}_{i}$ in the USVT step in this real data example. When $t=2\sqrt{70}$, $267$ pairs achieved improved matching performance when USVT centering first, and $271$ pairs achieved improved matching performance when not centering. When $t=\sqrt{70}$, $264$ pairs achieved improved matching performance when USVT centering first, and $343$ pairs achieved improved matching performance when not centering. These results suggest two important take-aways: First, performance is intimately tied to properly thresholding; and second, in this example, USVT-centering is more effective in the weighted edge case. This suggests both that the magnitude of weights are contributing significantly to the mismatch and that USVT centering is effective at ameliorating this edge weight heterogeneity; this is not entirely unexpected as the centering is precisely trying to eliminate the different edge probability/weight structures across the $Q_{i}$.

4 Discussion

Understanding the limits of $\delta$-matchability is an essential step in robust multiple graph inference regimes. When graphs are not $\delta$-matchable—i.e., the true node correspondence cannot be recovered in the face of noise—paired graph inference methodologies that utilize the across graph correspondence (see, for example, [50, 2]) cannot gainfully be employed. Non-$\delta$-matchability can limit analysis to methods which rely on graph statistics which are invariant to relabeling of the vertices, which can be useful but lack the full power of their parametric (with the labeling as a parameter) counterparts. In this paper, we establish initial theoretical results on $\delta$-matchability when the graphs to be match differ in distribution and when only a fraction of the graphs are matchable.

While our theoretical results and subsequent simulations and experiments provide a basis for a deeper understanding of the effect that distributional heterogeneity has on $\delta$-matchability, there is still much to be done. For example, in our present USVT centering step, the across graph correlation provided by $R$ is not utilized. We suspect that the error in the USVT steps could be greatly reduced by leveraging $R$ or an estimate thereof. We are also exploring graph normalization strategies other than centering, such as kernel smoothing (using a small number of a priori known correspondences to choose the proper smoothing kernels), which may be more appropriate in the presence of multiplicative or other non-linear noise structures. We suspect that the growth rate of $k_{0}$ in Theorem 9 is not sharp, as in simulation and real data settings the true correspondence is almost perfectly recovered via USVT centering before matching; however, sharpening the lower bound on $k_{0}$ with our present methods does not seem feasible and new ideas and techniques need be employed.

We suspect that the bounds on $n_{j}$ versus $n_{c}$ obtained in Theorem 14 are also suboptimal. As an informal argument we can consider known results for the quadratic assignment problem for i.i.d. entries [8]. In the uncorrelated dense homogeneous Erdős-Rényi setting, these results imply that the best best solution to the graph matching problem, Eq. (1) will reduce the objective function $\Theta(n_{j}^{3/2})$ as compared to a random guess. On the other hand, in the dense homogeneous Erdős-Rényi setting with constant correlation, the best solution is $\Theta(n_{c}^{2})$ better than a random guess. Under the heuristic that in order for the core vertices to be matched correctly, the signal from correlation must be greater than the possible improvements in the all noise setting, we can conjecture that a core matchability threshold at approximately $n_{c}=O(n^{3/4})$ may be possible. Using our present proof technique, we are unable to achieve this rate. This heuristic argument, though problematic, provides a potential guidepost for future work.

In addition, while the theoretical results presented herein are for the case that the graphs are simple undirected graph with no edge-weights, the graph matching framework of Definition 1 is flexible, allowing us to accommodate many of the features—both those considered above, and additional eccentricities—inherent to real data settings. In the weighted, loopy setting, the matching can occur between the weighted adjacency matrices or normalized Laplacian matrices, $\mathcal{L}_{A}=D_{A}^{-1/2}AD_{A}^{-1/2}$ (where $D_{A}$ is the diagonal matrix with $(i,i)$-th entry equal to $\sum_{j}D_{A}(i,j)$) and the similarly defined $\mathcal{L}_{B}$. To match directed graphs, the graphs can either be made undirected (for example, by matching $A^{\prime}=(A+A^{T})/2$ to $B^{\prime}=(B+B^{T})/2$) or the directed adjacency matrices can be directly plugged into Eq. (1). Developing similar results to Theorems 9 and 14 in models (akin to our CorrER model) that incorporate these graph features as well as additional vertex and edge features is a natural next step.

The information and computational limits for $\delta$-matchability are still open problems for which we have pushed the boundaries, but significant more work is to be done. These problems are in analogy to the recently addressed problems of detection and recovery for the planted partition and planted clique problems for a single graph [23, 44, 1]. For these settings, exact fundamental limits have been established and polynomial time algorithms have been shown to achieve or nearly achieve these limits. Obtaining similar results for the graph matching problem are key steps towards a robust statistical framework for multiple graph inference.

Acknowledgments

Dr. Sussman’s contribution to this work was partially supported by a grant from MIT Lincoln Labs and the Department of Defense. This material is based on research sponsored by the Air Force Research Laboratory and DARPA under agreement number FA8750-18-2-0066. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Air Force Research Laboratory and DARPA or the U.S. Government. We would also like to thank Prof. Carey E. Priebe, Prof. Minh Tang, and Joshua Cape for their helpful discussions in the writing of this manuscript.

Appendix A Proofs

Herein we collect proofs of the main theoretical results in this manuscript. Before stating our proofs, we first state some well-known facts about the bivariate Bernoulli distribution.

Indeed, if $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ model, then for each $\{u,v\}\in\binom{[n]}{2}$, $A(u,v)$ and $B(u,v)$ can be realized as a bivariate Bernoulli random variable. This will be a key insight in the proof of our main results, Theorems 6 and 9.

Spelling this out further, a pair of Bernoulli random variables $(X,Y)$ has a BiBernoulli distribution with

[TABLE]

if $\mathbb{P}[X=x,Y=y]=p_{xy}$ for each $(x,y)\in\{0,1\}^{2}$. A key property of BiBernoulli random variables is that they can be generated by a triple of independent Bernoulli random variables. For $(X,Y)$ as above, setting $Z_{0}\sim\mathrm{Bern}(p_{11}+p_{10}$), $Z_{1}\sim\mathrm{Bern}(p_{01}/(p_{01}+p_{00}))$, and $Z_{2}\sim\mathrm{Bern}(p_{11}/(p_{11}+p_{10}))$, with $Z_{0},Z_{1}$ and $Z_{2}$ independent yields

[TABLE]

In the $(A,B)\sim\text{CorrER}(Q_{1},Q_{2},R)$ model, we then have that

[TABLE]

where

[TABLE]

are independent Bernoulli random variables.

A.1 Proof of Theorem 6

The key to the Proof of Theorem 6 is the well-known McDiarmid’s inequality [43].

Proposition 15** (McDiarmid’s inequality).**

Let $X_{1},\dotsc,X_{n}\in\mathcal{X}$ be a sequence of independent random variables. Let $f:\mathcal{X}^{n}\mapsto\mathbb{R}$ be such that for all $i\in[n]$, and for all $x_{1},\ldots,x_{n},x_{i}^{\prime}\in\mathcal{X}$

[TABLE]

If $Y=f(X_{1},\dotsc,X_{n})$, then for any $t>0$,

[TABLE]

Proof of Theorem 6: Let $G_{1},G_{2}\sim\text{CorrER}(Q_{1},Q_{2},R)$, and consider $\widetilde{A}=A-\mathbb{E}(B)$ and $\widetilde{B}=B-\mathbb{E}(B)$.

If $X$ and $Y$ are $\varrho$-correlated Bernoulli random variables with respective parameters $p$ and $q$, then it follows that

[TABLE]

Let $P\in\Pi(n)$ be a permutation matrix that permutes exactly $k$ labels and let $\tau$ be the associated permutation for $P$. Note that

[TABLE]

satisfies

[TABLE]

where

[TABLE]

is the number of transpositions induced by $P$. Note that $|T_{P}|\leq k/2$, and so

[TABLE]

For each $Q\in\Pi(n)$, define the matrix $\mathcal{K}_{A,B,Q}$ as in Eq. 4. We have then

[TABLE]

For ease of notation, we define

[TABLE]

so that

[TABLE]

As $X_{P}=\sum_{\{u,v\}\in\mathcal{E}_{P}}\widetilde{A}(u,v)\left(\widetilde{B}(u,v)-\widetilde{B}(\tau(u),\tau(v))\right)$, and each $\{A(u,v),B(u,v)\}$ pair is a function of three independent Bernoulli random variables, we have that $X_{P}$ is a function of

[TABLE]

and so is a function of $N_{P}:=3|\mathcal{E}_{P}|$ independent Bernoulli random variables, where $N_{P}$ then satisfies

[TABLE]

Changing the value of one of these Bernoulli random variables leaving all others fixed can change the value of at most one $\{A(u,v),B(u,v)\}$ pair, and this pair appears in two terms in the sum $X_{P}$. As each term in the sum of $X_{P}$ is bounded in $[-2,2]$, we have that, in the notation of Proposition 15, each $c_{i}$ can be uniformly set to $8$. Proposition 15 then yields (setting $t=\mathcal{X}_{P}$)

[TABLE]

with $C$ being an appropriate positive constant that may change line to line. If $\mathcal{X}_{P}=\omega(k\sqrt{n\log(n)})$ then

[TABLE]

To finish the proof, we apply a union bound on all such $P$. The number of such permutations $P$ that permute $k$ vertex labels is upper bounded by $n^{k}$. Combining with Eq. (18), we have that under the assumptions of the Theorem

[TABLE]

as desired.

A.2 Proof of Theorem 9

Key to the proof of Theorem 9 are the following Lemmas, adapted here from [55, Lemmas 1 and 2].

Lemma 16**.**

Let $X,Y\in\mathbb{R}^{n\times n}$. Suppose $\tau\geq(1+\delta)\|X-Y\|$ for $\delta>0$ fixed. Let $X=\sum_{i}\sigma_{i}(X)u_{i}v_{i}^{T}$ be the singular value decomposition of $X$, and let

[TABLE]

Then

[TABLE]

where $\sigma_{i}(Y)\geq\sigma_{i}(Y)\geq\cdots\geq\sigma_{i}(Y)$ are the singular values of $Y$.

Lemma 17**.**

Let $X\sim$ER($Q$); i.e., $X$ is a hollow, symmetric matrix with $X(u,v)\stackrel{{\scriptstyle ind.}}{{\sim}}Q(u,v)$. Assume $Q\leq r$ for some $r>0$. If $nr\geq C\log n$ for a constant $C>0$, then for all $c>0$ there exists a constant $\kappa>0$ such that

[TABLE]

Note first that

[TABLE]

with the analogous result holding for $B$. Under the assumptions of Theorem 9, Lemma 17 implies that there exists constants $c_{1},c_{2}>0$ such that

[TABLE]

with probability at least $1-n^{-2}$, and therefore there exists constants $c^{\prime}_{1},c^{\prime}_{2}>0$ such that

[TABLE]

with probability at least $1-n^{-2}$.

Next, we apply Lemma 16 with $X=A$, $Y=Q_{1}$ (resp., $X=B$, $Y=Q_{2}$). With probability at least $1-n^{-2}$, there exists constants $a_{1},C_{1}>0$ such that if $\tau=a_{1}\sqrt{nr_{1}}$ then (where $\breve{A}$ is as defined in the USVT pseudocode, Algorithm 1)

[TABLE]

where the equality follows from the rank assumption on $Q_{1}$ in Theorem 9; similarly, for $B$ we have

[TABLE]

Combining the above, we have that there exists an event $\mathcal{E}_{1}$ such that $\mathbb{P}(\mathcal{E}_{1})\geq 1-n^{-2}$, and on $\mathcal{E}_{1}$,

[TABLE]

To prove Theorem 9, we proceed as follows. Fix $P\in\Pi(n)$ so that $\sum_{i}P(i,i)=n-k$; i.e., $P$ permutes exactly $k$ labels. Two simple applications of the triangle inequality yields that

[TABLE]

and

[TABLE]

Combining the above, we have that

[TABLE]

In the proof of Theorem 6, if we set $t=\mathcal{X}_{P}/2$ in Eq. 17 when applying McDiarmid’s inequality, then under the assumptions of Theorem 9, there exists an event $\mathcal{E}_{2,P}$ with $\mathbb{P}(\mathcal{E}_{2,P})\geq 1-2\text{exp}\{-\omega(k\log n)\}$ such that on $\mathcal{E}_{2,P}$,

[TABLE]

Next, note that

[TABLE]

Hoeffding’s inequality (see, for example, [12]) yields that

[TABLE]

with probability at least

[TABLE]

(where the last inequality followed from the assumptions in the Theorem, as under the assumptions $nr_{i}=\omega(\log n)$). Therefore there exists an event $\mathcal{E}_{3}$ such that Eq. (19) and (20) hold on $\mathcal{E}_{3}$, and $\mathbb{P}(\mathcal{E}_{3})\geq 1-2e^{-\omega(\log^{2}n)}.$

Writing

[TABLE]

we see that

[TABLE]

We see then that on $\mathcal{E}_{1}\cap\mathcal{E}_{2,P}\cap\mathcal{E}_{3}$,

[TABLE]

Let $k_{0}$ be such that

[TABLE]

If $P\in\Pi(n,k)$ for $k\geq k_{0}$, $\|\widehat{A}-P\widehat{B}P^{T}\|_{F}-\|\widehat{A}-\widehat{B}\|_{F}>0$ on $\mathcal{E}_{1}\cap\mathcal{E}_{2,P}\cap\mathcal{E}_{3}$ where

[TABLE]

Define the event

[TABLE]

and note that

[TABLE]

Combined, this yields

[TABLE]

as desired.

A.3 Proof of Theorem 12

For a given permutation $\tau$ on $[n]$, we define the permutation $\tau_{\mathrm{id}}$ uniquely as follows:

[TABLE]

For example, if $n_{c}=4$, $n_{j}=4$, and

[TABLE]

then

[TABLE]

For a permutation matrix $P$, we define $P_{\mathrm{id}}\in\mathcal{P}_{n}^{\mathcal{C}}$ analogously, where we recall here that $\mathcal{P}_{n}^{\mathcal{C}}$ is the set of permutation matrices $P$ in $\Pi(n)$ satisfying $\sum_{i=1}^{n_{c}}P(i,i)=n_{c}$ (i.e., fixing all core labels). Define $\mathcal{O}_{n}^{\mathcal{C}}=\Pi_{n}\setminus\mathcal{P}_{n}^{\mathcal{C}}$. Define the events

[TABLE]

$\mathcal{B}_{1}$ is the event that the optimal GMP permutation is not in $\mathcal{P}_{n}^{\mathcal{C}}$ and the graphs are not core $\delta$-matchable for $\delta(A,PBP^{T})=\|A-\mathbb{E}(A)-P(B-\mathbb{E}(B))P^{T}\|_{F}$. As $Q_{\mathrm{id}}\in\mathcal{P}_{n}^{\mathcal{C}}$, we have that $\mathcal{B}_{1}\subset\mathcal{B}_{2}$.

Suppose that $Q\in\mathcal{O}_{n}^{\mathcal{C}}$ (with corresponding permutation $\tau\in S(n)$) permutes $k_{c}+k_{j}>0$ core labels, where

[TABLE]

Applying the results in Appendix A on the Bivariate Bernoulli distribution, we see that $\|\widetilde{A}Q-Q\widetilde{B}\|^{2}_{F}-\|\widetilde{A}Q_{\mathrm{id}}-Q_{\mathrm{id}}\widetilde{B}\|^{2}_{F}$ is a function of $N_{P}$ independent Bernoulli random variables, where

[TABLE]

As in the proof of Theorem 6, we next apply Proposition 15 to bound the probability that $Q$ provides a better matching than $Q_{\text{id}}$. By the assumption that $R(u,v)=0$ if either $u$ or $v$ is a junk vertex, it holds that

[TABLE]

To ease notation, we define $X_{Q}:=\frac{1}{2}\left(\|\widetilde{A}Q-Q\widetilde{B}\|^{2}_{F}-\|\widetilde{A}Q_{\mathrm{id}}-Q_{\mathrm{id}}\widetilde{B}\|^{2}_{F}\right)$, so that

[TABLE]

To use a union bound, note that the number of permutations $Q\in\mathcal{O}_{n}^{\mathcal{C}}$ with error counts in Eqs. (24)–(25) given by $k_{c}$ and $k_{j}$ is bounded above by $n_{c}^{2(k_{c}+k_{j})}n_{j}^{2n_{j}}$. Let $k:=k_{c}+k_{j}$ be the number of core vertices permuted by $Q$. Hence, if $n_{c}\geq n_{j}$ and

[TABLE]

then we have that

[TABLE]

as desired.