Average whenever you meet: Opportunistic protocols for community detection

TL;DR

This paper introduces opportunistic, averaging-based protocols for community detection in graphs, demonstrating their effectiveness in revealing community structure with minimal local computation and communication.

Contribution

It provides the first analysis of averaging protocols for community detection in opportunistic, asynchronous settings, including first- and second-moment analyses for various graph models.

Findings

Values reflect community structure after O(n log n) rounds

Most nodes can identify their community efficiently in regular graphs

Protocols recover community structure with polylogarithmic work per node

Abstract

Consider the following asynchronous, opportunistic communication model over a graph $G$: in each round, one edge is activated uniformly and independently at random and (only) its two endpoints can exchange messages and perform local computations. Under this model, we study the following random process: The first time a vertex is an endpoint of an active edge, it chooses a random number, say $\pm 1$ with probability $1/2$; then, in each round, the two endpoints of the currently active edge update their values to their average. We show that, if $G$ exhibits a two-community structure (for example, two expanders connected by a sparse cut), the values held by the nodes will collectively reflect the underlying community structure over a suitable phase of the above process, allowing efficient and effective recovery in important cases. In more detail, we first provide a first-moment analysis showing that, for a large class of almost-regular clustered graphs that includes the stochastic block model, the expected values held by all but a negligible fraction of the nodes eventually reflect the underlying cut signal. We prove this property emerges after a mixing period of length $\mathcal O(n\log n)$. We further provide a second-moment analysis for a more restricted class of regular clustered graphs that includes the regular stochastic block model. For this case, we are able to show that most nodes can efficiently and locally identify their community of reference over a suitable time window. This results in the first opportunistic protocols that approximately recover community structure using only polylogarithmic work per node. Even for the above class of regular graphs, our second moment analysis requires new concentration bounds on the product of certain random matrices that are technically challenging and possibly of independent interest.

Equations

$$\Delta({\bf h}_{u},{\bf h}_{v})\left\{\begin{array}[]{cll}\leqslant c_{1}m&\mbox{ if }i_{u}=i_{v}&\mbox{\emph{(Case (i))},}\\[5.69054pt] \geqslant c_{2}m&\mbox{ otherwise}&\mbox{\emph{(Case (ii))},}\end{array}\right.$$

$$W_{t}(i,j)=\left\{\begin{array}[]{cl}0&{\mbox{ if }i\neq j\mbox{ and }\{i,j\}\mbox{ is not sampled (at round $t$),}}\\[5.69054pt] 1/2&\begin{array}[]{l}{\mbox{if $i=j$ and an edge with endpoint $i$ is sampled}}\\ {\mbox{or $i\neq j$ and edge $\{i,j\}$ is sampled,}}\end{array}\\[8.53581pt] 1&{\mbox{ if }i=j\mbox{ and }i\mbox{ is not an endpoint of sampled edge.}}\end{array}\right.$$

$$\overline{W}:=\mathbb{E}\left[W_t\right]=I-\frac{1}{2m}L,$$

$$\frac{d}{2m}(1-2\gamma )\left({\lambda }_3-{\lambda }_2\right)⩽{\bar{\lambda }}_2-{\bar{\lambda }}_3⩽\frac{d}{2m}(1+2\gamma )\left({\lambda }_3-{\lambda }_2\right)$$

$$\mathbb{E}\left[{\mathbf{x}}_u^{(t)}\right]={\alpha }_1+{\alpha }_2{\bar{\lambda }}_2^t\left[{\mathbf{\chi }}_u+\sqrt{n}{\mathbf{f}}_{\perp ,u}\right]+{\mathbf{e}}_u^{(t)},$$

$$\mathbf{sgn}\left(\mathbb{E}\left[{\mathbf{x}}_u^{(t-1)}\right]-\mathbb{E}\left[{\mathbf{x}}_u^{(t)}\right]\right)=\mathbf{sgn}\left({\mathbf{\chi }}_u\right)$$

$$\mathbf{sgn}\left(\mathbb{E}\left[{\mathbf{x}}_u^{(t)}\right]\right)=\mathbf{sgn}({\alpha }_2{\mathbf{\chi }}_u)$$

$$t⩾3\log \left(\frac{n}{1-\epsilon }\right)/\log ({\bar{\lambda }}_2/{\bar{\lambda }}_3).$$

$$\mathbb{P}\left[{\mathbf{h}}^{\text{mbox}jump,(t)}=\mathbf{sgn}({\alpha }_2{\mathbf{\chi }}_u)\right]=\Omega (1),$$

$$\frac{1}{\log (1/{\bar{\lambda }}_3)}\log (n/|{\alpha }_1|)⩽t⩽\frac{1}{\log (1/{\bar{\lambda }}_2)}\log \left(\frac{|{\alpha }_2|(1-\epsilon )}{2|{\alpha }_1|}\right).$$

$$\overline{W}=\left(1-\frac{1}{n}\right)I+\frac{1}{n}P=I-\frac{1}{n}\mathcal{L}$$

$${\mathbf{x}}^{(t)}={\mathbf{x}}_{\Vert }+{\mathbf{y}}^{(t)}+{\mathbf{z}}^{(t)}.$$

$$\mathbb{E}\left[{\Vert {\mathbf{y}}^{(t)}+{\mathbf{z}}^{(t)}-{\mathbf{y}}^{(0)}\Vert }^2\right]⩽\frac{3{\lambda }_{2}t}{n}.$$

$$({\mathbf{x}}_v^{(t)}-({\mathbf{x}}_{\Vert ,v}+{\mathbf{y}}_v^{(0)}){)}^2⩽\frac{{\epsilon }^2}{n}\Vert {\mathbf{y}}^{(0)}{\Vert }^2,$$

$$|B_t|⩽\frac{n}{{\epsilon }^2\Vert {\mathbf{y}}^{(0)}{\Vert }^2}\Vert {\mathbf{y}}^{(t)}+{\mathbf{z}}^{(t)}-{\mathbf{y}}^{(0)}{\Vert }^2.$$

$$\mathbb{P}\left[|B_t|>\epsilon n|{\mathbf{x}}^{(0)}=\mathbf{x}\right]⩽\epsilon .$$

$$\mathbb{P}\left[|B_t|⩽3\epsilon \cdot n,\forall t:6\frac{n}{{\lambda }_3}\log n⩽t⩽12\frac{n}{{\lambda }_3}\log n\right]⩾1-\epsilon .$$

$${\mathbb{P}}_{{\mathbf{x}}^{(0)},\mathcal{E}}\left[\Vert {\mathbf{z}}^{(t)}{\Vert }^2⩽\sqrt{\frac{\delta b}{d({\lambda }_3-{\lambda }_2)}}\Vert {\mathbf{y}}^{(t)}{\Vert }^2\right]⩾1-\mathcal{O}\left(\sqrt[3]{\frac{\delta b}{d({\lambda }_3-{\lambda }_2)}}+\frac{1}{\sqrt{n}}\right).$$

$${\mathbb{P}}_{{\mathbf{x}}^{(0)},\mathcal{E}}\left[\Vert {\mathbf{z}}^{(t)}{\Vert }^2⩽\epsilon \Vert {\mathbf{y}}^{(t)}{\Vert }^2\right]⩾1-\epsilon .$$

$$\mathbb{E}[\Vert {\mathbf{y}}^{(t)}{\Vert }^2]⩽\left(1-\frac{4\delta {\lambda }_2}{n}+\frac{8{\delta }^2{\lambda }_2}{n^2}\right)\Vert {\mathbf{y}}^{(t-1)}{\Vert }^2+\left(\frac{8{\delta }^2{\lambda }_2}{n^2}\right)\Vert {\mathbf{z}}^{(t-1)}{\Vert }^2$$

$$\mathbb{E}[\Vert {\mathbf{z}}^{(t)}{\Vert }^2]⩽\left(\frac{4{\delta }^2{\lambda }_2}{n}\right)\Vert {\mathbf{y}}^{(t-1)}{\Vert }^2+\left(1-\frac{4\delta (1-\delta ){\lambda }_3}{n}\right)\Vert {\mathbf{z}}^{(t-1)}{\Vert }^2$$

$$\mathbb{E}[\Vert {\mathbf{y}}^{(t)}{\Vert }^2]⩽{\left(1-\frac{4\delta {\lambda }_2}{n}+\frac{8{\delta }^2{\lambda }_2}{n^2}\right)}^t\Vert {\mathbf{y}}^{(0)}{\Vert }^2$$

$$\mathbb{E}[\Vert {\mathbf{z}}^{(t)}{\Vert }^2]⩽{\left(1-\frac{4\delta (1-\delta ){\lambda }_3}{n}\right)}^t\Vert {\mathbf{z}}^{(0)}{\Vert }^2.$$

$$\mathbb{E}[{\mathbf{y}}^{(t)}]={\left(1-\frac{2\delta {\lambda }_2}{n}\right)}^t{\mathbf{y}}^{(0)}.$$

$$\text{Var}(a_y(t))$$

$$={\mathcal{O}}_{\delta ,{\lambda }_2}(t/n^2){\left(\mathbb{E}[a_y(t)]\right)}^2.$$

$$\mathbb{P}\left[|\mathbf{x}\cdot 1\pm \mathbf{x}\cdot \mathbf{\chi }|⩾\epsilon \cdot \sqrt{n}\right]⩾1-\mathcal{O}(\epsilon )\text{and}$$

$$\mathbb{P}\left[|\mathbf{x}\cdot 1|⩽|\mathbf{x}\cdot \mathbf{\chi }|-\epsilon \cdot \sqrt{n}||\mathbf{x}\cdot 1\pm \mathbf{x}\cdot \mathbf{\chi }|⩾\epsilon \cdot \sqrt{n}\right]=\frac{1}{2}.$$

$$\mathbb{P}(‘‘|\mathbf{x}\cdot 1+\mathbf{x}\cdot \mathbf{\chi }|⩾\epsilon \sqrt{n}"\wedge ‘‘|\mathbf{x}\cdot 1-\mathbf{x}\cdot \mathbf{\chi }|⩾\epsilon \sqrt{n}")⩾1-\frac{4}{\sqrt{2\pi }}\epsilon .$$

$$w_{i,j}=\left\{\begin{array}[]{cl}0&\mbox{ if }i\neq j\mbox{ and }\{i,j\}\mbox{ not sampled}\\ 1/2&\mbox{ if $i=j$ and some edge incident on $i$ sampled or $i\neq j$ and edge $\{i,j\}$ sampled}\\ 1&\mbox{ if }i=j\mbox{ and }i\mbox{ not incident to a sampled edge. }\end{array}\right.$$

Full text

In more detail, we first provide a first-moment analysis showing that, for a large class of almost-regular clustered graphs that includes the stochastic block model, the expected values held by all but a negligible fraction of the nodes eventually reflect the underlying cut signal. We prove this property emerges after a “mixing” period of length $\mathcal{O}(n\log n)$. We further provide a second-moment analysis for a more restricted class of regular clustered graphs that includes the regular stochastic block model. For this case, we are able to show that most nodes can efficiently and locally identify their community of reference over a suitable time window. This results in the first opportunistic protocols that approximately recover community structure using only logarithmic (or polylogarithmic, depending on the sparsity of the cut) work per node. Even for the above class of regular graphs, our second moment analysis requires new concentration bounds on the product of certain random matrices that are technically challenging and possibly of independent interest.

Keywords: Distributed Community Detection, Asynchronous Protocols, Random Processes, Spectral Analysis.

1 Introduction

Consider the following, elementary distributed process on an undirected graph $G=(V,E)$ with $|V|=n$ nodes and $|E|=m$ edges. Each node $v$ holds a real number $x_{v}$ (which we call the state of node $v$); at each time step, one random edge $\{u,v\}$ becomes active and its endpoints $u$ and $v$ update their states to their average.

Viewed as a protocol, the above process is consistent with asynchronous, opportunistic communication models, such as those considered in [AAER07] for population-protocols; here, in every round, one edge is activated uniformly and independently at random and (only) its two endpoints can exchange messages and perform local computations in that round111In an essentially equivalent continuous-time model, each edge has a clock that ticks at random intervals with a Poisson distribution of average 1; when the clock ticks, then the edge endpoints become activated. For $t$ larger than $n\log n$, the behavior of the continuous time process for $t/n$ units of time and the behavior of the discrete-time process for $t$ steps are roughly equivalent.. We further assume no global clock is available (nodes can at most count the number of local activations) and that the network is anonymous, i.e., nodes are not aware of theirs or their neighbors’ identities and all nodes run the same process at all times.

The long-term behavior of the process outlined above is well-understood: assuming $G$ is connected, for each initial global state ${\bf x}\in{\mathbb{R}}^{V}$ the system converges to a global state in which all nodes share a common value, namely, the average of their initial states. A variant of an argument of Boyd et al. [BGPS06] shows that convergence occurs in $\mathcal{O}\left(\frac{1}{\lambda_{2}}n\log n\right)$ steps, where $\lambda_{2}$ is the second smallest eigenvalue of the normalized Laplacian of $G$.

Suppose now that $G$ is well-clustered, i.e. it exhibits a community structure which in the simplest case consists of two equal-sized expanders, connected by a sparse cut: This structure arises, for instance, when the graph is sampled from the popular stochastic block model [MNS16] $\mathcal{G}_{n,p,q}$ for $p\gg q$ and $p\geqslant\log n/n$. If we let the averaging process unfold on such a graph, for example starting from an initial $\pm 1$ random global state, one might reasonably expect a faster, transient convergence toward some local average within each community, accompanied by a slower, global convergence toward the average taken over the entire graph. If, as is likely the case, a gap exists between the local averages of the two communities, the global state during the transient phase would reflect the graph’s underlying community structure. This intuition suggests the main questions we address in this paper:

Is there a phase in which the global state carries information about community structure? If so, how strong is the corresponding “signal”? Finally, can nodes leverage local history to recover this information?

The idea of using averaging local rules to perform distributed community detection is not new: In [BCN*+*17], Becchetti et al. consider a deterministic dynamics in which, at every round, each node updates its local state to the average of its neighbors. The authors show that this results in a fast clustering algorithm with provable accuracy on a wide class of almost-regular graphs that includes the stochastic block model. We remark that the algorithm in [BCN*+*17] works in a synchronous, parallel communication model where every node exchanges data with all its neighbors in each round. This implies considerable work and communication costs, especially when the graph is dense. On the other hand, each step of the process is described by the same matrix and its evolution unfolds according to the power of this matrix applied to the initial state. In contrast, the averaging process we consider in this paper is considerably harder to analyze than the one in [BCN*+*17], since each step is described by a random, possibly different averaging matrix.

Differently from [BCN*+*17], our goal here is the design of simple, lightweight protocols for fully-decentralized community detection which fit the asynchronous, opportunistic communication model, in which a (random) link activation represents an opportunistic meeting that the endpoints can exploit to exchange one-to-one messages. More specifically, by “lightweight” we mean protocols that require minimalistic assumptions as to network capabilities, while performing their task with minimal work, storage and communication per node (at most logarithmic or polylogarithmic in our case). In this respect, any clustering strategies (like the one in [SZ17]) which construct (and then work over) some static, sparse subgraph of the underlying graph are unfeasible in the opportunistic model we consider here. This restrictive setting is motivated by network scenarios in which individual agents need to autonomously and locally uncover underlying, implicit communities of which they are members. This has widespread applicability, for example in communication systems where lightweight data can be locally shared via wireless opportunistic meetings when agents come within close range [WWA12].

We next discuss what it means to recover the “underlying community structure” in a distributed setting, a notion that can come in stronger or weaker flavors. Ideally, we would like the protocol to reach a state in which, at least with high probability, each node can use a simple rule to assign itself one of two possible labels, so that labelling within each community is consistent and nodes in different communities are assigned different labels. Achieving this corresponds to exact (block) reconstruction. The next best guarantee is weak (block) reconstruction (see Definition 2.3). In this case, with high probability the above property is true for all but a small fraction of misclassified nodes. In this paper, we introduce a third notion, which we call community-sensitive labeling (CSL for short): in this case, there is a predicate that can be applied to pairs of labels so that, for all but a small fraction of outliers, the labels of any two nodes within the same community satisfy the predicate, whereas the converse occurs when they belong to different communities222Note that a weak reconstruction protocol entails a community-sensitive labeling. In this case, the predicate is true if two labels are the same.. In this paper, informally speaking, nodes are labelled with binary signatures of logarithmic length, while two labels satisfy the predicate whenever their Hamming distance is below a certain threshold. This introduces a notion of similarity between nodes of the graph, with labels behaving like profiles that reflect community membership333Hence the phrase community-sensitive Labeling we use to refer to our approach.. Note that this weaker notion of community-detection allows nodes to locally tell “friends” in their community from “foes” in the other community, which is the main application of distributed community detection in the opportunistic setting we consider here.

1.1 Our results

First moment analysis. Our first contribution is an analysis of the expected evolution of the averaging process over a wide class of almost-regular graphs (see Definition 2.1) that possess a hidden and balanced partition of the nodes with the following properties: (i) The cut separating the two communities is sparse, i.e., it contains $o(m)$ edges; (ii) the subgraphs induced by the two communities are expanders, i.e., the gap $\lambda_{3}-\lambda_{2}$ between the third and the second eigenvalues of the normalized Laplacian matrix $\mathcal{L}$ of the graph is constant. The above conditions on the underlying graph are satisfied, for instance, by graphs sampled from the stochastic block model444See Subsection 1.2 for the definition of $\mathcal{G}_{n,p,q}$ and for more details about our results for $\mathcal{G}_{n,p,q}$. $\mathcal{G}_{n,p,q}$ for $q=o(p)$ and $p\geqslant\log n/n$.

Let $L=D-A$ be the Laplacian matrix of $G$. The first moment analysis considers the deterministic process described by the linear equation ${\bf x}^{(t+1)}=\overline{W}^{t}\cdot{\bf x}^{(0)}$ ($t\geqslant 1$), where ${\bf x}^{(0)}={\bf x}$ is the vector with components the nodes’ initial random values and $\overline{W}:=\mathbb{E}\!\left[W\right]=I-\frac{1}{2m}L$ is the expectation of the random matrix that describes a single step of the averaging process. While a formal proof of the above equation can be found in Section 3, our analysis reveals that the expected values held by the nodes are correlated with the underlying cut. This phenomenon follows from structural connections between the underlying graph’s community structure and some spectral properties of $\overline{W}$. This allows us to show that, after an initial “mixing” phase of $\Theta(n\log n)$ rounds and for all but $o(n)$ nodes, the following properties hold: (i) There exists a relatively large time window in which the signs of the expected values of nodes are correlated with the community they belong to. (ii) The expected values of nodes belonging to one of the communities increase in each round, while those of nodes in the other community decrease.

The formal statements of the above claims can be found in Theorem 3.1. Here, we note that these results suggest two different local criteria for community-sensitive labeling: (i) According to the first one, every node uses the sign of its own state within the aforementioned time window to set the generic component of its binary label (we in fact use independent copies of the averaging process to get binary labels of logarithmic size - see Protocol Sign-Labeling in Section 4.2). (ii) According to the second criterion, every node uses the signs of fluctuations of its own value along consecutive rounds to set the generic component of its binary label (see Protocol Jump-Labeling in Section 5.2) 555Having a node set its label within the correct time window is technically challenging in the asynchronous, opportunistic communication model we consider. This issue is briefly discussed in this and the following sections and formally addressed in Appendix C.4)..

The above analysis describes the “expected” behaviour of the averaging process over a large class of well-clustered graphs, at the same time showing that our approach might lead to efficient, opportunistic protocols for block reconstruction. Yet, designing and analyzing protocols with provable, high-probability guarantees, requires addressing the following questions:

Do realizations of the averaging process approximately follow its expected behavior with high, or even constant, probability? 2. 2.

If this is the case, how can nodes locally and asynchronously recover the cut signal, let alone guess the “right” global time window?

Second moment analysis. The first question above essentially requires a characterization of the variance of the process over time, which turns out to be an extremely challenging task. The main reason for this is that a realization of the averaging process is now described by an equation of the form ${\bf x}^{(t)}=W_{t}\cdot.....\cdot W_{1}{\bf x}$, where the $W_{i}$’s are sampled independently and uniformly from some matrix distribution (see Eq. (1) in Section 3). Here, matrix $W_{i}$ “encodes” both the $i$-th edge selected for activation and the averaging of the values held by its endpoints at the end of the $(i-1)$-th step.

Not much is known about concentration of the products of identically distributed random matrices, but we are able to accurately characterize the class of regular clustered graphs. We point out that many of the technical results and tools we develop to this purpose apply to far more general settings than the regular case and may be of independent interest. In more detail, we are able to provide accurate upper bounds on the norm of ${\bf x}^{(t)}$’s projection onto the subspace spanned by the first and second eigenvector of $\overline{W}$ (see the proof’s outline of Theorem 3.1 and Lemma B.2) for a class of regular clustered graphs that includes the regular stochastic block model666See Subsection 1.2 for more details about our results for regular stochastic block models. [BCN*+*17, BDG*+*15, MNS14] - see Definition 2.2.

These bounds are derived separately for two different regimes, defined by the sparseness of the cut separating the two communities. Assuming a good inner expansion of the communities, the first concentration result concerns cuts of size $o(m/\log^{2}n)$ and it is given in Subsection 4.1 while, for the case of cuts of size up to $\alpha m$ for any $\alpha<1$, the obtained concentration results are described in Subsection D.1.

These bounds alone are not sufficient to prove accuracy of the clustering criteria, due to the asynchronous nature of the communication model we consider, whereby every node only updates its value once every $m/d$ rounds in expectation, where $d$ is the degree of the nodes.

Desynchronization. The variance analysis outlined in the previous paragraph ensures that, for any fixed step $t$, the actual states of a large fraction of the nodes are “close” to their expectations, with high probability. Unfortunately, the asynchrony of the model we consider does not allow to easily apply this result (e.g., using a union bound) to prove that most nodes will eventually label themselves within the right global window and in a way that is consistent with the graph’s community structure. Rather, we show that there exists a large fraction of non-ephemeral “good nodes” whose states remain close to their expectations over a suitable time-window. The technical form of the concentration bound and the relative time window again depend on the sparsity of the cut: See Definition 4.1 and Lemma 4.2 for sparse cuts and Theorem 5.1 for dense cuts, respectively.

Distributed community detection. We exploit the second moment analysis and the desynchronization above to devise two different opportunistic protocols for community detection on regular clustered graphs.

• In the case of sparse cuts (i.e. of size $o(m/\log^{2}n)$), the obtained bound on the variance of non-ephemeral nodes (see Lemma 4.2) holds over a time window that essentially equals the one “suggested” by our first moment analysis. This allows us to give rigorous bounds on the performance of the opportunistic Protocol Sign-Labeling based on the sign criterion (see Section 4.2). This “good” time-window begins after $\mathcal{O}(n\log n)$ rounds: So, if the underlying graph has dense communities and a sparse cut, nodes can collectively compute an accurare labeling before the global mixing time of the graph. For instance, if the cut is $\mathcal{O}(m/n^{\gamma})$, for some constant $\gamma<2$, our protocol is polynomially faster than the global mixing time. In more detail, we prove that, given any regular clustered graph with cut of size $o(m/\log^{2}n)$, Protocol Sign-Labeling performs community-sensitive labeling for $n-o(n)$ nodes within global time777The extra logarithmic factor is needed to let every node update each component of its $\Theta(\log n)$-size binary label, independently (see Subsection 4.2 for details). $\mathcal{O}(n\log^{2}n)$ and with work per node $\mathcal{O}(\log^{2}n)$, with high probability (see Theorem 4.3 and its Corollary 4.2 for formal and more general statements about the performances of the protocol). Importantly enough, the costs of our first protocol do not depend on the cardinality of the edge set $E$.

• The bound on the variance that allows us to adopt the sign-based criterion above does not hold when the cut is not sparse, i.e., whenever it is $\omega(m/\log^{2}n)$. For such dense cuts, we use a different bound on the variance of nodes’ values given in Theorem 5.1, which starts to hold after the global mixing time of the underlying graph and over a time window of length $\Theta(n^{2})$. In this case, the specific form of the concentration bound leads to adoption of the second clustering criterion suggested by our first moment analysis, i.e., the one based on monotonicity of the values of non-ephemeral nodes. To this aim, we consider a “lazy” version of the averaging process equipped with a local clustering criterion, whereby nodes use the signs of fluctuations of their own values along consecutive rounds to label themselves (see Protocol Jump-Labeling in Section 5.2). A restricted but relevant version of the algorithmic result we achieve in this setting can be stated as follows (see Theorem 5.3 and its corollaries for more general statements): Given any regular clustered graph consisting of two expanders as communities and a cut of size up to $\alpha m$ (for any $\alpha<1$), the opportunistic protocol Jump-Labeling achieves weak reconstruction for a fraction $(1-\varepsilon)n$ nodes (where $\varepsilon$ is an arbitrary positive constant), with high probability. The protocol converges within $\mathcal{O}(n\log n\,(\log^{2}n+m/m_{1,2}))$ rounds (where we named $m_{1,2}=|E(V_{1},V_{2})|$ the size of the cut) and every node performs $\mathcal{O}(\log^{2}n+(m/m_{1,2})\log n)$ work, with high probability. Notice that this second protocol achieves a stronger form of community detection than the first one, but it is less efficient, especially when the underlying graph is dense and exhibits a sparse cut (it requires to “wait” for the global mixing time of the graph). On the other hand, the two protocols have comparable costs in the parameter ranges they were designed for.888As for the fraction of outliers guaranteed by the two protocols, please see the technical discussion after Corollary 5.5.

1.2 Comparison to previous work

We earlier compared our results to those of [BCN*+*17]. The advantage of [BCN*+*17] is that their analysis achieves concentration over a class of graphs that are almost-regular and extends to the case of more than two communities. Furthermore, in graphs in which the indicator of the cut is an eigenvector of $\lambda_{2}$, the algorithm of [BCN*+*17] achieves exact reconstruction. On the other hand, as previously remarked, the advantage of our work over [BCN*+*17] is that, for the first time, it applies to the asynchronous opportunistic model and the communication cost per node does not depend on the degree of the graph, so it is much more efficient in dense graphs.

If we do not restrict to asynchronous and/or opportunistic protocols, recently, in [SZ17], Sun and Zanetti999Before the technical report in [SZ17], the same authors in [SZ16] presented a synchronous distributed algorithm able to perform approximate reconstruction with multiple communities. Sun and Zanetti then discovered a gap in their analysis (personal communication), and they retracted the claims in [SZ16]. introduced a synchronous, averaging-based protocol that first computes a fixed random subgraph of the underlying graph and then, working on this sparse subgraph, returns an efficient block-reconstruction for a wide class of almost-regular clustered graphs including the stochastic block model. We remark that, besides having no desynchronization issue to deal with, their second moment analysis uses Chernoff-like concentration bounds on some random submatrix (rather than a product of them, as in our setting) which essentially show that, under reasonable hypothesis, the signal of the cut can still be recovered from the corresponding sparse subgraph the algorithm works on.

Further techniques for community detection and spectral clustering exist, which are not based on averaging. In particular, Kempe and McSherry showed that the top $k$ eigenvectors of the adjacency matrix of the underlying graph can be computed in a distributed fashion [KM04]. These eigenvectors can then be used to partition the graph; in our settings, since we assume that the indicator of the cut is the second eigenvector of the graph, applying Kempe and McSherry’s algorithm with $k=2$ immediately reveals the underlying partition. Again, we note here that the downside of this algorithm is that it is synchronous and quite complex. In particular, the algorithm requires a computation of $Q_{1}{\bf x}$, for which $\lambda_{2}^{-1}\log n$ work per node is a bottleneck, while our first algorithm only requires $\lambda_{3}^{-1}\log n$ work per node, a difference that can become significant for very sparse cuts.

At a technical level, we note that our analysis establishes concentration results for products of certain i.d.d. random matrices; concentrations of such products have been studied in the ergodic theory literature [CPV93, Ips15], but under assumptions that are not met in our setting, and with convergence rates that are not suitable for our applications.

While we only focused on decentralized settings so far, we note that the question of community detection, especially in stochastic block models, has been extensively studied in centralized computational models [ABH14, CO10, DKMZ11, DF89, HLL83, JS98, McS01]. The stochastic block model offers a popular framework for the probabilistic modelling of graphs that exhibit good clustering or community properties. In its simplest version, the random graph $\mathcal{G}_{n,p,q}$ consists of $n$ nodes and an edge probability distribution defined as follows: The node set is partitioned into two subsets $V_{1}$ and $V_{2}$, each of size $n/2$; edges linking nodes belonging to the same partition appear in $E$ independently at random with probability $p=p(n)$, while edges connecting nodes from different partitions appear with probability $q=q(n)<p$. In the centralized setting, the focus of most studies on stochastic block models is on determining the threshold at which weak recovery becomes possible, rather than simplicity or running time of the algorithm (as most algorithms are already reasonably simple and efficient). After a remarkable line of work [DKMZ11, MNS14, Mas14, MNS13], such a threshold has now been precisely determined.

Calling $a=pn$ and $b=qn$, it is known [BCN*+*17] that graphs sampled from $\mathcal{G}_{n,p,q}$ satisfy (w.h.p.) the approximate regularity and spectral gap conditions required by our first moment analysis (i.e. Theorem 3.1) whenever $b=o(a)$ and $a=\Omega(\log n)$. Versions of the stochastic block model in which the random graph is regular have also been considered [MNS14, BDG*+*15]. In particular Brito et al. [BDG*+*15] show that strong reconstruction is possible in polynomial-time when $a-b>2\sqrt{a+b-1}$. As for these regular random graphs, we remark that our opportunistic protocol for sparse cut works whenever $a/b\geqslant\log^{2}n$ (see also [Bor15, BDG*+*15]), while our protocol for dense cuts works for any parameter $a$ and $b$ such that $a-b>2(1+\rho)\sqrt{a+b}$, where $\rho$ is any positive constant. Since it is (information-theoretically) impossible to reconstruct the graph when $a-b\leqslant\mathcal{O}(\sqrt{a+b})$ [MNS14], our result comes within a constant factor of this threshold.

1.3 Roadmap of the paper

After presenting some preliminaries in Section 2, the first moment analysis for almost-regular graphs is given in Section 3. The analysis of the variance of the averaging process in regular graphs for the case of sparse cuts and the analysis of the resulting sign-based protocol are described in Section 4. In Section 5, we address the case of dense cuts: Similarly to the previous section, we first give a second moment analysis and then show how to apply it to devise a suitable opportunistic protocol for this regime.

Due to the considerable length of this paper, most of the technical results are given in a separate appendix.

2 Preliminaries

We study the weighted version of the Averaging process described in the introduction. In each round, one edge of the graph is sampled uniformly at random and the two endpoints of the sampled edge execute the following algorithm.

For a graph $G$ with $n$ nodes and adjacency matrix $A$, let $0=\lambda_{1}\leqslant\cdots\leqslant\lambda_{n}$ be the eigenvalues of the normalized Laplacian $\mathcal{L}=I-D^{-1/2}AD^{-1/2}$, where $D$ is the diagonal matrix with the degrees of the nodes. We consider the following classes of graphs.

Definition 2.1** (Almost-regular graphs).**

An $(n,d,\beta)$-almost-regular graph101010This class is more general than the one introduced in [BCN*+*17], since there is no regularity constraint on the outer node degree, i.e., on the number of edges a node can have towards the other community. $G=(V,E)$ is a connected, non-bipartite graph over vertex set $V$, such that every node has degree $d\pm\beta d$.

Definition 2.2** (Clustered regular graphs).**

Let $n\geqslant 2$ be an even integer and $d$ and $b$ two positive integers such that $2b<d<n$. An $(n,d,b)$-clustered regular graph* $G=((V_{1},V_{2}),E)$ is a graph over node set $V=V_{1}\cup V_{2}$, with $|V_{1}|=|V_{2}|=n/2$ and such that: (i) Every node has degree $d$ and (ii) Every node in $V_{1}$ has $b$ neighbors in $V_{2}$ and every node in $V_{2}$ has $b$ neighbors in $V_{1}$.*

We remark that if a graph is clustered regular then we easily get that the indicator vector ${\bm{\chi}}$ of the cut $(V_{1},V_{2})$ is an eigenvector of $\mathcal{L}$ with eigenvalue $\frac{2b}{d}$; If we further assume that $\lambda_{3}>\frac{2b}{d}$, then ${\bm{\chi}}$ is an eigenvector of $\lambda_{2}$. We next recall the notion of weak reconstruction [BCN*+*17].

Definition 2.3** (Weak Reconstruction).**

*A function $f:V\rightarrow\{\pm 1\}$ is said to be an $\varepsilon$-weak reconstruction of $G$ if subsets $W_{1}\subseteq V_{1}$ and $W_{2}\subseteq V_{2}$ exist, each of size at least $(1-\varepsilon)n/2$, such that $f(W_{1})\cap f(W_{2})=\emptyset$. *

In this paper, we introduce a weaker notion of distributed community detection. Namely, let $\Delta(\mathbf{x},\mathbf{y})$ denote the Hamming distance between two binary strings $\mathbf{x}$ and $\mathbf{y}$.

Definition 2.4** (Community-sensitive labeling).**

Let $G=(V,E)$ be a graph, let $(V_{1},V_{2})$ be a partition of $V$ and let $\gamma\in(0,1]$. For some $m\in\mathbb{N}$, a function ${\bf h}\,:\,V_{1}\cup V_{2}\rightarrow\{0,1\}^{m}$ is a $\gamma$-community-sensitive labeling for $(V_{1},V_{2})$ if a subset $\tilde{V}\subseteq V$ with size $|\tilde{V}|\geqslant(1-\gamma)|V|$ and two constants $0\leqslant c_{1}<c_{2}\leqslant 1$ exist, such that for all $u,v\in\tilde{V}$ it holds that

[TABLE]

where $i_{u}=1$ if $u\in V_{1}$ and $i_{u}=2$ if $u\in V_{2}$.

3 First Moment Analysis

In this section, we analyze the expected behaviour of Algorithm Averaging$(1/2)$ on an almost-regular graph $G$ (see Definition 2.1). The evolution of the resulting process can be formally described by the recursion ${\bf x}^{(t+1)}=W_{t}\cdot{\bf x}^{(t)}$, where $W_{t}=(W_{t}(i,j))$ is the random matrix that defines the updates of the values at round $t$, i.e.,

[TABLE]

and the initial random vector ${\bf x}^{(0)}$ is uniformly distributed in $\{-1,1\}^{n}$.111111Notice that, since each node chooses value $\pm 1$ with probability $1/2$ the first time it is active, by using the principle of deferred decisions we can assume there exists an “initial” random vector ${\bf x}^{(0)}$ uniformly distributed in $\{-1,+1\}^{n}$.

Notice that random matrices $\{W_{t}\,:\,t\geqslant 0\}$ are independent and identically distributed and simple calculus shows that their expectation can be expressed as (see Observation A.2 in the Appendix):

[TABLE]

where $L=D-A$ is the Laplacian matrix of $G$. Matrix $\overline{W}$ is thus symmetric and doubly-stochastic. We denote its eigenvalues as $\bar{\lambda}_{1},\dots,\bar{\lambda}_{n}$, with $1=\bar{\lambda}_{1}\geqslant\bar{\lambda}_{2}\geqslant\cdots\bar{\lambda}_{n}\geqslant-1\,.$

We next provide a first moment analysis for $(n,d,\beta)$-almost regular graphs that exhibit a clustered structure. Our analysis proves the following results.

Theorem 3.1**.**

Let $G=(V,E)$ be an $(n,d,\beta)$-almost regular graph $G=(V,E)$ with a balanced partition $V=(V_{1},V_{2})$ and such that: (i) The cut $E(V_{1},V_{2})$ is sparse, i.e., $m_{1,2}=|E(V_{1},V_{2})|=o(m)$; (ii) The gap $\lambda_{3}-\lambda_{2}=\Omega(1)$.121212In practice, this means that each of the subgraphs induced by community $V_{i}$ ($i=1,2$) is an expander. If nodes of $G$ execute Protocol Averaging then, with constant probability w.r.t. the initial random vector ${\bf x}^{(0)}\in\{-1,1\}^{n}$, after $\Theta(n\log n)$ rounds the following holds for all but $o(n)$ nodes: (i) The expected value of a node $u$ increases or decreases depending on the community it belongs to, i.e., $\mathbf{sgn}\left(\mathbb{E}\!\left[{\bf x}^{(t-1)}_{u}\,|\,{\bf x}^{(0)}\right]-\mathbb{E}\!\left[{\bf x}^{(t)}_{u}\,|\,{\bf x}^{(0)}\right]\right)=\mathbf{sgn}\left({\bm{\chi}}_{u}\right)$; (ii) Over a time window of length $\Omega(n\log n)$ the sign of the expected value of a node $u$ reflects the community $u$ belongs to, i.e., $\mathbf{sgn}\left(\mathbb{E}\!\left[{\bf x}^{(t)}_{u}\,|\,{\bf x}^{(0)}\right]\right)=\mathbf{sgn}\left(\alpha_{2}{\bm{\chi}}_{u}\right)$, for some $\alpha_{2}=\alpha_{2}({\bf x}^{(0)})$.

Proof of Theorem 3.1: An outline

The proof makes a black-box use of technical results that are rigorously given in Appendix B.1. We believe, these results are interesting in their own right, since they shed light on the evolution of the dynamics and its algebraic structure.

The hypotheses of Theorem 3.1 involve the eigenvalues of the normalized Laplacian matrix $\mathcal{L}$ of the graph, while the expected evolution of the process is governed by matrix $\overline{W}$ and its eigenvalues $1=\bar{\lambda}_{1}\geqslant\cdots\bar{\lambda}_{n}\geqslant-1$ (see Lemma B.1). However, $(n,d,\beta)$-almost regularity implies that the spectra of these two matrices are related. In particular, it is easy to see that, under the hypotheses of Theorem 3.1, we have (see Observation A.5 in the Appendix)

[TABLE]

In Lemma B.1, we decompose $\mathbb{E}\!\left[{\bf x}^{(t)}\right]$ into its components along the first two eigenvectors of $\overline{W}$ and into the corresponding orthogonal component ${\bf e}^{(t)}$ (note that $\overline{W}$ admits an orthonormal eigenvector basis since it is symmetric). We further decompose the component along the second eigenvector of $\overline{W}$ into its component parallel to the partition indicator vector ($\alpha_{2}\bar{\lambda}_{2}^{t}{\bm{\chi}}$) and into the corresponding orthogonal one ($\alpha_{2}\sqrt{n}\,\bar{\lambda}_{2}^{t}{\bf f}_{\perp}$). As a consequence, we can rewrite $\mathbb{E}\!\left[{\bf x}^{(t)}\right]$ as

[TABLE]

where $\left\|{\bf e}^{(t)}\right\|\leqslant\bar{\lambda}_{3}^{t}\sqrt{n}$. Hence, if $\alpha_{2}\neq 0$ and $\bar{\lambda}_{3}<\bar{\lambda}_{2}$, the term ${\bf e}^{(t)}_{u}$ becomes negligible w.r.t. the other two from some round $t$ onward. Moreover, for any node $u$ with ${\bf f}_{\perp,u}<1/\sqrt{n}$, $\mathbf{sgn}(\left[{\bm{\chi}}_{u}+\sqrt{n}\,{\bf f}_{\perp,u}\right])=\mathbf{sgn}({\bm{\chi}}_{u})$, i.e., ${\bf x}^{(t)}_{u}$ identifies the community $V_{h}$ node $u$ belongs to. Accordingly, we say a node $u\in[n]$ is $\varepsilon$-bad if it does not satisfy the above property (see Definition B.1).131313Consistently, a node is $\varepsilon$-good otherwise. Next, we derive an upper bound on the number of $\varepsilon$-bad nodes. To this purpose, we first prove an upper bound on the square norm of ${\bf f}_{\perp}$, as a function of the gap $\bar{\lambda}_{2}-\bar{\lambda}_{3}$ and the ratio between the size of the cut $m_{1,2}$ and the total number of edges in the graph (Lemma B.2). This easily implies an upper bound on the number of $\varepsilon$-bad nodes (Corollary B.1) as a function of the gap $\bar{\lambda}_{2}-\bar{\lambda}_{3}$. From (3) and the hypothesis $\lambda_{3}-\lambda_{2}=\Omega(1)$ the upper bound in Corollary B.1 turns out to be $\mathcal{O}(m_{1,2}/d)$, which in turn is $o(n)$ under the hypothesis $m_{1,2}=o(m)$.

These results and (4) imply the following conclusions for all $\varepsilon$-good nodes $u$:

(i) For all $t=\Omega(n\log n)$, the evolution of $\mathbb{E}\!\left[{\bf x}^{(t)}_{u}\right]$ along two consecutive rounds identifies the block $u$ belongs to (Lemma B.3), namely:

[TABLE]

(ii) If $|\alpha_{2}|$ is sufficiently larger than $|\alpha_{1}|$ and the second and third largest eigenvalues of $\overline{W}$ satisfy appropriate conditions, for all $t$ falling in a suitable time window, the sign of $\mathbb{E}\!\left[{\bf x}^{(t)}_{u}\right]$ identifies the community node $u$ belongs to (Lemma B.4), namely:

[TABLE]

Moreover, Lemma B.5 implies that the initial random vector ${\bf x}$ satisfies the hypotheses of Lemma B.3 w.h.p. (i.e., $\alpha_{2}({\bf x})\neq 0$ w.h.p.) and those of Lemma B.4 with constant probability (i.e., $|\alpha_{2}({\bf x})|\geqslant 2|\alpha_{1}({\bf x})|/(1-\varepsilon)$ with constant probability).

As a result, we can claim the following for any non-bad node $u$: if we consider the r.v. ${\bf h}^{\mbox{jump},(t)}_{u}=\mathbf{sgn}\left(\mathbb{E}\!\left[{\bf x}^{(t-1)}_{u}-{\bf x}^{(t)}_{u}\;|\;{\bf x}^{(0)}\right]\right)$, Lemma B.3 implies ${\bf h}^{\mbox{jump},(t)}_{u}=\mathbf{sgn}(\alpha_{2}{\bm{\chi}}_{u})$ w.h.p., for every $t$ such that

[TABLE]

Likewise, if we consider the r.v. ${\bf h}^{\mbox{sign},(t)}_{u}=\mathbf{sgn}\left(\mathbb{E}\!\left[{\bf x}^{(t)}_{u}\right]\right)$, Lemma B.4 implies

[TABLE]

for all $t$ such that

[TABLE]

Finally, note that under the hypothesis $\lambda_{3}-\lambda_{2}=\Omega(1)$, the lower bounds on $t$ in (5) and (6) are both $\mathcal{O}(n\log n)$.

4 Regular Graphs with a Sparse Cut

We next provide a second moment analysis of the $\mbox{\sc Averaging}(\delta)$ with $\delta=1/2$ on the class of $(n,d,b)$-clustered regular graphs (see Definition 2.2) when the cut between the two communities is relatively sparse, i.e., for $\lambda_{2}=2b/d=o(\lambda_{3}/\log n)$. This analysis is consistent with the “expected” clustering behaviour of the dynamics explored in the previous section and highlights clustering properties that emerge well before global mixing time, as we show in Section 4.2. In particular, the main analysis results are discussed in Section 4.1, while in Section 4.2, we describe how the above analysis in concentration can be exploited to get an opportunistic protocol for provably-good community-sensitive labeling.

4.1 Second moment analysis for sparse cuts

Restriction to $(n,d,b)$-clustered regular graphs simplifies the analysis of the Averaging dynamics. When $G$ is regular, $\overline{W}$ defined in (2) can be written as

[TABLE]

This obviously implies that $\overline{W}$ and $\mathcal{L}$ share the same eigenvectors, while every eigenvalue $\lambda_{i}$ of $\mathcal{L}$ corresponds to an eigenvalue $\bar{\lambda}_{i}=1-\lambda_{i}/n$ of $\overline{W}$. For $(n,d,b)$-clustered regular graphs, these facts and our preliminary remarks in Section 2 further imply $\bar{\lambda}_{2}=1-\lambda_{2}/n=1-2b/dn$ whenever $\lambda_{3}>\frac{2b}{d}$ while, very importantly, the partition indicator vector ${\bm{\chi}}$ turns out to be the eigenvector of $\overline{W}$ corresponding to $\bar{\lambda}_{2}$ (see (2)). As a consequence, the orthogonal component ${\bf f}_{\perp}$ in (4) is ${\mathbf{0}}$ in this case.

On the other hand, even in this restricted setting, our second moment analysis requires new, non-standard concentration results for the product of random matrices that apply to far more general settings and may be of independent interest.

For the sake of readability, we here denote by ${\bf y}^{(t)}=Q_{2}{\bf x}^{(t)}$ the component of the state vector in the eigenspace of the second eigenvalue of $\overline{W}$, while ${\bf z}^{(t)}=Q_{3\cdots n}{\bf x}^{(t)}$ denotes ${\bf x}^{(t)}$’s projection onto the subspace orthogonal to ${\mathbf{1}}$ and ${\bm{\chi}}$. If we also set ${\bf x}_{\|}=Q_{1}{\bf x}^{(0)}$, we can write:

[TABLE]

Notice that, by taking expectations in the equation above, we get (4) with $\mathbb{E}\!\left[{\bf y}^{(t)}\right]=\alpha_{2}\bar{\lambda}_{2}^{t}{\bm{\chi}}$ and $\mathbb{E}\!\left[{\bf z}^{(t)}\right]={\bf e}^{(t)}$.

Our analysis of the process induced by Averaging$(1/2)$ provides the following bound.

Theorem 4.1** (Second moment analysis).**

Let $G$ be an $(n,d,b)$-clustered regular graph with $\lambda_{2}=\frac{2b}{d}=o\left(\lambda_{3}/\log n\right)$. Then, for every $\frac{3n}{\lambda_{3}}\log n\leqslant t\leqslant\frac{n}{4\lambda_{2}}$ it holds that

[TABLE]

We prove Theorem 4.1 by bounding and tracking the lengths of the projections of ${\bf x}^{(t)}$ onto the eigenspace of $\lambda_{2}$ and onto the space orthogonal to ${\mathbf{1}}$ and ${\bm{\chi}}$, i.e. $\|{\bf y}^{(t)}\|^{2}$ and $\|{\bf z}^{(t)}\|^{2}$. Due to lack of space, the proof is deferred to Appendix C.1.

We here want just to remark that the only part using the regularity of the graph is the derivation of the upper bound on $\mathbb{E}\!\left[\|{\bf y}^{(t+1)}\|^{2}\right]$ (see Lemma C.2), in particular its second addend. This term arises from an expression involving the Laplacian of $G$, which is far from simple in general, but that very nicely simplifies in the regular case. We suspect that increasingly weaker bounds should be achievable as the graph deviates from regularity.

Theorem 4.1 gives an upper bound on the squared norm of the difference of the state vector at step $t$ with the state vector at step [math]. Corollary 4.1 below shows how such a global bound can be used to derive pointwise bounds on the values of the nodes.

Definition 4.1**.**

A node $v$ is $\varepsilon$-good at time $t$ if

[TABLE]

it is $\varepsilon$-bad otherwise. We define by $B_{t}$ the set of nodes that are $\varepsilon$-bad at time $t$: B_{t}=\{u:u\makebox{ is \varepsilon$-bad at time$t}\}.

Observe first that, by definition of $\varepsilon$-bad node and some counting argument, we can prove both the next inequality and the corollary below (see Appendix C.2 for their proofs)

[TABLE]

Corollary 4.1**.**

Assume $3\frac{n}{\lambda_{3}}\log n\leqslant t\leqslant 3c\frac{n}{\lambda_{3}}\log n$ for any absolute constant $c\geqslant 1$ and $\lambda_{2}/\lambda_{3}\leqslant\varepsilon^{4}/(4c\log n)$:

[TABLE]

The next lemma gives a bound on the number of nodes that are good over a relatively large time-window. This is the key-property that we will use to analyse the asynchronous protocol Sign-Labeling (see the next subsection and Lemma C.7).

Lemma 4.2** (Non-ephemeral good nodes).**

Let $\varepsilon>0$ be an arbitrarily small value, let $G$ be an $(n,d,b)$-clustered regular graph with $\frac{\lambda_{2}}{\lambda_{3}}\leqslant\frac{\lambda_{3}\varepsilon^{4}}{c\log^{2}n}$, for a large enough costant $c$. If we execute Averaging$(1/2)$ on $G$, it holds that

[TABLE]

4.1.1 Proof of Lemma 4.2: An overview

The main idea of the proof is to first show that with probability strictly larger than $1-\varepsilon$, the number of $\varepsilon$-good nodes is at least $n\cdot(1-\varepsilon/\log n)$ in every round $t\in[t_{1},2t_{1}]$. Theorem 4.1 already ensures this to be true in any given time step within a suitable window, but simply taking a union bound will not work, since we have $n\log n$ time steps and only a $1-\varepsilon$ probability of observing the desired outcome in each of them. We will instead argue about the possible magnitude of the change in $\|{\bf y}^{(t)}+{\bf z}^{(t)}-{\bf y}^{(0)}\|^{2}$ over time, assuming this quantity is small at time $6\frac{n}{\lambda_{3}}\log n$. We will then show that our argument implies that, with probability $1-\varepsilon$, at least $n-\varepsilon n$ nodes remain $\varepsilon$-good over the entire window $[6\frac{n}{\lambda_{3}}\log n,12\frac{n}{\lambda_{3}}\log n]$.

The full proof of Lemma 4.2 is given in Appendix C.3.

4.2 The Sign-Labeling protocol

Leveraging the results of Subsection 4.1, we next propose a simple, lightweight opportunistic protocol that provides community-sensitive labeling for graphs that exhibit a relatively sparse cut.

The algorithm, denoted as Sign-Labeling, adds a simple labeling rule to the Averaging$(1/2)$ process: Each node keeps track of the number of times it is activated. Upon its $T$-th activation, for a suitable $T=\Theta(\log n)$, the node uses the sign of its current value as a binary label. The above local strategy is applied to $\ell$ independent runs of Averaging$(1/2)$, so that every node is eventually assigned a binary signature of length $\ell$.

Algorithm Sign-Labeling achieves community-sensitive labeling (see Definition 2.4), as stated in the following theorem and corollary.

Theorem 4.3** (Community-sensitive labeling).**

Let $\varepsilon>0$ be an arbitrarily small value, let $G$ be an $(n,d,b)$-clustered regular graph with $\frac{\lambda_{2}}{\lambda_{3}}\leqslant\frac{\lambda_{3}\varepsilon^{4}}{c\log^{2}n}$, for a large enough constant $c$. Then, protocol Sign-Labeling $(T,\ell)$ with $T=(8/\lambda_{3})\log n$ and $\ell=10\varepsilon^{-1}\log n$ performs a $\gamma$-community-sensitive labeling of $G$ according to Definition 2.4 with $c_{1}=4\varepsilon$, $c_{2}=1/6$ and $\gamma=6\varepsilon$, w.h.p. The convergence time is $\mathcal{O}(n\ell\log n/\lambda_{3})$ and the work per node is $\mathcal{O}(\ell\log n/\lambda_{3})$, w.h.p.

Notice that, according to the hypothesis of Theorem 4.3, in order to set local parameters $T$ and $\ell$, nodes should know parameters $\varepsilon$ and $\lambda_{3}$ (in addition to a polynomial upper bound on the number of the nodes). However, it easy to restate it in a slightly restricted form that does not require such assumptions on what nodes know about the underlying graph.

Corollary 4.2**.**

Protocol Sign-Labeling $(80\log n,600\log n)$ performs a $(1/10)$-community-sensitive labeling, according to Definition 2.4 with $c_{1}=1/15$ and $c_{2}=1/6$, of any $(n,d,b)$-clustered regular graph $G$ with $\lambda_{3}\geqslant 1/10$ and $\lambda_{2}\leqslant 1/(c\log^{2}n)$ for a large enough constant $c$.

4.2.1 Proof of Theorem 4.3: An Overview

We here sketch the main arguments proving Theorem 4.3: Its full proof is deferred to Appendix C.4.

Lemma 4.2 essentially states that over a suitable time window of size $\Theta(n\log n)$, for all nodes $u$ but a fraction $\mathcal{O}\left(\varepsilon/\log n\right)$, we have $\mathbf{sgn}({\bf x}^{(t)}_{u})=\mathbf{sgn}({\bf x}_{\|,{u}}+{\bf y}^{(0)}_{u})))$. Recalling that ${\bf x}_{\|}$ and ${\bf y}^{(0)}$ respectively are ${\bf x}^{(0)}$’s projections along ${\bm{\chi}}/\sqrt{n}$ and ${\mathbf{1}}/\sqrt{n}$, this immediately implies that, with probability $1-\varepsilon$ and up to a fraction $\varepsilon$ of the nodes, $\mathbf{sgn}({\bf x}^{(t)}_{u})=\mathbf{sgn}({\bf x}^{(t)}_{v})$, whenever $u$ and $v$ belong to the same community and $t$ falls within the aforementioned window. As to the latter condition, we prove that each node labels itself within the right window with probability at least $1-1/n$.151515 It may be worth noting that $\mathbf{sgn}({\bf x}^{(t)}_{u})=\mathbf{sgn}({\bf x}^{(t)}_{v})$ for $u$ and $v$ belonging to the same community does not imply $\mathbf{sgn}({\bf x}^{(t)}_{u})\neq\mathbf{sgn}({\bf x}^{(t)}_{v})$ when they don’t. Moreover, $\mathbf{sgn}({\bf x}_{\|,{u}}+{\bf y}^{(0)}_{u})))=\mathbf{sgn}({\bm{\chi}}_{u})$, whenever ${\bf y}^{(0)}_{u}$ exceeds ${\bf x}_{\|,{u}}$ in modulus, which occurs with probability $1/2-o(1)$ from the (independent) Rademacher initialization. As a consequence, if we run $\ell$ suitably independent copies of the process (see Algorithm 2), the following will happen for all but a fraction $\mathcal{O}(\varepsilon)$ of the nodes: the signatures of two nodes belonging to the same community will agree on $\ell-o(1)$ bits, whereas those of two nodes belonging to different communities will disagree on $\Omega(\ell)$ bits, i.e., our algorithm returns a community-sensitive labeling of the graph.

5 Regular Graphs with a Dense Cut

In this section, we extend our study to the lazy averaging algorithm Averaging$(\delta)$ where $\delta<1/2$. Similar to the previous section, we assume that the underlying graph $G$ is an $(n,d,b)$-clustered regular graph and $\lambda_{3}>\lambda_{2}=2b/d$. However, this new analyses and the clustering protocol we derive from will work even for large (constant) $\lambda_{2}$, in contrast to those in Section 4 which only works for small $\lambda_{2}\ll 1/\log^{2}n$. The structure of this section is similar to the previous one. Indeed, in Subsection 5.1, we propose a second moment analysis of the Averaging$(\delta)$ for the above-mentioned regime of $\lambda_{2}$. Then, in Subsection 5.2, we exploit the analysis above to devise a protocol that guarantees a weak reconstruction for the underlying graph with arbitrarily-large constant probability and thus, by running independent “copies” of the protocol (so, similarly to the previous section), we easily obtain a community-sensitive labeling of the graph, with high probability.

5.1 Second moment analysis for large $\lambda_{2}$

Informally speaking, we show that, for an appropriate value of $\delta$ and any $t$ such that $\Omega(n\log n)\leqslant t\leqslant\mathcal{O}(n^{2})$, with large probability, the vector ${\bf y}^{(t)}+{\bf z}^{(t)}$ is almost parallel to $\chi$, i.e., $\|{\bf z}^{(t)}\|$ is much smaller than $\|{\bf y}^{(t)}\|$. A more precise statement is given below as Theorem 5.1. Note that, for brevity, we write $\mathcal{E}$ here to denote the sequence $\{(u_{t},v_{t})\}_{t\in\mathbb{N}}$ of the edges chosen by the protocol.

Theorem 5.1**.**

For any sufficiently large $n\in\mathbb{N}$, any161616Here 0.8 is arbitrary and can be changed to any constant less than 1. However, we pick an absolute constant here to avoid introducing another parameter to our theorem. $\delta\in(0,0.8(\lambda_{3}-\lambda_{2}))$ and any $t\in\left[\Omega\left(\frac{n}{\delta(\lambda_{3}-\lambda_{2})}\log\left(n/\delta\right)\right),\mathcal{O}\left(\frac{n^{2}}{\delta(\lambda_{3}-\lambda_{2})}\left(\frac{d(\lambda_{3}-\lambda_{2})}{\delta b}\right)^{2/3}\right)\right]$, we have

[TABLE]

Theorem 5.1 should be compared to Theorem 4.1: both assert that $\|{\bf y}^{(t)}\|$ is much larger than $\|{\bf z}^{(t)}\|$, but Theorem 5.1 works even when $\lambda_{2}$ is quite large whereas Theorem 4.1 only holds for $\lambda_{2}\ll 1/\log n$.

While the parameter dependencies in Theorem 5.1 may look confusing at first, there are mainly two cases that are interesting here. First, for any error parameter $\varepsilon$, we can pick $\delta$ depending only on $\varepsilon$ and $\lambda_{3}-\lambda_{2}$ in such a way that Theorem 5.1 implies that, with probability $1-\varepsilon$, $\|{\bf z}^{(t)}\|^{2}$ is at most $\varepsilon\|{\bf y}^{(t)}\|^{2}$, as stated below.

Corollary 5.1**.**

For any constant $\varepsilon>0$ and for any $\lambda_{3}>\lambda_{2}$, there exists $\delta$ depending only on $\varepsilon$ and $\lambda_{3}-\lambda_{2}$ such that, for any sufficiently large $n$ and for any $t\in[\Omega_{\varepsilon,\lambda_{3}-\lambda_{2}}(n\log n),\mathcal{O}(n^{2})]$, we have

[TABLE]

Another interesting case is when $\delta=1/2$ (i.e., we consider the basic averaging protocol). Recalling that $\lambda_{2}=2b/d$, observe that $\lambda_{2}$ appears in both the bound on $\|{\bf z}^{(t)}\|^{2}$ and the error probability. Hence, we can derive a similar lemma as the one above, but with $\lambda_{2}$ depending on $\varepsilon$ instead of $\delta$:

Corollary 5.2**.**

Fix $\delta=1/2$. For any constant $\varepsilon>0$, any1717170.7 here can be replaced by any constant larger than 0.5. $\lambda_{3}>0.7$, any sufficiently small $\lambda_{2}$ depending only on $\varepsilon$, any sufficiently large $n$ and any $t\in[\Omega_{\varepsilon}(n\log n),\mathcal{O}(n^{2})]$, we have

[TABLE]

5.1.1 Proof of Theorem 5.1: An Overview

Due to space constraint, the full proof of Theorem 5.1 is deferred to Appendix D. We provide a brief summary of the ideas behind the proof here. Compared to the proof of Theorem 4.1, the main additional technical challenge in the new proof is to show that $\|{\bf y}^{(t)}\|$ is large with reasonably high probability. In Theorem 4.1, this is true because $\lambda_{2}$ is so small that ${\bf y}^{(t)}$ remains almost unchanged from ${\bf y}^{(0)}$. However, in the setting of large $\lambda_{2}$, this is not true anymore; for constant $\lambda_{2}$, even $\mathop{\mathbb{E}}[{\bf y}^{(t)}]$ shrinks by a constant factor from ${\bf y}^{(0)}$ when $t\geqslant\Omega_{\lambda_{2}}(n)$.

As a result, we need to develop a more fine-grained understanding of how $\|{\bf y}^{(t)}\|$, $\|{\bf z}^{(t)}\|$ changes over time. Specifically, at the heart of our analysis lies the following lemma181818Lemma 5.2 with its full statement and proof is given in Appendix D as Lemma D.2. Recall that $\lambda_{2}=2b/d.$ which allows us to understand how $\|{\bf y}^{(t)}\|$, $\|{\bf z}^{(t)}\|$ behave, given $\|{\bf y}^{(t-1)}\|$, $\|{\bf z}^{(t-1)}\|$:

Lemma 5.2**.**

For any $t\in\mathbb{N}$,

[TABLE]

and

[TABLE]

where the expectation is over the random edge selected at time $t$.

For simplicity of the overview, let us pretend that the cross terms were not there, i.e., that $\mathop{\mathbb{E}}[\|{\bf y}^{(t)}\|^{2}]\leqslant\left(1-\frac{4\delta\lambda_{2}}{n}+\frac{8\delta^{2}\lambda_{2}}{n^{2}}\right)\|{\bf y}^{(t-1)}\|^{2}$ and $\mathop{\mathbb{E}}[\|{\bf z}^{(t)}\|^{2}]\leqslant\left(1-\frac{4\delta(1-\delta)\lambda_{3}}{n}\right)\|{\bf z}^{(t-1)}\|^{2}$. These imply that

[TABLE]

and

[TABLE]

Now, by Markov’s inequality, (11) implies that, with 0.99 probability, $\|{\bf z}^{(t)}\|$ is at most $\mathcal{O}\left(\left(1-\frac{2\delta(1-\delta)\lambda_{3}}{n}\right)^{t}\|{\bf z}^{(0)}\|\right)$. However, it is not immediately clear how (10) can be used to lower bound $\|{\bf y}^{(t)}\|$. Fortunately for us, it is rather simple to see that, for a fixed ${\bf y}^{(0)}$, $\mathop{\mathbb{E}}[{\bf y}^{(t)}]$ can be computed exactly; in particular,

[TABLE]

Let $a_{y}(t)\in\mathbb{R}$ be such that ${\bf y}^{(t)}=a_{y}(t)\cdot(\chi/\sqrt{n})$. (12) can equivalently be stated as $\mathop{\mathbb{E}}[a_{y}(t)]=(1-2\delta\lambda_{2}/n)^{t}a_{y}(0)$. This, together with (10), can be used to bound the variance of $a_{y}(t)$ as follows:

[TABLE]

Hence, when $t\ll n^{2}$, Chebyshev’s inequality implies that $a_{y}(t)$ concentrates around $\mathop{\mathbb{E}}[a_{y}(t)]$ or, equivalently, $\|{\bf y}^{(t)}\|$ concentrates around $\left(1-\frac{2\delta\lambda_{2}}{n}\right)^{t}\|{\bf y}_{0}\|$.

Finally, observe that, since $\lambda_{2}<\lambda_{3}$, for sufficiently small $\delta$, we have $2\delta\lambda_{2}<2\delta(1-\delta)\lambda_{3}$. Hence, when $t\gg n\log n$, $\left(1-\frac{2\delta\lambda_{2}}{n}\right)^{t}$ is polynomially (say $n^{10}$ times) larger than $\left(1-\frac{2\delta(1-\delta)\lambda_{3}}{n}\right)^{t}$. It is also not hard to see that, for a random starting vector, $\|{\bf z}^{(0)}\|\ll n^{10}\|{\bf y}^{(0)}\|$ with high probability. This means that, for this range of $t$, we have $\left(1-\frac{2\delta\lambda_{2}}{n}\right)^{t}\|{\bf y}^{(0)}\|\gg\left(1-\frac{2\delta(1-\delta)\lambda_{3}}{n}\right)^{t}\|{\bf z}^{(0)}\|$ with high probability. Since $\|{\bf y}^{(t)}\|$ concentrates on the former quantity whereas $\|{\bf z}^{(t)}\|$ often does not exceed a constant factor of the latter, we can conclude that $\|{\bf y}^{(t)}\|$ is indeed often much larger than $\|{\bf z}^{(t)}\|$.

This wraps up our proof overview of Theorem 5.1.

5.2 The Jump-Labeling protocol

Relying on our insights from the previous section, we propose a lightweight protocol named Jump-Labeling, which makes use of the lazy version of the averaging process. Here $\delta\in[0,1]$ and $\tau^{\text{s}},\tilde{\tau}^{\text{s}},\tau^{\text{e}},\tilde{\tau}^{\text{e}}\in\mathbb{N}$ are parameters that will be chosen later. Intuitively, protocol Jump-Labeling exploits the expected monotonicity in the behaviour of $\mathbf{sgn}({\bf x}_{u}^{(t)}-{\bf x}^{(t-1)})$ highlighted in Section 3. Though this property does not hold for a single realization of the averaging process in general, the results of Section 5 allow us to show that the sign of ${\bf x}^{(\tau_{u}^{e})}-{\bf x}^{(\tau_{u}^{s})}$ reflects $u$’s community membership for most vertices with probability $1-o(1)$ (i.e., the algorithm achieves weak reconstruction) when $\tau_{u}^{s}$ and $\tau_{u}^{e}$ are randomly chosen within a suitable interval. This is the intuition behind the main result of this section. Due to space constraints, the full proof of Theorem 5.3 below is deferred to Appendix E.

Theorem 5.3**.**

Let $n$ be any sufficiently large even positive integer. For any $0<\delta<0.8(\lambda_{3}-\lambda_{2})$, there exist $\tau^{\text{s}},\tilde{\tau}^{\text{s}},\tau^{\text{e}},\tilde{\tau}^{\text{e}}\in\mathbb{N}$ such that, with probability $1-\mathcal{O}\left(\sqrt[8]{\frac{\delta b}{d(\lambda_{3}-\lambda_{2})}}+\sqrt[4]{\frac{1}{\log n}}\right)$, after $\mathcal{O}\left(\frac{n}{\delta(\lambda_{3}-\lambda_{2})}\log\left(n/\delta\right)+\frac{nd}{b\delta}\right)$ rounds of   Jump-Labeling($\delta,\tau^{\text{s}},\tilde{\tau}^{\text{s}},\tau^{\text{e}},\tilde{\tau}^{\text{e}}$), every node labels its cluster and this labelling is a $\left(\sqrt[8]{\frac{\delta b}{d(\lambda_{3}-\lambda_{2})}}+\sqrt[4]{\frac{1}{\log n}}\right)$-weak reconstruction of $G$. The convergence time of this algorithm is $\Omega_{\delta}\left(n\left(\log n+\frac{d}{b}\right)\right)$.

Remark 1**.**

The $nd/b$ dependency in the running time is necessary; imagine we start with a good state where ${\bf x}^{(0)}={\bf z}^{(0)}=0$. In this case, the values on one side of the partition are all $a_{y}(0)$ and the values on the other side are $-a_{y}(0)$. It is simple to see that, after $o(nd/b)$ steps of our protocol, $1-o(1)$ fraction of the values remain the same. For these nodes, it is impossible them to determine which cluster they are in and, hence, no good reconstruction can be achieved.

Similarly to our concentration result in Section 5, let us demonstrate the use of Theorem 5.3 to the two interesting cases. First, let us start with the case where $\lambda_{3}-\lambda_{2}$ is constant. Again, in this case, for any error parameter $\varepsilon>0$, we can pick $\delta=\delta(\varepsilon,\lambda_{2}-\lambda_{3})$ sufficiently small so that, with probability $1-\varepsilon$, the protocol achieves $\varepsilon$-weak reconstruction, as stated below.

Corollary 5.3**.**

For any constant $\varepsilon>0$ and for any $\lambda_{3},\lambda_{2}$, there exists $\delta$ depending only on $\varepsilon$ and $\lambda_{3}-\lambda_{2}$ such that, for any sufficiently large $n$, there exists $\tau^{\text{s}},\tilde{\tau}^{\text{s}},\tau^{\text{e}},\tilde{\tau}^{\text{e}}\in\mathbb{N}$ such that, with probability $1-\varepsilon$, after $\mathcal{O}_{\varepsilon,\lambda_{3}-\lambda_{2}}\left(n\log n+\frac{n}{\lambda_{2}}\right)$ rounds of Jump-Labeling($\delta,\tau^{\text{s}},\tilde{\tau}^{\text{s}},\tau^{\text{e}},\tilde{\tau}^{\text{e}}$), every node labels its cluster and this labelling is a $\varepsilon$-weak reconstruction of $G$.

As in Section 5, we can consider the (non-lazy) averaging protocol and view $\lambda_{2}$ instead as a parameter. On this front, we arrive at the following reconstruction guarantee.

Corollary 5.4**.**

Fix $\delta=1/2$. For any constant $\varepsilon>0$, any $\lambda_{3}>0.7$, any sufficiently small $\lambda_{2}$ depending only on $\varepsilon$, any sufficiently large $n$, there exists $\tau^{\text{s}},\tilde{\tau}^{\text{s}},\tau^{\text{e}},\tilde{\tau}^{\text{e}}\in\mathbb{N}$ such that, with probability $1-\varepsilon$, after $\mathcal{O}_{\varepsilon}\left(n\log n+\frac{n}{\lambda_{2}}\right)$ rounds of Jump-Labeling($\delta,\tau^{\text{s}},\tilde{\tau}^{\text{s}},\tau^{\text{e}},\tilde{\tau}^{\text{e}}$), the nodes’ labelling is a $\varepsilon$-weak reconstruction of $G$.

While the weak reconstruction in the above claims is guaranteed only with arbitrarily-large constant probability, we can boost this success probability considering the same approach we used in Subsection 4.2 to get community-sensitive binary strings of size $\ell=\Theta(\log n)$ from the sign-based protocol.

Indeed, we first run $\ell=\Theta_{\varepsilon}(\log n)$ copies of Jump-Labeling where, similarly to Algorithm 2, “running $\ell$ copies” of Jump-Labeling means that each node keeps $\ell$ copies of the states of Jump-Labeling and, when an edge $\{u,v\}$ is activated, $u$ and $v$ jointly sample a random $j\in[\ell]$ and run the $j$-th copy of Jump-Labeling.

In the previous section, we have seen that Lemma 4.2 and the repetition approach above allowed us to get a good community-sensitive labeling, w.h.p. (not a good weak-reconstruction). Interestingly enough, the somewhat stronger concentration results given in this section allow us to “add” a simple majority rule on the top of the $\ell$ components and get a “good” single-bit label, as described below.

When all $\ell$ components of a node $u$ have been set, node $u$ sets ${\bf h}^{jump}_{u}=\sc{Majority}_{j\in[\ell]}({\bf h}^{jump}_{u}(i))$ where ${\bf h}^{jump}_{u}(j)$ is the binary label of $u$ from the $j$-th copy of the protocol.

Observe that the weak reconstruction guarantee of Jump-Labeling shown earlier implies that the expected number of mislabelings of each copy is at most $2\varepsilon n$, i.e., $\mathop{\mathbb{E}}[\{u\in V\mid|{\bf h}^{jump}_{u}(i)\neq\chi_{u}|\}]\leqslant 2\varepsilon n$. Now, since the number of mislabelings of each copy is independent, the total number of mislabelings is at most $\mathcal{O}(\varepsilon n\ell)$, w.h.p. However, if the eventual label of $u$ is incorrect, it must contributes to mislabeling across at least $\ell/2$ copies. As a result, there are at most $\mathcal{O}(\varepsilon n)$ mislabelings in the new protocol, w.h.p.

The above approach in fact works for any weak reconstruction protocol (not just Jump-Labeling) and, in our case, it easily gives the following result.

Corollary 5.5**.**

For any constant $\varepsilon>0$ and $\lambda_{3}>\lambda_{2}$, there is a protocol that yields an $\varepsilon$-weak reconstruction of $G$ , w.h.p. The convergence time is $\Theta_{\varepsilon,\lambda_{3}-\lambda_{2}}\left(n\left(\log^{2}n+\frac{\log n}{\lambda_{2}}\right)\right)$ rounds, while the work per node is $\mathcal{O}_{\varepsilon,\lambda_{3}-\lambda_{2}}\left(\log^{2}n+\frac{\log n}{\lambda_{2}}\right)$.

We finally remark that, for the dense-cut case we focus on in this section (i.e. $\lambda_{2}=2b/d=\Theta(1)$), the fraction of outliers turns out to be a constant we can made arbitrarily small. If we relax the condition to $\lambda_{2}=o(1)$, then this fraction can be made $o(1)$, accordingly. This issue will be clarified in the full version of the paper.

5.2.1 Proof of Theorem 5.3: An Overview

We now give an informal overview of our proof, which builds on the concentration results from Section 5. Since our discussion here will involve both local times and global times, let us define the following notation to facilitate the discussion: for each vertex $u\in V$, let $T_{u}:\mathbb{N}\to\mathbb{N}$ be a function that maps the local time of $u$ to the global time, i.e., $T_{u}(\tau)\triangleq\min\{t\in\mathbb{N}\mid|\{i\leqslant t\mid u\in\{u_{i},v_{i}\}\}|\geqslant\tau\}$ where $(\{u_{i},v_{i}\})_{i\in\mathbb{N}}$ is the sequence of active edges.

Recall from the previous section that we let $a_{y}(t)\in\mathbb{R}$ be such that ${\bf y}^{(t)}=a_{y}(t)\cdot(\chi/\sqrt{n})$. Let us also assume without loss of generality that $a_{y}(0)\geqslant 0$. Observe first that our concentration result implies the following: for any $t$ such that $\Omega(n\log n)\leqslant t\leqslant\mathcal{O}(n^{2})$, with large probability, $\chi_{u}({\bf x}_{u}^{(t)}-{\bf x}_{||,u})$ is roughly $\mathop{\mathbb{E}}_{\mathcal{E}}a_{y}(t)/n$ for most vertices $u\in V$; let us call these vertices good for time $t$. Imagine for a moment that we change the protocol in such a way that each $u$ has access to the global time $t$ and $u$ assigns ${\bf h}^{jump}_{u}=\mathbf{sgn}({\bf x}_{u}^{(t^{\text{e}})}-{\bf x}_{u}^{(t^{\text{s}})})$ for some $t^{\text{s}},t^{\text{e}}\in[\Omega(n\log n),\mathcal{O}(n^{2})]$ that do not depend on $u$. If $t^{\text{e}}-t^{\text{s}}$ is large enough, then $\mathop{\mathbb{E}}_{\mathcal{E}}a_{y}(t^{\text{s}})\gg\mathop{\mathbb{E}}_{\mathcal{E}}a_{y}(t^{\text{e}})$. This means that, if a vertex $u\in V$ is good at both times $t^{\text{s}}$ and $t^{\text{e}}$, then we have that $\chi_{u}({\bf x}_{u}^{(t^{\text{s}})}-{\bf x}_{||,u})\approx\mathop{\mathbb{E}}_{\mathcal{E}}a_{y}(t^{\text{s}})/n\gg\mathop{\mathbb{E}}_{\mathcal{E}}a_{y}(t^{\text{e}})/n\approx\chi_{u}({\bf x}_{u}^{(t^{\text{e}})}-{\bf x}_{||,u})$. Note that when $\chi_{u}\cdot{\bf x}_{u}^{(t^{\text{s}})}>\chi_{u}\cdot{\bf x}_{u}^{(t^{\text{e}})}$, we have ${\bf h}^{jump}_{u}=\chi_{u}$. From this and from almost all vertices are good at both times $t^{\text{s}}$ and $t^{\text{e}}$, ${\bf h}^{jump}$ is indeed a good weak reconstruction for the graph!

The problem of the modified protocol above is of course that, in our settings, each vertex does not know the global time $t$. Perhaps the simplest approach to imitate the above algorithm in this regime is to fix $\tau^{\text{s}},\tau^{\text{e}}\in[\Omega(\log n),\mathcal{O}(n)]$ and, for each $u\in V$, proceed as in Jump-Labeling except with $\tau_{u}^{\text{s}}=\tau^{\text{s}}$ and $\tau_{u}^{\text{e}}=\tau^{\text{e}}$. In other words, $u$ assigns ${\bf h}^{jump}_{u}=\mathbf{sgn}({\bf x}_{u}^{(T_{u}(\tau^{\text{s}}))}-{\bf x}_{u}^{(T_{u}(\tau^{\text{e}}))})$. The problem about this approach is that, while we know that $\mathop{\mathbb{E}}_{\mathcal{E}}T_{u}(\tau^{\text{s}})=0.5n\tau^{\text{s}}$ and $\mathop{\mathbb{E}}_{\mathcal{E}}T_{u}(\tau^{\text{e}})=0.5n\tau^{\text{e}}$, the actual values of $T_{u}(\tau^{\text{s}})$ and $T_{u}(\tau^{\text{e}})$ differ quite a bit from their means, i.e., on average they will be $\Omega(n\sqrt{\log n})$ of away their mean. Since our concentration result only says that, at each time $t$, we expect 99% of the vertices to be good, it is unclear how this can rule out the following extreme case: for many $u\in V$, $T_{u}(\tau^{\text{s}})$ or $T_{u}(\tau^{\text{e}})$ is a time step at which $u$ is bad. This case results in ${\bf h}^{jump}$ not being a good weak reconstruction of $V$.

The above issue motivates us to arrive at our eventual algorithm, in which $\tau^{\text{s}}_{u}$ and $\tau^{\text{e}}_{u}$ are not fixed to be the same for every $u$, but instead each $u$ pick these values randomly from specified intervals $[\tau^{\text{s}},\tilde{\tau}^{\text{s}}]$ and $[\tau^{\text{e}},\tilde{\tau}^{\text{e}}]$. To demonstrate why this overcomes the above problem, let us focus on the interval $[\tau^{\text{s}},\tilde{\tau}^{\text{s}}]$. While $T_{u}(\tau^{\text{s}})$ and $T_{u}(\tilde{\tau}^{\text{s}})$ can still differ from their means, the interval $[T_{u}(\tau^{\text{s}}),T_{u}(\tilde{\tau}^{\text{s}})]$ still, with large probability, overlaps with most of $[0.5n\tau^{\text{s}},0.5n\tilde{\tau}^{\text{s}}]$ if $\tilde{\tau}^{\text{s}}-\tau^{\text{s}}$ is sufficiently large. Now, if $T_{u}(\tau+1)-T_{u}(\tau)$ are the same for all $\tau\in[\tau^{\text{s}},\tilde{\tau}^{\text{s}}]$, then the distribution of ${\bf x}_{u}^{(T_{u}(\tau^{\text{s}}))}$ is very close to ${\bf x}_{u}^{(t^{\text{s}}_{u})}$ if we pick $t^{\text{s}}_{u}$ randomly from $[0.5n\tau^{\text{s}},0.5n\tilde{\tau}^{\text{s}}]$. From the usual global time step argument, it is easy to see that the latter distribution results in most $u$ being good at time $t^{\text{s}}_{u}$. Of course, $T_{u}(\tau+1)-T_{u}(\tau)$ will not be the same for all $\tau\in[\tau^{\text{s}},\tilde{\tau}^{\text{s}}]$, but we will be able to argue that, for almost all such $\tau$, $T_{u}(\tau+1)-T_{u}(\tau)$ is not too small, which is sufficient for our purpose.

Appendix A Tools from linear algebra

A.1 Projections on the main eigenspaces

Lemma A.1** (Projection on the first two eigenvectors).**

For all $\varepsilon\in(0,1)$, for a random ${\bf x}\in\{-1,1\}^{n}$, with probability at least $1-\mathcal{O}(\varepsilon)$ we have,

[TABLE]

Proof.

Note that ${\bf x}\cdot({\mathbf{1}}+{\bm{\chi}})=2{\bf x}\cdot{\mathbf{1}}_{V_{1}}$ and ${\bf x}\cdot({\mathbf{1}}-{\bm{\chi}})=2{\bf x}\cdot{\mathbf{1}}_{V_{2}}$. Using properties of the binomial distribution, it is easy to see that

[TABLE]

The above event implies $\left||{\bf x}\cdot{\mathbf{1}}|-|{\bf x}\cdot{\bm{\chi}}|\right|\geqslant\varepsilon\sqrt{n}$. Since ${\bf x}\cdot{\mathbf{1}}$ and ${\bf x}\cdot{\bm{\chi}}$ are independent sums of Rademacher random variables, they have the same chances of being positive or negative, thus with probability at least $\frac{1}{2}$ we will have $|{\bf x}\cdot{\mathbf{1}}|\leqslant|{\bf x}\cdot{\bm{\chi}}|$. ∎

A.2 Properties of the spectrum of the main matrices

We consider here Algorithm Averaging$(\delta)$ assuming $\delta=1/2$ and recall the main notations:

• •

$A$ is the adjacency matrix of the clustered graph $G((V_{1},V_{2});E)$, with $|V_{h}|=n/2$, $m=|E|$ and $m_{1,2}=|E(V_{1},V_{2})|$ is the number of edges in the cut $(V_{1},V_{2})$;

• •

$D$ is the diagonal matrix with the degrees of nodes;

• •

$L=D-A$ is the Laplacian matrix;

• •

$\mathcal{L}=D^{-1/2}LD^{-1/2}$is the normalized Laplacian;

• •

$P=D^{-1}A$ is the transition matrix;

• •

For each node $i=1,\dots,n$, we name $d_{i}=a_{i}+b_{i}$ the degree of node $i$, where $a_{i}$ is the number of neighbors in its own block and $b_{i}$ is the number of neighbors in the other block

The next facts are often used in our analysis.

Observation A.1**.**

Let $W=(W(i,j))\sim\mathcal{W}$ be the random matrix of one step of the averaging process, then

[TABLE]

Observation A.2**.**

The expectation of $W$ is

[TABLE]

Proof.

If $i\neq j$ then $\mathbb{E}\!\left[W(i,j)\right]=\frac{1}{2m}$ 2. 2.

If $i=j$ then $\mathbb{E}\!\left[W(i,j)\right]=1\left(1-\frac{d_{i}}{m}\right)+\frac{1}{2}\frac{d_{i}}{m}=1-\frac{d_{i}}{2m}$

∎

As for the spectrum of the main matrices above, defined by the averaging process, we have the following useful properties we can derive from standard spectral algebra.

Observation A.3**.**

Il $\lambda$ is an egenvalue of $L$ then $1-\lambda/(2m)$ is an eigenvalue of $\overline{W}$. 2. 2.

Vector ${\mathbf{1}}$ is an eigenvector of $\overline{W}$. 3. 3.

If the underlying graph $G$ is $(n,d,b)$-regular then ${\bm{\chi}}$ is an eigenvector of $\overline{W}$.

Observation A.4**.**

Consider a graph $G$ with adjacency matrix $A$ and diagonal degree matrix $D$.

Let $1=\lambda_{1}^{P}\geqslant\lambda_{2}^{P}\geqslant\cdots\geqslant\lambda_{n}^{P}$ be the eigenvalues of the transition matrix $P=D^{-1}A$ and let $0=\lambda_{1}\leqslant\lambda_{1}\leqslant\cdots\leqslant\lambda_{1}$ be the eigenvalues of the normalized Laplacian $\mathcal{L}=I-D^{-1/2}AD^{-1/2}$. For every $i=1,\dots,n$ it holds that

[TABLE] 2. 2.

Let $1=\bar{\lambda}_{1}\geqslant\bar{\lambda}_{2}\geqslant\cdots\geqslant\bar{\lambda}_{n}$ be the eigenvalues of $\overline{W}=I-L/(2m)$ and let $0=\lambda_{1}^{L}\leqslant\lambda_{2}^{L}\leqslant\cdots\leqslant\lambda_{n}^{L}$ be the eigenvalues of the Laplacian matrix $L=D-A$. For every $i=1,\dots,n$ it holds that

[TABLE] 3. 3.

For every $i=1,\dots,n$, the eigenvalues of $L$ and $\mathcal{L}$ satisfy

[TABLE]

where $d_{\text{min}}$ and $d_{\text{max}}$ are the minimum and the maximum degree of the nodes, respectively.

As a consequence of the above relationships among the eigenvalues of matrices $L$, $\mathcal{L}$, and $\overline{W}$, we easily get the following useful bounds.

Observation A.5**.**

Let $G$ be an $(n,d,\gamma)$-clustered graph and let $\bar{\lambda}_{i}$ and $\lambda_{i}$, for $i=1,\dots,n$, be the eigenvalues of $\overline{W}$ and $\mathcal{L}$, respectively, in non-decreasing order. It holds that

[TABLE]

Proof.

We have:

[TABLE]

To derive the lower bound, we write:

[TABLE]

Here, the first inequality is a direct consequence of Observation A.4, the second follows from the definition of $(n,d,\gamma)$-clustered graph, while the last inequality follows since $\lambda_{3}+\lambda_{2}\leqslant 2(\lambda_{3}-\lambda_{2})$, whenever $\lambda_{3}\geqslant 3\lambda_{2}$.191919Note that the latter condition holds, since the hypotheses of Theorem 3.1 state that $\lambda_{3}-\lambda_{2}=\Omega(1)$, while the conditions on the cut implies that the graph’s conductance is $o(1)$. The condition $\lambda_{3}\geqslant 3\lambda_{2}$ is thus a consequence of Cheeger’s inequality. The upper bound is derived in the same way, again using $\lambda_{3}\geqslant 3\lambda_{2}$. ∎

Observation A.6**.**

Since the random edges sequentially selected by the process are mutually independent, the expected state of the process at time $t$ can be written as :

[TABLE]

Appendix B Proofs for Section 3

B.1 Proof of Theorem 3.1: Technical lemmas

The next lemma decomposes the state of the system at time $t$, explicitely identifying components parallel to ${\mathbf{1}}$ and ${\bm{\chi}}$ respectively.

Lemma B.1** (Main decomposition).**

Let ${\bf x}\in\{-1,1\}^{n}$ be an arbitrary initial vector of values and, for $h=1,2$, let $\mu_{h}=\mu_{h}({\bf x})=(2/n)\sum_{i\in V_{h}}{\bf x}(i)$ be the average of the initial values in block $h$. The expected vector of values at round $t$ conditional on the initial vector being ${\bf x}$ can be written as

[TABLE]

where

[TABLE]

and, moreover,

[TABLE]

Proof.

Let $1=\bar{\lambda}_{1}>\bar{\lambda}_{2}\geqslant\cdots\geqslant\bar{\lambda}_{n}$ be the eigenvalues of $\overline{W}$ and let ${\bf w}_{1}={\mathbf{1}}/\sqrt{n},{\bf w}_{2},\dots,{\bf w}_{n}$ be a basis of orthonormal eigenvectors of $\overline{W}$, so that we can write

[TABLE]

Since $\bar{\lambda}_{1}=1$ and ${\bf w}_{1}={\mathbf{1}}/\sqrt{n}$, we have that $\langle{\bf x},\,{\bf w}_{1}\rangle{\bf w}_{1}=(1/n)\langle{\bf x},\,{\mathbf{1}}\rangle{\mathbf{1}}$. Hence, $\alpha_{1}=(1/n)\sum_{i\in V}{\bf x}(i)=(\mu_{1}+\mu_{2})/2$.

Since ${\bf w}_{2}=({\bf f}-{\bf f}_{\perp})/\|{\bf f}-{\bf f}_{\perp}\|$ we have that

[TABLE]

Hence,

[TABLE]

Finally, the bound on $\left\|{\bf e}^{(t)}\right\|$ easily follows since, by definition, ${\bf e}\perp{\bf w}_{1},{\bf w}_{2}$ and $\bar{\lambda}_{3}\geqslant\max\{\bar{\lambda}_{i}\,|\,i\geqslant 4\}$. ∎

Lemma B.2**.**

Recall that we name $m_{1,2}=|E(V_{1},V_{2})|$ the size of the cut. It holds that

[TABLE]

Proof.

Observe that since ${\bf f}$ is orthogonal to ${\mathbf{1}}$, we can write ${\bf f}^{\intercal}\overline{W}{\bf f}$ as

[TABLE]

where we used the fact that $\overline{W}$ is symmetric, thus ${\bf f}_{\perp}^{\intercal}\overline{W}{\bf f}_{\parallel}={\bf f}_{\parallel}^{\intercal}\overline{W}{\bf f}_{\perp}=0$, and the fact that ${\bf f}_{\parallel}$ is in the second eigenspace of $\overline{W}$, thus ${\bf f}_{\parallel}^{\intercal}\overline{W}{\bf f}_{\parallel}=\bar{\lambda}_{2}\|{\bf f}_{\parallel}\|^{2}$. Moreover, since ${\bf f}_{\perp}$ is orthogonal to the first two eigenspaces of $\overline{W}$, the third eigenvalue of $\overline{W}$ is $\bar{\lambda}_{3}=\sup_{{\bf x}\perp{\mathbf{1}},{\bf w}_{2}}\frac{{\bf x}^{\intercal}\overline{W}{\bf x}}{\|{\bf x}\|^{2}}\geqslant\frac{{\bf f}_{\perp}^{\intercal}\overline{W}{\bf f}_{\perp}}{\|{\bf f}_{\perp}\|^{2}}$ from (B.1) it follows that

[TABLE]

Observe that, since $\overline{W}=I-L/(2m)=I-\frac{D-A}{2m}$ we can also write ${\bf f}^{\intercal}\overline{W}{\bf f}$ as a function of $m_{1,2}$, indeed

[TABLE]

where we used the fact that ${\bm{\chi}}^{\intercal}D{\bm{\chi}}=2m$, the fact that ${\bm{\chi}}^{\intercal}A{\bm{\chi}}=\sum_{i}a_{i}-\sum_{i}b_{i}=2m-4m_{1,2}$.

From (17) and (18) and the fact that $1=\|{\bf f}\|^{2}=\|{\bf f}_{\parallel}\|^{2}+\|{\bf f}_{\perp}\|^{2}$ we have

[TABLE]

and thus

[TABLE]

∎

Definition B.1** (Bad nodes).**

We say a node $u\in[n]$ is $\varepsilon$-bad if $|{\bf f}_{\perp,u}|\geqslant\varepsilon/\sqrt{n}$, for some $\varepsilon>0$ and we call $B_{\varepsilon}$ the set of $\varepsilon$-bad nodes.

Notice that the property of being a bad node only depends on the graph and on the protocol, not on the “execution” of the protocol.

From the Lemma B.2, an upper bound on the number of $\varepsilon$-bad nodes easily follows.

Corollary B.1** (Number of bad nodes).**

The number $|B_{\varepsilon}|$ of $\varepsilon$-bad nodes is upper bounded by

[TABLE]

Proof.

Assume $B_{\varepsilon}$ vertices satisfy $({\bf f}_{\perp})_{i}>\frac{\varepsilon}{\sqrt{n}}$. Lemma B.2 implies:

[TABLE]

from which the thesis follows. ∎

Lemma B.3** (Monotonicity property).**

Let $G$ be a connected graph, let $\bar{\lambda}_{i}$ for $i=1,\dots,n$ be the eigenvalues of matrix $\overline{W}$, let $\varepsilon$ be such that $0<\varepsilon<1$, and let ${\bf x}\in\{-1,1\}^{n}$ be an arbitrary initial vector such that $|\alpha_{2}|=|\alpha_{2}({\bf x})|>0$. Then, for every node $u\notin B_{\varepsilon}$ and for any round $t$ such that $t\geqslant 3\log\left(\frac{n}{1-\varepsilon}\right)/\log(\bar{\lambda}_{2}/\bar{\lambda}_{3})$, it holds that

[TABLE]

Proof.

Thanks to Lemma B.1, the expected difference of the value of a node in two consecutive rounds is

[TABLE]

From the above equation we get that, as soon as

[TABLE]

the sign of the expected difference between two consecutive values of a non $\varepsilon$-bad node $u$ indicates the community node $u$ belongs to. Moreover, since $|{\bm{\chi}}_{u}\pm\varepsilon|\geqslant 1-\varepsilon$ and $|{\bf e}^{(t-1)}_{u}-{\bf e}^{(t)}_{u}|\leqslant 2\,\bar{\lambda}_{3}^{t}$, the above sign property turns out to be true for every round $t$ such that

[TABLE]

The thesis thus follows from the fact that, if $|\alpha_{2}|>0$ then it is at least $1/n$ and if the graph is connected then $1-\bar{\lambda}_{2}\geqslant 1/n$. Hence, any

[TABLE]

satisfies (20). ∎

Lemma B.4** (Sign property).**

Let $G$ be a connected graph, let $\bar{\lambda}_{i}$ for $i=1,\dots,n$ be the eigenvalues of matrix $\overline{W}$, let $\varepsilon$ be such that $0<\varepsilon<1$, and let ${\bf x}\in\{-1,1\}^{n}$ be an arbitrary initial vector such that $\alpha_{1}=\alpha_{1}({\bf x})$ and $\alpha_{2}=\alpha_{2}({\bf x})$ satisfy $|\alpha_{2}|>2|\alpha_{1}|/(1-\varepsilon)$. Then, for every node $u\notin B_{\varepsilon}$ and for any round $t$ such that

[TABLE]

it holds that $\mathbf{sgn}\left(\mathbb{E}\!\left[{\bf x}^{(t)}_{u}\,|\,{\bf x}^{(0)}={\bf x}\right]\right)=\mathbf{sgn}\left(\alpha_{2}{\bm{\chi}}_{u}\right)$.

Proof.

From (4) it follows that, if $|\alpha_{1}+{\bf e}^{(t)}_{u}|<|\alpha_{2}|\,\bar{\lambda}_{2}^{t}\,|{\bm{\chi}}_{u}\pm\varepsilon|$, then the sign of the expected value of a non $\varepsilon$-bad node $u$ indicates the block node $u$ belongs to. Notice that, for

[TABLE]

we have that

[TABLE]

And for

[TABLE]

we have that

[TABLE]

Hence, if the time-window defined by (21) and (22) is non-empty then for all $t$ in it

[TABLE]

the sign of the expected value of a non-bad node $u$ equals the sign of $\alpha_{2}{\bm{\chi}}_{u}$. ∎

Lemma B.5** (Projection of the initial random vector.).**

Let ${\bf x}$ be chosen uniformly at random in $\{-1,1\}^{n}$. Then, two absolute constants $\beta_{1},\beta_{2}>0$ exist, such that:

• •

with probability at least $\beta_{1}$, it holds that $\alpha_{2}=\alpha_{2}({\bf x})>0$,

• •

with probability at least $\beta_{2}$, it holds that $|\alpha_{2}|>2|\alpha_{1}|/(1-\varepsilon)$.

*Sketch of the Proof. *Both $\alpha_{1}$ and $\alpha_{2}$ are linear combinations of a sum of $n$ independent Rademacher random variables. Then, the two claims can be derived easily from Lemma A.1. ∎

Appendix C Proofs for Section 4.1

C.1 Proof of Theorem 4.1

From the fact that random matrix $W\sim\mathcal{W}$ defined by one step of our averaging process is symmetric and idempotent ($W^{\intercal}W=W$) we get the following upper bound on the expected squared norm of ${\bf y}+{\bf z}$ at the next step as a function of their squared norm at the current step. For readability sake, in the following proofs of this section we use ${\bf y}^{\prime}$ and ${\bf z}^{\prime}$ for random variables ${\bf y}^{(t+1)}$ and ${\bf z}^{(t+1)}$ conditional on the state at round $t$ being ${\bf x}^{(t)}={\bf x}={\bf x}_{\|}+{\bf y}+{\bf z}$.

Lemma C.1**.**

Let ${\bf x}={\bf x}_{\|}+{\bf y}+{\bf z}\in[-1,1]^{n}$ be an arbitrary vector of states. After one step of Algorithm 1 it holds that

[TABLE]

Proof.

Since random matrix $W$ is symmetric and idempotent, it holds that

[TABLE]

∎

The squared norm of ${\bf y}$ at the next step can be lower bounded as a function of the squared norms of ${\bf y}$ and ${\bf z}$ at the current time step as follows. If the underlying graph is $(n,d,b)$-regular, we can get an upper bound as well.

Lemma C.2**.**

Let ${\bf x}={\bf x}_{\|}+{\bf y}+{\bf z}\in[-1,1]^{n}$ be an arbitrary vector of states. After one step of Algorithm 1 it holds that

[TABLE]

Moreover, if the underlying graph $G$ is an $(n,d,b)$-clustered regular graph with $\lambda_{2}=2b/d=o(\lambda_{3}/\log n)$ we also have that

[TABLE]

Proof.

Let $\{u,v\}\in E$ be the random edge sampled at step $t$, call $W_{u,v}\sim\mathcal{W}$ be the corresponding random matrix with $L_{u,v}$ be such that $W=I-\frac{1}{2}L_{u,v}$. As for the lower bound we have

[TABLE]

where the last equality follows since $Q_{2}$ is the projector along the direction of ${\bf v}_{2}={\bm{\chi}}/\|{\bm{\chi}}\|$, which in turn is $L$’s second eigenvector.

As for the upper bound, it holds that

[TABLE]

Next, note that we have

[TABLE]

where we used that $\lambda_{2}=2b/d$ and the fact that ${\bf v}_{2}(u)=1/\sqrt{n}$ if $u$ belongs to first community and ${\bf v}_{2}(u)=-1/\sqrt{n}$ when $v$ belongs to the second community. We further get that

[TABLE]

where it is understood that if $(u,v)$ belongs to the cut, then $u\in V_{1}$ and $v\in V_{2}$ and where, to derive the last equality, we recall that ${\bf z}\perp\mathbf{span}\{{\mathbf{1}},{\bm{\chi}}\}$.

Finally, we get that

[TABLE]

where the last inequality follows by observing that $\sum_{(u,v)\in E(V_{1},V_{2})}\frac{({\bf z}(u)-{\bf z}(v))^{2}}{\|{\bf z}\|^{2}}$ is the Rayleigh quotient of the unnormalized Laplacian of a bipartite $b$-regular graph and the largest possible eigenvalue is $2b$. The thesis follows by using (24), (C.1), and (26) in (C.1). ∎

Lemma C.3**.**

Let ${\bf x}={\bf x}_{\|}+{\bf y}+{\bf z}\in[-1,1]^{n}$ be an arbitrary vector of states. After one step of Algorithm 1 it holds that

[TABLE]

Proof.

From Pythagoras’ Theorem and Lemmas C.1 and C.2, we get

[TABLE]

∎

Finally, by unrolling the double recursion, we get that the expected squared norm of ${\bf z}$ and ${\bf y}$ at round $t$ satisfy the following inequality.

Lemma C.4**.**

Let $G$ be an $(n,d,b)$-clustered regular graph with $\lambda_{2}=\frac{2b}{d}=o\left(\lambda_{3}/\log n\right)$. For every starting state ${\bf x}^{(0)}\in\{-1,+1\}^{n}$ and for every $t\in\mathbb{N}$ it holds that

[TABLE]

Proof.

We first prove the following inequality

[TABLE]

Indeed, from Lemma C.3 we get

[TABLE]

Next we observe from Lemma C.2, for each $i$ we have

[TABLE]

which means that

[TABLE]

Hence,

[TABLE]

The thesis follows by using (29) in (28). ∎

Wrapping up: Proof of Theorem 4.1

Since ${\bf y}^{(t)}-{\bf y}^{(0)}$ and ${\bf z}^{(t)}$ are orthogonal we can write

[TABLE]

The proof proceeds by bounding the two terms above separately. As for the first term, observe that

[TABLE]

where the last inequality follows from the lower bound in Lemma C.2. By taking the expectation of both sides of the previous inequality with respect to the random choice of the initial state, we immediately have that (C.1) holds if $\lVert{\bf y}^{(0)}\rVert^{2}$ is replaced by $\mathbb{E}\!\left[\lVert{\bf y}^{(0)}\rVert^{2}\right]$. Moreover, the upper bound in Lemma C.2 yields

[TABLE]

Recall the upper bound on $\mathbb{E}\!\left[\lVert{\bf z}^{(t-1)}\rVert^{2}\right]$ given by Lemma C.4, namely

[TABLE]

We thus get, for $t=\mathcal{O}(n/\lambda_{2})$,

[TABLE]

where

[TABLE]

for the values of $t$ under consideration. We can unfold the recursion above to obtain

[TABLE]

As for the second term in (30), from (32) we have that

[TABLE]

where in the last inequality we used the fact that $\left(1-\lambda_{2}/n\right)^{-t}\leqslant 2$ for $t\leqslant n/(4\lambda_{2})$ and the fact that $\left(1-\lambda_{3}/n\right)^{t}\leqslant 1/n^{3}$ for $t\geqslant(3n/\lambda_{3})\log n$.

Using (C.1) in (C.1) and then (C.1) and (C.1) in (30) we get

[TABLE]

The thesis then follows from the fact that $\mathbb{E}\!\left[\|{\bf y}^{(0)}\|^{2}\right]=1$ and $\mathbb{E}\!\left[\|{\bf z}^{(0)}\|^{2}\right]\leqslant 1/n$. ∎

C.2 Proofs of Corollary 4.1 and of Equation (8)

• As for Corollary 4.1, we first note that $t$ meets the conditions of Theorem 4.1. Moreover, $t\leqslant c\frac{n}{\lambda_{3}}\log n$, so we immediately have:

[TABLE]

which is at most $\varepsilon^{2}n$, whenever $\frac{\lambda_{2}}{\lambda_{3}}\leqslant\frac{\varepsilon^{4}}{4c\log n}$. Hence, the second claim follows directly from Markov’s inequality.

• As for Equation (8), note that the definition of $\varepsilon$-bad node implies:

[TABLE]

where in $(a)$ we used (7). This easily implies (8).

C.3 Proof of Lemma 4.2

In order to prove Lemma 4.2, we need some preliminary results.

Claim 1**.**

Let $W_{1},\ldots,W_{t}$ be denote a sequence of matrices describing $t$ steps of the Averaging protocol that includes $c$ cross edges and $t-c$ internal edges. Then

[TABLE]

Furthermore, if $W_{1},\ldots,W_{t}$ are chosen randomly according to the Averaging protocol we have,

[TABLE]

Proof.

We have

[TABLE]

To complete the proof, observe that for every vector ${\bf w}$ such that $\|{\bf w}\|_{\infty}\leqslant 1$, we have ${\bm{\chi}}^{T}W_{u,v}{\bf w}={\bm{\chi}}^{T}{\bf w}$ if $(u,v)$ is an internal edge and

[TABLE]

By induction we thus get

[TABLE]

which implies the first part of the lemma. The furthermore part follows by noting that the average of $c$ is $bt/d$, and that Chernoff bounds imply that $c$ is concentrated around its average. ∎

Claim 2**.**

Let $t_{1}=6\frac{n}{\lambda_{3}}\log n$ and assume $\frac{\lambda_{2}}{\lambda_{3}}\leqslant\frac{\varepsilon^{4}}{96\log n}$. Then

[TABLE]

Proof.

Recall that ${\bf y}^{(t)}+{\bf z}^{(t)}=W_{t}\ldots W_{t_{1}}({\bf y}^{(t_{1})}+{\bf z}^{(t_{1})})$. Then, we deterministically have:

[TABLE]

where the first inequality follows from obvious manipulations, the second follows since $W_{t}\ldots W_{t_{1}}$ has norm one, while the last equality follows from Pythagoras’ Theorem.

The proof of Claim 2 next proceeds in the following steps:

1. Upper bound to $\lVert{\bf y}^{(t_{1})}+{\bf z}^{(t_{1})}-{\bf y}^{(0)}\rVert^{2}$.

Theorem 4.1 and Markov’s inequality immediately imply:

[TABLE]

Moreover, since $\lVert{\bf y}^{(t_{1})}+{\bf z}^{(t_{1})}-{\bf y}^{(0)}\rVert^{2}=\lVert{\bf y}^{(t_{1})}-{\bf y}^{(0)}\rVert^{2}+\lVert{\bf z}^{(t_{1})}\rVert^{2}$, we have the following useful derivations:

[TABLE]

2. Upper bound to $\|W_{t}\ldots W_{t_{1}}{\bf y}^{(t_{1})}-{\bf y}^{(t_{1})}\|^{2}$, for $t\in\{t_{1},\ldots,2t_{1}\}$.

We have:

[TABLE]

Next, for any steps $\tau_{1},\tau_{2}\geqslant t_{1}$, denote by $C_{[\tau_{1},\tau_{2}]}$ the number of cross edges sampled over the steps $\tau_{1},\ldots,\tau_{2}$. Claim 1 then implies:

[TABLE]

The deterministic upper bound stated by (41) immediately implies:

[TABLE]

On the other hand, we immediately have $\mathbb{E}\!\left[C_{[1,t]}\right]\leqslant\frac{2b}{d}t=\lambda_{2}t$. Application of Chernoff bound then implies:

[TABLE]

since $t\geqslant t_{1}=6\frac{n}{\lambda_{3}}\log n$ and $\lambda_{2}\geqslant 1/n$, given the the graph is regular. As a result:

[TABLE]

Recalling (38) and using (39) and (42) we finally obtain:

[TABLE]

In particular, from (39), (40) and (42), we know that with probability $1-\varepsilon-\frac{\log n}{n^{2}}$:

[TABLE]

The last inequality in the derivations above follows by noting that $\frac{36\lambda_{2}t}{n}\left(1+\frac{4\lambda_{2}t_{1}}{\varepsilon n}\right)\leqslant\frac{36\lambda_{2}t}{\varepsilon n}$ since $t\leqslant 2t_{1}\leqslant 4n/\lambda_{2}$, which in turn follows if $\frac{\lambda_{2}}{\lambda_{3}}\leqslant\frac{\varepsilon^{4}}{96\log n}$ as we assume.

Upper bound on $|B_{t}|$. (8) and (37) immediately imply that, with probability at least $1-\varepsilon-\log n/n^{2}$

[TABLE]

This concludes the proof of Claim 2 . ∎

Wrapping up: Proof of Lemma 4.2

Let $t_{1}=6\frac{n}{\lambda_{3}}\log n$ and define $A_{t_{1}+1}:=V-B_{t_{1}+1}$ the complement of $B_{t_{1}+1}$ and, for each $t\in[t_{1}+1,2t_{1}]$, let $A_{t}$ denote the set of nodes in $A_{t_{1}+1}$ that have not been averaged along a cross edge or with a node in $B_{t}$. Inductively, if $e_{t}=(u_{t},v_{t})$ is the edge chosen at time $t$ then

[TABLE]

We say that we sampled a good edge at time $t$ in the first case, otherwise we say that we sampled a bad edge. The proof proceeds in two steps:

Deterministic lower bound on the number of goods nodes. We note that $|A_{2t_{1}}|$ is a lower bound on the number of nodes that are $\varepsilon$-good at every step $t\in[t_{1},2t_{1}]$. Indeed, every node $v$ in $A_{2t_{1}}$ was $\varepsilon$-good at all times between $t_{1}$ and $2t_{1}$, because it is a node whose value was good at time $t_{1}+1$, and then was averaged only with nodes $u$ on the same side of the partition (that is, such that $\mathbf{sgn}({\bf y}_{u})=\mathbf{sgn}({\bf y}_{v})$) and at times in which $u$ was good as well. Moreover, $|A_{t}|$ decreases by at most 2 compared to $|A_{t-1}|$ and this can only happen if we sample a bad edge. As a consequence, $|A_{2t_{1}}|\geqslant|A_{t_{1}}|-Z$, with $Z$ the number of bad edges sampled in the interval $[t_{1},2t_{1}]$.

Lower bound on $|A_{2t_{1}}|$. Next, we condition on the event ${\mathcal{B}}=(\forall t\in\{t_{1},\ldots,2t_{1}\}:|B_{t}|\leqslant\frac{\lambda_{3}\varepsilon n}{12\log n})$. From Claim 2, this event holds with probability at least $1-\varepsilon$, whenever $\frac{\lambda_{2}}{\lambda_{3}}\leqslant\frac{\lambda_{3}\varepsilon^{4}}{6192\log^{2}n}$. Now, observe that, for a given time step $t$, conditioning on the event ${\mathcal{B}}$ can only increase the probability of sampling a good edge. As a consequence, recalling that, without conditioning we sample edges uniformly at random, for any integer $x>0$ we have:

[TABLE]

where $\hat{Z}$ is the sum of independent Bernoulli variables with parameter $\frac{2b}{d}+\frac{\lambda_{3}\varepsilon}{12\log n}$. Here, the first term is the probability of sampling a cross edge, while the second is worst-case upper bound on the probability of sampling an edge with an endpoint in $A_{t}$ and the other in $B_{t}$, provided that $|B_{t}|\leqslant\frac{\lambda_{3}\varepsilon n}{12\log n}$. As a consequence, recalling that $\lambda_{2}=2b/d$:

[TABLE]

where the last inequality obviously follows given our assumptions on $\lambda_{2}/\lambda_{3}$. At this point, a simple application of Chernoff bound allows to conclude that $\hat{Z}$ (and thus $Z$, when conditioned to $\mathcal{B}$), is at most $2\varepsilon n$ with probability $1-e^{-\frac{\varepsilon n}{2}}$. So, conditioned to the event $\mathcal{B}$, $A_{2t_{1}}\geqslant n-\frac{\lambda_{3}n}{12\log n}-2\varepsilon n$ with probability $1-e^{-\frac{\varepsilon n}{2}}$. This concludes the proof. ∎

C.4 Tools for the analysis of Algorithm 2

Proof of Theorem 4.3

In the next two subsections, we provide Lemmas C.5 and C.7, respectively. The two lemmas together easily imply Theorem 4.3.

Lucky nodes and community-sensitive labeling

We next consider the behavior of Algorithm $\mbox{\sc Sign-Labeling}(T,1)$ on a graph $G=(V,E)$ where $V$ has a sparse cut $(V_{1},V_{2})$ that we wish to discover. For every node $u$, let $p^{sign}_{u}(T)$ be the probability that node $u$ sets ${\bf h}_{u}^{sign}$ differently from the sign of the average of the initial values of the nodes in its own community

[TABLE]

where the randomness is over the initial choice of ${\bf x}\sim\{-1,1\}^{n}$ and over the execution of Algorithm $\mbox{\sc Sign-Labeling}(T,1)$. Notice that, for a given graph $G$ and partition of the nodes $V_{1},V_{2}$, $p^{sign}_{u}(T)$ depends on the node $u$, on the protocol (sign), and on the number of activations $T$ after which the node sets ${\bf h}^{\mbox{sign}}_{u}$ (we will omit parameter $T$ from $p^{sign}_{u}$ when clear from context).

To understand the point of this definition, consider the extreme case in which the cut $(V_{1},V_{2})$ is empty, while $V_{1}$ and $V_{2}$ induce connected graphs. In this case, the averaging process Averaging will converge to a global state in which all nodes in $V_{1}$ have a local state close to the average $\frac{2}{n}\sum_{v\in V_{1}}{\bf x}_{v}$ and all nodes in $V_{2}$ have a state close to $\frac{2}{n}\sum_{v\in V_{2}}{\bf x}_{v}$. We call $\tau_{u}$ the (random) step in which the node $u$ sets its ${\bf h}^{\mbox{sign}}_{u}$ to the sign of the state of $u$, i.e., the global round when $u$ achieves $T$ activations. If $T$ is chosen so that $\tau_{u}$ is large enough, we would expect ${\bf h}^{\mbox{sign}}_{u}$ to agree with the sign of $\frac{2}{n}\sum_{v\in V_{1}}{\bf x}_{v}$ if $u\in V_{1}$ and with the sign of $\frac{2}{n}\sum_{v\in V_{2}}{\bf x}_{v}$ if $u\in V_{2}$, with $p^{sign}_{u}$ small for all $u$. It seems reasonable that a possibly weakened version of the considerations above should apply to graphs exhibiting a sparse, rather than empty, cut, provided the subgraphs induced by $V_{1}$ and $V_{2}$ are good expanders. To quantitavely capture this intuition, we introduce the notion of a (un)lucky node.

Definition C.1** (Unlucky nodes).**

We say that node $u$ is $\varepsilon$-unlucky if $p^{sign}_{u}$ is larger than $\varepsilon$. We thus define the set of $\varepsilon$-unlucky nodes as follows

[TABLE]

We write $U^{\varepsilon,sign}$ in place of $U^{\varepsilon,sign}_{G,(V_{1},V_{2})}$, when the underlying graph and partition of the nodes are clear from the context.

Lemma C.5**.**

Let $G=(V,E)$ be a graph, $V_{1},V_{2}$ be a partition of $V$ and fix $\varepsilon\in(0,\frac{1}{12}]$. Then, Sign-Labeling $(T,10\varepsilon^{-1}\log n)$ performs a community-sensitive labeling of $G$ according to Definition 2.4, with $c_{1}=4\varepsilon$, $c_{2}=1/6$ and $\gamma=|U^{\varepsilon,T}|/n$.

Proof.

The proof of the theorem relies on the mutual independence among the components of any label ${\bf h}(\cdot)$ and some standard arguments. We remark that the independence crucially depends on the fact that Sign-Labeling (T,m) updates one component per interaction: the evolution of ${\bf x}(j_{1})$ and ${\bf x}(j_{2})$ depends solely on the respective initial vector values (which are independent), and on the sequence of sampled edges which update component $j_{1}$ and $j_{2}$ (which are independent conditional on their number).

Call $\ell=\frac{10\log n}{\varepsilon}$ and denote ${\bf h}_{V_{i}}:=(h_{V_{i}}(1),\ldots,h_{V_{i}}(\ell))$ where $h_{V_{i}}(j):=\mathbf{sgn}\left(\sum_{v\in V_{i}}x_{v}\right)$.

We first claim that w.h.p. for every vertex $u\in V_{1}\setminus U^{\varepsilon}$, $\Delta({\bf h}_{u},{\bf h}_{V_{1}})\leqslant 2\varepsilon\ell$. Observe that by definition of $U^{\varepsilon}$,

[TABLE]

Since the $\ell$ components are mutually independent, the Chernoff bound [DP09] implies that $\Delta({\bf h}_{u},{\bf h}_{V_{1}})\leqslant 2\varepsilon\ell$, w.h.p. A union bound over vertices in $V_{1}\setminus U^{\varepsilon}$ implies the claim.

Henceforth, let us assume that $\Delta({\bf h}_{u},{\bf h}_{V_{1}})\leqslant 2\varepsilon\ell$ for each $u\in V_{1}\setminus U^{\varepsilon}$ and a similar claim for all vertices $v\in V_{2}\setminus U^{\varepsilon}$.

As for Case (i), w.l.o.g. let us consider $u,v\in V_{1}\setminus U$. By triangle inequality, we get the desired claim.

[TABLE]

As for Case (ii), since the initial values of $x_{u}(j)$ ($u\in V_{1}\cup V_{2}$) are chosen independently and uniformly at random in $\{-1,1\}$, simple symmetry arguments show that the probability of the event “$\mathbf{sgn}(\sum_{u\in V_{1}}x_{u})=\mathbf{sgn}(\sum_{u\in V_{2}}x_{u})$” is $1/2$. Hence, $\mathbb{E}\!\left[\Delta({\bf h}_{V_{1}},{\bf h}_{V_{2}})\right]=\ell/2$ and from Chernoff bounds we get that

[TABLE]

with all but a probability exponentially small in $\ell$. Henceforth, let us condition on the event that $\Delta({\bf h}_{V_{1}},{\bf h}_{V_{2}})\leqslant\frac{\ell}{3}$

Consider $u\in V_{1}$ and $v\in V_{2}$. By triangle inequality, we have that

[TABLE]

concluding the proof.

∎

C.4.1 A bound on the number of unlucky nodes

In Lemma C.7 we give an upper bound on the number of $\varepsilon$-unlucky nodes. This is the second key step toward proving Theorem 4.3.

We first prove the following technical lemma on the range in which $\tau_{v}$ falls w.h.p.

Lemma C.6**.**

If $T>72\log n$ and $t_{1}=3Tn/4$ then

[TABLE]

Proof.

For each node $v$, let $X_{v}^{(i)}={\mathbf{1}}_{[v\text{ is activated at round }i]}$. Fix a node $v$. By applying the Chernoff bound on the i.i.d. random variables $\{X_{v}^{(i)}\}_{i\geqslant 0}$, we have

[TABLE]

The claim follows by applying a union bound over the nodes. ∎

Lemma C.7** (Number of unlucky nodes).**

Let $\varepsilon>0$ be an arbitrarily small value and let $G$ be an $(n,d,b)$-clustered regular graph with $\frac{\lambda_{2}}{\lambda_{3}}\leqslant\frac{\lambda_{3}\varepsilon^{4}}{c\log^{2}n}$, for a large enough costant $c$. If $T=\frac{8}{\lambda_{3}}\log n$ then the number of $\sqrt{\varepsilon}$-unlucky nodes is

[TABLE]

Proof.

Let $L^{sign}$ be the set of nodes that freeze their sign ${\bf h}_{v}^{sign}$ according to the sign of ${\bf x}_{\|,v}+{\bf y}^{(0)}_{v}$,

[TABLE]

We first observe that, given any $\varepsilon>0$, if we have a lower bound on the expected size of $L^{sign}$, namely $\mathbb{E}\!\left[|L^{sign}|\right]\geqslant n-\varepsilon n$, then we have an upper bound on the number of $\sqrt{\varepsilon}$-unlucky nodes, namely $|U^{\sqrt{\varepsilon},sign}|\leqslant 6\sqrt{\varepsilon}n$. Indeed,

[TABLE]

We now give a lower bound on the expected size of $L^{sign}$. Let $\Gamma$ be the event

[TABLE]

Recall that $\tau_{v}$ denotes the time at which node $v$ freezes its value of ${\bf h}^{sign}_{v}$. Notice that the value $|{\bf y}_{v}^{(0)}+{\bf x}_{\|,v}|$ does not depend on the node $v$, only on the initial assignment. Hence, for any node $u\in V_{1}\cup V_{2}$ we have that

[TABLE]

From Lemmas A.1 and C.6 it follows that $\mathbb{P}\left[\overline{\Gamma}\right]\leqslant\mathcal{O}(\varepsilon)$ and that $\mathbb{P}\left[\tau_{u}\notin[t_{1},2t_{1}]\right]\leqslant 1/n$. Hence, from (49) the expected size of $L^{sign}$ is

[TABLE]

Finally, from Lemma 4.2 we have

[TABLE]

Thus from (50) and the previous inequality we get that

[TABLE]

and the thesis follows from (C.4.1). ∎

Appendix D Omitted Proofs from Section 5

The main goal of this subsection is to prove Theorem 5.1. For the sake of convenience, we rewrite the state of the averaging process as

[TABLE]

where $a_{||},a_{y}(t)\in{\mathbb{R}}$ and ${\bf z}^{(t)}$ is orthogonal to both $\chi$ and ${\mathbf{1}}$. Recall also that ${\bf y}^{(t)}=a_{y}(t)\cdot\left(\chi/\sqrt{n}\right)$ and that $a_{||}$ remains unchanged throughout the algorithm. Suppose now we fix a starting vector ${\bf x}^{(0)}$, then we can exactly compute the expectation of $a_{y}(t)$ as stated more formally below.

Observation D.1**.**

For all $t\in\mathbb{Z}_{\geqslant 0}$, we have $\mathop{\mathbb{E}}_{\mathcal{E}}[a_{y}(t)]=\left(1-\frac{2\delta\lambda_{2}}{n}\right)^{t}a_{y}(0).$

Proof of Observation D.1.

To prove the above statement, it is enough to show that $\mathop{\mathbb{E}}_{(u_{t},v_{t})}[a_{y}(t)]=\left(1-\frac{2\delta\lambda_{2}}{n}\right)a_{y}(t-1)$ for every $t\in\mathbb{N}$. Indeed, $\mathop{\mathbb{E}}_{(u_{t},v_{t})}[a_{y}(t)]$ can be rewritten as follows.

[TABLE]

concluding the proof. ∎

Let $\mu(t)\triangleq\mathop{\mathbb{E}}[a_{y}(t)]=\left(1-\frac{4\delta b}{dn}\right)^{t}a_{y}(0)$ be the expectation of $a_{y}(t)$. We will show that, if we start with ${\bf x}^{(0)}$ such that $\|{\bf z}^{(0)}\|^{2}$ is not too much larger than $n\|{\bf y}^{(0)}\|^{2}$, then $a_{y}(t)$ concentrates around $\mu(t)$, as long as $t\leqslant O_{b,d,\delta}(n^{2})$. Moreover, we will also show that, for $t\geqslant\Omega_{b,d,\delta}(n\log n)$, $\|{\bf z}^{(t)}\|$ becomes small compared to $\mu(t)$. This is stated more precisely below.

Theorem D.1**.**

Let $\beta$ be any real number such that $1\leqslant\beta\leqslant\frac{d}{\varepsilon b}$. For any initial vector ${\bf x}^{(0)}$ that satisfies satisfies $\|{\bf z}^{(0)}\|^{2}\leqslant n\beta\|{\bf y}{(0)}\|^{2}$ and for any $t\in\left[\frac{8n}{\delta(\lambda_{3}-\lambda_{2})}\log\left(\frac{nd\beta}{\varepsilon b}\right),\frac{n^{2}\beta}{128\delta(\lambda_{3}-\lambda_{2})}\right]$, we have

[TABLE]

and

[TABLE]

We defer the proof of Theorem D.1 to Subsection D.1. For now, let us turn our attention back to the proof of Theorem 5.1. To go from here to Theorem 5.1, we will also need to upper bound the probability that $\|{\bf z}^{(0)}\|^{2}>n\beta\|{\bf y}^{(0)}\|^{2}$. More specifically, when ${\bf x}^{(0)}$ is a random $\pm 1$ vector, we have the following bound.

Proposition 1**.**

For any $\beta>0$, we have $\mathop{\mathbb{P}}_{{\bf x}^{(0)}\sim\{\pm 1\}^{n}}\left[\|{\bf z}^{(0)}\|^{2}>n\beta\|{\bf y}^{(0)}\|^{2}\right]\leqslant O(1/\sqrt{\beta}+1/\sqrt{n}).$

This proposition was also essentially proved in [BCN*+*17]; we repeat the proof from [BCN*+*17] below for completeness.

Proof of Proposition 1.

First, note that $\|{\bf z}(0)\|^{2}\leqslant\|{\bf x}(0)\|^{2}=n$. Hence, it suffices to upper bound the probability that $\|{\bf y}(0)\|^{2}$ is less than $1/\beta$. Since $\|{\bf y}(0)\|=\left|\frac{\chi^{T}{\bf x}(0)}{\sqrt{n}}\right|$, the probability that $\|{\bf y}(0)\|^{2}<1/\beta$ is exactly equal to the probability that a sum of $n$ i.i.d. Rademacher random variables lie in $[-\sqrt{n/\beta},\sqrt{n/\beta}]$. The latter probability is exactly equal to

[TABLE]

where the second inequality comes from a well-known fact that $\binom{n}{n/2}=O(2^{n}/\sqrt{n})$. ∎

By combining Theorem D.1 and Proposition 1, we immediately get Theorem 5.1.

Proof of Theorem 5.1.

choosing $\beta=\left(\frac{d}{\varepsilon b}\right)^{2/3}$, we can upper bound $\mathop{\mathbb{P}}_{{\bf x}^{(0)},\mathcal{E}}\left[\|{\bf z}^{(t)}\|^{2}\leqslant\sqrt{\varepsilon b/d}\|{\bf y}^{(t)}\|^{2}\right]$ by

[TABLE]

Then, from Theorem D.1, the first term is at most $O(\varepsilon\beta b/d)+O(\sqrt{\varepsilon b/d})=O(\sqrt[3]{\varepsilon b/d})$. Moreover, from Proposition 1, the second term is also at most $O(1/\sqrt{\beta}+1/\sqrt{n})=O(\sqrt[3]{\varepsilon b/d}+1/\sqrt{n})$. ∎∎

D.1 Proof of Theorem D.1

D.1.1 Evolution of State in One Time Step

The first step in proving Theorem D.1 is to understand what happens in a single update. Specifically, we would like to understand how $\|{\bf y}^{(t)}\|$ and $\|{\bf z}^{(t)}\|$ behave, given $\|{\bf y}^{(t-1)}\|$ and $\|{\bf z}^{(t-1)}\|$. To this end, we prove the following lemma, which gives bounds on expectations of $\|{\bf y}^{(t)}\|^{2}$ and $\|{\bf z}^{(t)}\|^{2}$ based on $\|{\bf y}^{(t-1)}\|^{2}$ and $\|{\bf z}^{(t-1)}\|^{2}$.

Lemma D.2**.**

Let $G$ be as above. Let ${\bf y}$ be a vector parallel to $\chi$ and ${\bf z}$ be a vector orthogonal to ${\bf y}$ and to ${\mathbf{1}}$. Let $P_{\chi}$ be the projection matrix on $\chi$, that is, $P_{\chi}=\frac{1}{n}\chi\chi^{T}$ and let $P_{\perp}$ be the the projection matrix on the space orthogonal $\chi$, that is, $P_{\perp}=I-\frac{1}{n}\chi\chi^{T}$. Moreover, let ${\bf y}^{\prime}=P_{\chi}W(y+z)$ and ${\bf z}^{\prime}=P_{\perp}W(y+z)$ where $W$ is randomly selected according to $\mathcal{W}$. Then,

[TABLE]

and

[TABLE]

We note here that Lemma D.2 is simply a restatement of Lemma 5.2.

Proof.

Note that ${\bf y}^{\prime}$ and ${\bf z}^{\prime}$ are orthogonal. We will estimate the expectation of $\|{\bf y}^{\prime}+{\bf z}^{\prime}\|^{2}$ and of $\|{\bf y}^{\prime}\|^{2}$, and the we will use Pythagoras’s theorem to deduce a bound on $\|{\bf z}^{\prime}\|^{2}$.

To estimate the expected norm squared of ${\bf y}^{\prime}+{\bf z}^{\prime}$ we see that

[TABLE]

where $\bar{W}:=\mathbb{E}\!\left[W\right]$, and in $(a)$ we used that $\bar{W}{\bf y}=\left(1-\frac{4\delta b}{dn}\right){\bf y}$.

Moreover, note that ${\bf z}$ can be written as a linear combination of eigenvectors of $\bar{W}$ whose eigenvalues are at most $1-\frac{2\delta\lambda_{3}}{n}$; This implies that ${\bf z}^{T}\bar{W}{\bf z}\leqslant\left(1-\frac{2\delta\lambda_{3}}{n}\right)\|{\bf z}\|^{2}$. Plugging this inequality into the above equality, we have

[TABLE]

Now we estimate the expected squared norm of ${\bf y}^{\prime}$. Observe that

[TABLE]

Let us consider two cases: whether the edge $(u,v)$ defining $W$ is an internal edge, that is, an edge whose endpoints are on the same side of the partition, or it is a cross edge having endpoints on different sides of the partition.

If $(u,v)$ is an internal edge, which happens with probability $1-\frac{b}{d}$, then $W\chi=\chi$, and so $WP_{\chi}W=\frac{1}{n}W\chi\chi^{T}W=\frac{1}{n}\chi\chi^{T}=P_{\chi}$. This implies that $({\bf y}+{\bf z})^{T}W^{T}P_{\chi}W({\bf y}+{\bf z})=\|{\bf y}\|^{2}.$ 2. 2.

If $(u,v)$ is a cross edge such that $u\in V_{1}$ and $v\in V_{2}$, recall that $W=I-\delta{\bf e}_{u,v}{\bf e}_{u,v}^{T}$. To bound $\|P_{\chi}W({\bf y}+{\bf z})\|^{2}$, first observe that

[TABLE]

Hence, we have

[TABLE]

Now, let us consider $P_{\chi}W{\bf z}$. Observe that

[TABLE]

By combining (54) and (55), we have

[TABLE]

Now, if we take the expectation over cross edges $(u,v)$, the second term becomes zero, because both $z_{u}$ and $z_{v}$ average to zero for a random cross edge (the margninal of $u$ is uniform over $V_{1}$ and the marginal of $v$ is uniform over $V_{2}$). Moreover, we have

[TABLE]

where the first equality follows from the fact that each vertex has $b$ cross edges.

Thus, in this case, we have

[TABLE]

Putting the two cases together, we arrive at the following inequality.

[TABLE]

Finally, note that the upper bound in (56) is already the desired upper bound for $\mathop{\mathbb{E}}_{W}[\|{\bf y}^{\prime}\|^{2}]$ and that the lower bound in (56) together with (53) implies the desired upper bound on $\mathop{\mathbb{E}}_{W}[\|{\bf z}^{\prime}\|^{2}]$. ∎

D.1.2 From Evolution of State to Bounds on $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf y}^{(t)}\|^{2}]$ and $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf z}^{(t)}\|^{2}]$

We next turn the bounds from Lemma D.2 to bounds on $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf y}^{(t)}\|^{2}]$ and $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf z}^{(t)}\|^{2}]$ based only on $\|{\bf y}(0)\|^{2},\|{\bf z}(0)\|^{2},\delta$ and the parameters of our graph. This bound will indeed be enough for us to prove certain concentrations of $\|{\bf y}^{(t)}\|$ and $\|{\bf z}^{(t)}\|$, which are at the heart of the analysis. Before we state the bounds on $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf y}^{(t)}\|^{2}]$ and $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf z}^{(t)}\|^{2}]$, let us define the following shorthands for some expressions that will appear regularly throughout the rest of the section.

• •

Let $\xi\triangleq\left(1-\frac{4\delta b}{dn}\right)^{2},\xi_{1}\triangleq\left(1-\frac{8\delta b}{dn}+\frac{336\delta^{2}b}{dn^{2}}\right),\xi_{2}\triangleq\left(1-\frac{4\delta(1-\delta)\lambda_{3}}{n}\right)$. Note that $\mu(t)=\xi^{t/2}a_{y}(0)$.

• •

Let $\kappa\triangleq 1+(40\varepsilon b/d)\beta$. Recall that $\beta$ is a parameter in Theorem D.1 which satisfies $\beta\geqslant\frac{\|{\bf z}(0)\|^{2}}{n\|{\bf y}(0)\|^{2}}$.

We can now state our bounds on $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf y}^{(t)}\|^{2}]$ and $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf z}^{(t)}\|^{2}]$:

Lemma D.3**.**

For any $t\in\mathbb{Z}_{\geqslant 0}$, we have

[TABLE]

We defer the proof of Lemma D.3, which is essentially solving the recurrence relation from Lemma D.2, to Subsection D.1.5. Let us now proceed to use this lemma to derive concentrations of $\|{\bf y}^{(t)}\|,\|{\bf z}^{(t)}\|$.

D.1.3 Concentrations of $\|{\bf y}^{(t)}\|$ and $\|{\bf z}^{(t)}\|$

A direction application of Markov’s inequality to the bound on $\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf z}^{(t)}\|^{2}]$ from Lemma D.3 gives us the desired concentration for $\|{\bf z}^{(t)}\|$:

Lemma D.4**.**

For every $t\in\mathbb{Z}_{\geqslant 0}$, $\mathop{\mathbb{P}}_{\mathcal{E}}\left[\|{\bf z}^{(t)}\|^{2}\geqslant 0.25\sqrt{\varepsilon b/d}(\mu(t))^{2}\right]\leqslant 80\sqrt{\varepsilon b/d}\kappa(\xi_{1}/\xi)^{t}+\frac{4\beta n}{\sqrt{\varepsilon b/d}}(\xi_{2}/\xi)^{t}.$

For $\|{\bf y}^{(t)}\|$, since we know that ${\|{\bf y}^{(t)}\|}^{2}$ is simply $a_{y}(t)^{2}$ and we also know $\mu(t)=\mathop{\mathbb{E}}_{\mathcal{E}}[a_{y}(t)]$, we can apply Cherbychev’s inequality on $a_{y}(t)$, which results in the following lemma.

Lemma D.5**.**

For every $t\in\mathbb{Z}_{\geqslant 0}$, $\mathop{\mathbb{P}}_{\mathcal{E}}[a_{y}(t)\notin(0.5\mu(t),1.5\mu(t))]\leqslant 4\left(\kappa(\xi_{1}/\xi)^{t}-1\right).$

Proof.

Recall from Observation D.1 that $\mathop{\mathbb{E}}_{\mathcal{E}}[a_{y}(t)]=\mu(t)=\xi^{t/2}a_{y}(0)$. Moreover, from Lemma D.3, we have $\mathop{\mathbb{E}}_{\mathcal{E}}[a_{y}(t)^{2}]=\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf y}^{(t)}\|^{2}]\leqslant(\kappa\xi_{1}^{t})(a_{y^{(0)}})^{2}$. Hence, from Chebyshev’s inequality, we have

[TABLE]

as desired. ∎

D.1.4 Putting things together

Finally, we will now prove Theorem D.1 by plugging in the appropriate value for variables in Lemma D.4 and Lemma D.5. To this end, let us first state a couple of inequalities that will be useful.

Lemma D.6**.**

If $\beta\leqslant\frac{d}{\varepsilon b}$, then, for any $t\leqslant\frac{n^{2}\beta}{1344\delta(\lambda_{3}-\lambda_{2})}$, we have $\kappa(\xi_{1}/\xi)^{t}\leqslant 1+81(\varepsilon b\beta/d)$.

Lemma D.7**.**

If $\beta\leqslant\frac{d}{\varepsilon b}$, then, for any $t\geqslant\frac{8n}{\delta(\lambda_{3}-\lambda_{2})}\log\left(\frac{nd\beta}{\varepsilon b}\right)$, we have $\frac{4\beta n}{\sqrt{\varepsilon b/d}}(\xi_{2}/\xi)^{t}\leqslant 4\sqrt{\varepsilon b/d}$.

We defer the proofs of both lemmas, which are basically calculations, to Appendix D.1.6. Let us now proceed to prove Theorem D.1.

Proof of Theorem D.1.

Let $t$ be any positive integer such that $\frac{8n}{\delta(\lambda_{3}-\lambda_{2})}\log\left(\frac{nd}{\varepsilon b}\right)\leqslant t\leqslant\frac{n^{2}\beta}{1344\delta(\lambda_{3}-\lambda_{2})}$. From Lemma D.5 and Lemma D.6, we have $\mathop{\mathbb{P}}_{\mathcal{E}}[a_{y}(t)\notin(0.5\mu(t),1.5\mu(t))]\leqslant O(\varepsilon b\beta/d)$.

Moreover, from Lemma D.4, Lemma D.6 and Lemma D.7, we have

[TABLE]

which concludes the proof of Theorem D.1.

∎

D.1.5 Proof of Lemma D.3

The goal of this subsection is to prove Lemma D.3, which is essentially just solving the recurrence relation from Lemma D.2. Before we proceed to the proof, we state a fact and an observation regarding eigenvalues and eigenvectors of certain $2\times 2$ natruces, which will be useful in our proof.

Fact D.1**.**

Let $A=\begin{bmatrix}a_{11}&a_{12}\\ a_{21}&a_{22}\end{bmatrix}\in\mathbb{R}^{2\times 2}$ be a $2\times 2$ real-valued matrix such that $a_{11}\neq 0$ and $(a_{11}-a_{22})^{2}\neq 4a_{12}a_{21}$. Then, its eigenvalues are

[TABLE]

and its eigenvectors are

[TABLE]

Observation D.2**.**

Let $A,a_{11},a_{12},a_{21},a_{22},\alpha_{1}(A),\alpha_{2}(A)$ be as in Fact D.1. Suppose further that $a_{11},a_{12},a_{21},a_{22}\geqslant 0,a_{11}>a_{22}$ and that $(a_{11}-a_{22})^{2}>4a_{12}a_{21}$. Then,

[TABLE]

Proof.

The inequalities come from an observation that $a_{11}-a_{22}\leqslant\sqrt{(a_{11}-a_{22})^{2}+4a_{12}a_{21}}\leqslant a_{11}-a_{22}+\frac{2a_{12}a_{21}}{a_{11}-a_{22}}$. ∎

We are now ready to prove Lemma D.3.

Proof of Lemma D.3.

Let $\{y^{(t)}\}_{t\in\mathbb{Z}_{\geqslant 0}}$ and $\{z^{(t)}\}_{t\in\mathbb{Z}_{\geqslant 0}}$ be sequence of non-negative real numbers defined by $y^{(0)}=\|{\bf y}(0)\|^{2},z^{(0)}=\|{\bf z}(0)\|^{2}$ and

[TABLE]

for every $t\in\mathbb{N}$. Note that, from Lemma D.2 and from the initial values $y^{(0)},z^{(0)}$, we have $y^{(t)}\geqslant\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf y}(t)\|^{2}]$ and $z^{(t)}\geqslant\mathop{\mathbb{E}}_{\mathcal{E}}[\|{\bf z}(t)\|^{2}]$ for every $t\in\mathbb{Z}_{\geqslant 0}$. Hence, to prove the lemma, it suffices to prove that

[TABLE]

and

[TABLE]

Let $a_{11},a_{12},a_{21},a_{22}$ be the entries of $A$. Note that

[TABLE]

Observe here that, since the sum of eigenvalues of $L$ is equal to $\operatorname*{tr}(L)=n$ and $\lambda_{1},\lambda_{2}\geqslant 0$, we have $\lambda_{3}\leqslant\frac{n}{n-2}$. Thus, from $\delta\leqslant 0.8(\lambda_{3}-\lambda_{2})$, we have that $\delta\lambda_{3}\leqslant 0.9(\lambda_{3}-\lambda_{2})$ for any sufficiently large $n$. Plugging this into the above inequality gives

[TABLE]

The above inequality implies that $a_{11}-a_{22}>0$. Moreover, for $n\geqslant 1000$, we have

[TABLE]

In other words, the conditions in Fact D.1 and Observation D.2 are satisfied. From Fact D.1, the eigenvalues of $A$ are

[TABLE]

and

[TABLE]

Furthermore, the eigenvectors of $A$ are

[TABLE]

Let $\gamma_{1}\triangleq\left(\frac{a_{11}-\alpha_{2}}{\alpha_{1}-\alpha_{2}}\right)y^{(0)}+\left(\frac{a_{12}}{\alpha_{1}-\alpha_{2}}\right)z^{(0)}$ and $\gamma_{2}\triangleq\left(\frac{\alpha_{1}-a_{11}}{\alpha_{1}-\alpha_{2}}\right)y^{(0)}-\left(\frac{a_{12}}{\alpha_{1}-\alpha_{2}}\right)z^{(0)}$. It is easy to see that

[TABLE]

Since ${\bf v}_{1}$ and ${\bf v}_{2}$ are eigenvectors of $A$ with eigenvalues $\alpha_{1}$ and $\alpha_{2}$ respectively, we have

[TABLE]

for every $t\in\mathbb{Z}$. In other words, we have

[TABLE]

Having derived the above formula, we will now bound $y^{(t)},z^{(t)}$ by appropriately bounding the eigenvalues and coefficients. Before we do so, let us list a few inequalities that will be useful.

• •

From (57), we have the following three ineqalities.

[TABLE]

[TABLE]

and

[TABLE]

where the second inequality comes from $\varepsilon<1$ and $b\leqslant d$.

• •

From (60) and from Observation D.2, we have

[TABLE]

Note also that the right-most term above is non-negative for sufficiently large $n$.

Bounding $y^{(t)}$.

With the above inequalities in place, it is now easy to bound $y^{(t)}$ as follows.

[TABLE]

as desired.

Bounding $z^{(t)}$.

Recall that $z^{(t)}=\left(\frac{\alpha_{1}-a_{11}}{a_{12}}\right)\gamma_{1}\alpha_{1}^{t}+\left(\frac{\alpha_{2}-a_{11}}{a_{12}}\right)\gamma_{2}\alpha_{2}^{t}$. Let us bound the two terms separately, starting with the first term $\left(\frac{\alpha_{1}-a_{11}}{a_{12}}\right)\gamma_{1}\alpha_{1}^{t}$. To this end, we can bound $\left(\frac{\alpha_{1}-a_{11}}{a_{12}}\right)\gamma_{1}$ as follows.

[TABLE]

Note that the first inequality comes from Observation D.2 and from $\gamma_{1}\geqslant 0$. From the above bound on $\left(\frac{\alpha_{1}-a_{11}}{a_{12}}\right)\gamma_{1}$ and our bound on $\alpha_{1}$ from (61), we have

[TABLE]

Let us next bound $\left(\frac{\alpha_{2}-a_{11}}{a_{12}}\right)\gamma_{2}\alpha_{2}^{t}$. Again, we first rearrange the coefficient $\left(\frac{\alpha_{2}-a_{11}}{a_{12}}\right)\gamma_{2}$ as

[TABLE]

Hence, from the above inequality and $(\ref{eq:eigenvaluesinq})$, we have

[TABLE]

Combining (62) and (63) indeed yields the desired bound on $z^{(t)}$. ∎

D.1.6 Proofs of Lemma D.6 and Lemma D.7

Proof of Lemma D.6.

Since $\kappa=1+40\varepsilon b\beta/d$ and since $\varepsilon b\beta/d\leqslant 1$, it suffices to show that $(\xi_{1}/\xi)^{t}\leqslant 1+\varepsilon b\beta/d$. To show this, let us rearrange $(\xi/\xi_{1})^{t}$ as follows.

[TABLE]

Note that we can apply Bernoulli’s inequality since $\frac{672\delta^{2}b}{dn^{2}}\leqslant 1$ for any sufficiently large $n$ (i.e. $n\geqslant 30$). Finally, note that the above inequality implies that $(\xi_{1}/\xi)^{t}\leqslant 1+\varepsilon b\beta/d$ since $\frac{1}{1-\frac{\varepsilon b\beta}{2d}}\leqslant 1+\varepsilon b\beta/d$ because $\varepsilon b\beta/d\leqslant 1$. ∎

Proof of Lemma D.7.

Observe that, in (57), we have already proved that $\xi-\xi_{2}\geqslant\frac{2\delta^{2}}{\varepsilon n}=\frac{2\delta(\lambda_{3}-\lambda_{2})}{n}$. Moreover, observe that $\xi_{2}\leqslant 1$. Hence, we have

[TABLE]

which implies the inequality stated in the lemma. ∎

Appendix E Omitted Proofs from Section 5.2

The main goal of this section is to prove Theorem 5.3. The actual proof deviates in a couple of subtle ways from the outline in Section 5.2.1:

• •

Firstly, we use a slightly different notion of “good at time $t$”. In the outline, we say that a node is good at time $t$ if $\chi_{u}({\bf x}_{u}^{(t)}-a_{||})\approx\mu(t)/n$. However, since $\chi_{u}\cdot{\bf x}_{u}^{(T_{u}(\tau^{\text{s}}_{u}))}>\chi_{u}\cdot{\bf x}_{u}^{(T_{u}(\tau^{\text{e}}_{u}))}$ suffices to conclude that ${\bf h}^{jump}_{u}=\chi_{u}$, it is enough for us to pick $\eta\in{\mathbb{R}}$ as a cutoff threshold and says that $u$ such that $\chi_{u}({\bf x}_{u}^{(T_{u}(t))}-a_{||})\geqslant\eta$ is good for stored time $t$ and $u$ such that $\chi_{u}({\bf x}_{u}^{(T_{u}(t))}-a_{||})<\eta$ is good for end time $t$. More formally, for each $t\in\mathbb{N}$, let $R^{\eta}_{t}\triangleq\{u\in V\mid\chi_{u}({\bf x}_{u}^{(t)}-a_{||})\geqslant\eta\}$ be the set of good nodes for stored time $t$ and $\bar{R}^{\eta}_{t}\triangleq V\setminus R^{\eta}_{t}$ be the set of good nodes for end time $t$.

• •

Secondly, instead of arguing that $[T_{u}(\tau^{\text{s}}),T_{u}(\tilde{\tau}^{\text{s}})]\cap[0.5n\tau^{\text{s}},0.5n\tilde{\tau}^{\text{s}}]$ is large for most $u$ (and similarly for the ending time), we will argue that $[T_{u}(\tau^{\text{s}}),T_{u}(\tilde{\tau}^{\text{s}})]\subseteq[0.4n\tau^{\text{s}},0.6n\tilde{\tau}^{\text{s}}]$ for most $u$, which suffices for our purpose.

More precisely, the main steps of the proof are as follows. After selecting appropriate values of $\tau^{\text{s}},\tilde{\tau}^{\text{s}},\tau^{\text{e}},\tilde{\tau}^{\text{e}},\eta$, our proof consists of three main steps as follows. For brevity, let us focus on the stored time here as the statements for the end time are analogous.

We start by using the concentration result from the previous section to argue that, for each $t\in[0.4n\tau^{\text{s}},0.6n\tilde{\tau}^{\text{s}}]$, most nodes are good for stored time $t$, i.e., $\mathop{\mathbb{E}}_{\mathcal{E}}|R^{\eta}_{t}|$ is large. In other words, we will show that $\mathop{\mathbb{E}}_{\mathcal{E}}|\bar{R}^{\eta}_{t}|$ is small for such $t$’s. 2. 2.

We next argue that, for most nodes $u$, $T_{u}(\tau^{\text{s}}),\dots,T_{u}(\tilde{\tau}^{\text{s}})$ are “sufficiently uniform” in the following sense: $[T_{u}(\tau^{\text{s}}),T_{u}(\tilde{\tau}^{\text{s}})]\subseteq[0.4n\tau^{\text{s}},0.6n\tilde{\tau}^{\text{s}}]$ and, for most $\tau\in[\tau^{\text{s}},\tilde{\tau}^{\text{s}}]$, $T_{u}(\tau+1)-T_{u}(\tau)$ is not too much smaller than its expected value, $n/2$. 3. 3.

Finally, we show that, if most nodes are uniform, then using local time is not much worse than using global time. In other words, we show that, if the average size of the sets of bad nodes $\bar{R}^{\eta}_{t}$ is small over $t\in[0.4n\tau^{\text{s}},0.6n\tilde{\tau}^{\text{s}}]$, then the average size of $\bar{R}^{\eta}_{T_{u}(\tau)}$ is also small over all $\tau\in[\tau^{\text{s}},\tilde{\tau}^{\text{s}}]$. The latter indeed implies that most $u$ is unlikely to be bad for random stored time $\tau^{\text{s}}_{u}\in[\tau^{\text{s}},\tilde{\tau}^{\text{s}}]$.

The values of the parameters that we will be using throughout this section are as follows.

• •

Pick $\tau^{\text{s}}\triangleq\frac{100\log\left(\frac{nd}{\varepsilon b}\right)}{\delta(\lambda_{3}-\lambda_{2})}$, $\tilde{\tau}^{\text{s}}\triangleq 2\tau^{\text{s}}$, $\tau^{\text{e}}\triangleq 3\tilde{\tau}^{\text{s}}+\frac{10d}{\delta b}$ and $\tilde{\tau}^{\text{e}}\triangleq 2\tau^{\text{e}}$.

• •

Let $\eta\triangleq 0.25\mu(0.6n\tilde{\tau}^{\text{s}})/n$.

E.1 The proof

Let us now proceed to the proof. The two main lemmas of the first step can be stated as follows. Since both lemmas follow easily from our concentration result, we defer their proofs to Appendix E.2.

Lemma E.1**.**

For any ${\bf x}^{(0)}$ such that $\|{\bf z}^{(0)}\|^{2}\leqslant n\sqrt{d/(\varepsilon b)}\|{\bf y}^{(0)}\|^{2}$ and every $t\in[0.4n\tau^{\text{s}},0.6n\tilde{\tau}^{\text{s}}]$, we have $\mathop{\mathbb{E}}_{\mathcal{E}}|\bar{R}^{\eta}_{t}|\leqslant O\left(n\sqrt{\varepsilon b/d}\right).$

Lemma E.2**.**

For any ${\bf x}^{(0)}$ such that $\|{\bf z}^{(0)}\|^{2}\leqslant n\sqrt{d/(\varepsilon b)}\|{\bf y}^{(0)}\|^{2}$ and every $t\in[0.4n\tau^{\text{e}},0.6n\tilde{\tau}^{\text{e}}]$, we have $\mathop{\mathbb{E}}_{\mathcal{E}}|R^{\eta}_{t}|\leqslant O\left(n\sqrt{\varepsilon b/d}\right).$

As stated in the proof overview, the second step is to argue that a random sequence of edges $\mathcal{E}$ is “sufficiently uniform” for most node $u$ with high probability. The notion of uniformity needed here is formalized below. Note that the parameters $a,b$ below will later be set to $\frac{\tau^{\text{s}}}{\log n},\frac{\tilde{\tau}^{\text{s}}}{\log n}$ to achieve uniformity for stored time and $\frac{\tau^{\text{e}}}{\log n},\frac{\tilde{\tau}^{\text{e}}}{\log n}$ to achieve uniformity for end time.

Definition E.1**.**

Let $a,b,\zeta$ be any positive real number such that $b\geqslant 2a$. We say that a node $u\in V$ is $(a,b,\zeta)$-uniform (with respect to a sequence of edges $\mathcal{E}$) if

• •

$T_{u}(a\log n)>0.4an\log n$* and $T_{u}(b\log n+1)\leqslant 0.6bn\log n$, and,*

• •

$\mathop{\mathbb{P}}_{\tau\in[a\log n,b\log n]}\left[T_{u}(\tau+1)<T_{u}(\tau)+\sqrt{\zeta}n\right]\leqslant 4\sqrt{\zeta}$.

We can argue that, for a random $\mathcal{E}$, with high probability, most nodes are uniform, as stated below. Since this follows from standard Chernoff bound, we defer the proof to Appendix E.3.

Lemma E.3**.**

With probability $1-n^{-\Omega(\sqrt{\zeta}a)}$, at least $n-n^{1-\Omega(\sqrt{\zeta}a)}$ nodes are $(a,b,\zeta)$-uniform.

To state the main lemma of the final step of the proof, let us defined an additional notation: we call a sequence $\{S_{t}\}_{t\in\mathbb{N}}$ of subsets $S_{t}\subseteq V$ compatible with $\mathcal{E}$ if, for every $t\in\mathbb{N}$, $S_{t}\triangle S_{t+1}\subseteq\{u_{t+1},v_{t+1}\}$, i.e., $S_{t+1}$ can only differ from $S_{t}$ on the endpoints of the edge in step $t$. Observe that the sequences $\{R^{\eta}_{t}\}_{t\in N}$and $\{\bar{R}^{\eta}_{t}\}_{t\in N}$ are compatible with $\mathcal{E}$ since ${\bf x}_{u}^{(t+1)}$ can change only when $u\in\{u_{t+1},v_{t+1}\}$. The main lemma of this part is stated below.

Lemma E.4**.**

For any sequence of edges $\mathcal{E}$ such that at least $(1-\sqrt{\zeta})n$ nodes are $(a,b,\zeta)$-uniform and for any sequence of subsets $\{S_{t}\}_{t\in\mathbb{N}}$ that is compatible with $\mathcal{E}$ and that $\mathop{\mathbb{E}}_{t\in[0.4an\log n,0.6bn\log n]}\left[|S_{t}|\right]\leqslant\zeta n$, we have

[TABLE]

$S_{t}$ should be thought of as the set of bad nodes for $t$; for stored time, we should think of $S_{t}$ as $\bar{R}^{\eta}_{t}$ whereas, for end time, we should think of $S_{t}$ as $R^{\eta}_{t}$. The above lemma asserts that, by randomly choosing a local time from $[a\log n,b\log n]$, most nodesill likely end up in a global time step where it is good. The proof of Lemma E.4 is deferred to Subsection E.4.

Let us now show how to use these lemmas to prove Theorem 5.3.

Proof of Theorem 5.3.

First, note that the fact that every node is labeled at time $O(\frac{n\log n}{\delta(\lambda_{3}-\lambda_{2})}+\frac{nd}{\delta b})$ w.h.p. follows easily from applying a union bound on top of a Chernoff bound on $\mathop{\mathbb{P}}_{\mathcal{E}}[T_{u}(\tilde{\tau}^{\text{e}})\leqslant 100n\tilde{\tau}^{\text{e}}]$ for each node $u\in V$; this latter probability is simply the same as the probability that sum of $100n\tilde{\tau}^{\text{e}}$ i.i.d. Bernoulli random variables each with mean $2/n$ is less than $\tilde{\tau}^{\text{e}}$.

To we prove the reconstruction guarantee, let us define the following notations for brevity:

• •

$\Theta^{\text{initial}}$ denotes the event that $\|{\bf z}^{(0)}\|^{2}\leqslant n\sqrt{d/(\varepsilon b)}\|{\bf y}^{(0)}\|^{2}$.

• •

$\Theta^{\text{s}}$ denotes the event that $\mathop{\mathbb{E}}_{t\in[0.4n\tau^{\text{s}},0.6n\tau^{\text{s}}]}|\bar{R}^{\eta}_{t}|\leqslant n\sqrt[4]{\varepsilon b/d}$.

• •

$\Theta^{\text{e}}$ denotes the event that $\mathop{\mathbb{E}}_{t\in[0.4n\tau^{\text{e}},0.6n\tau^{\text{e}}]}|R^{\eta}_{t}|\leqslant n\sqrt[4]{\varepsilon b/d}$.

• •

Let $a^{\text{s}}\triangleq\frac{\tau^{\text{s}}}{\log n},b^{\text{s}}\triangleq\frac{\tilde{\tau}^{\text{s}}}{\log n},a^{\text{e}}\triangleq\frac{\tau^{\text{e}}}{\log n}$ and $b^{\text{e}}\triangleq\frac{\tilde{\tau}^{\text{e}}}{\log n}$.

• •

Let $\zeta\triangleq\max\left\{\sqrt{\frac{\varepsilon b}{d}},\frac{1}{\log n}\right\}$.

• •

$\Theta^{\text{uniform},\text{s}}$ denotes the event that at least $(1-\sqrt{\zeta})n$ nodes are $(a^{\text{s}},b^{\text{s}},\zeta)$-uniform.

• •

$\Theta^{\text{uniform},\text{e}}$ denotes the event that at least $(1-\sqrt{\zeta})n$ nodes are $(a^{\text{e}},b^{\text{e}},\zeta)$-uniform.

• •

$\Theta$ denotes the event that $\Theta^{\text{initial}},\Theta^{\text{s}},\Theta^{\text{e}},\Theta^{\text{uniform},\text{s}}$ and $\Theta^{\text{uniform},\text{e}}$ all occur.

Note that $\Theta$ here is the “nice” event, where the conditions required in Lemma E.1, Lemma E.2 and Lemma E.4 are satisfied and we can invoke them. Our proof will proceed in two steps: we will first show that the probability that $\Theta$ occurs is large and, then, we will use our auxiliary lemmas to show that, conditioned on $\Theta$ happening, we achieve the desired reconstruction most of the time.

To bound $\mathop{\mathbb{P}}_{{\bf x}^{(0)},\mathcal{E}}\left[\neg\Theta\right]$, first note that, from Proposition 1, we have $\mathop{\mathbb{P}}_{{\bf x}^{(0)}}[\neg\Theta^{\text{initial}}]\leqslant O(\sqrt[4]{\varepsilon b/d})$. Moreover, from Lemma E.1, we have

[TABLE]

From Markov’s inequality, this implies that $\mathop{\mathbb{P}}_{{\bf x}^{(0)},\mathcal{E}}[\neg\Theta^{\text{s}}\mid\Theta^{\text{initial}}]\leqslant O\left(\sqrt[4]{\varepsilon b/d}\right)$. Similarly, Lemma E.2 implies that $\mathop{\mathbb{P}}_{{\bf x}^{(0)},\mathcal{E}}[\neg\Theta^{\text{e}}\mid\Theta^{\text{initial}}]\leqslant O\left(\sqrt[4]{\varepsilon b/d}\right)$. Now, note that, since $\zeta\geqslant 1/\log n$ and $a^{\text{s}}\geqslant 1$, we have that $\sqrt{\zeta}n\geqslant n^{1-\Omega(\sqrt{\zeta}a^{\text{s}})}$ for sufficiently large $n$. Hence, we can apply Lemma E.3, which implies that $\mathop{\mathbb{P}}_{\mathcal{E}}[\neg\Theta^{\text{uniform},\text{s}}]\leqslant n^{-\Omega(\sqrt{\zeta}a)}\leqslant O(\sqrt{\zeta})$. Similarly, we also have $\mathop{\mathbb{P}}_{\mathcal{E}}[\neg\Theta^{\text{uniform},\text{e}}]\leqslant O(\sqrt{\zeta})$. Finally, by combining these four bounds, we have

[TABLE]

We now proceed to the second part of the proof. Let us denote the set of nodes incorrectly labeled by $V^{\text{incorrect}}$, i.e., $V^{\text{incorrect}}=\left\{u\in V\mid\chi_{u}\left({\bf x}_{T_{u}(\tau_{u}^{\text{s}})}-{\bf x}_{T_{u}(\tau_{u}^{\text{e}})}\right)<0\right\}$. We will show that

[TABLE]

Before we prove the above inequality, let us first show how this implies the desired reconstruction property. By applying Markov’s inequality to (65), we can conclude that $\mathop{\mathbb{P}}\left[|V^{\text{incorrect}}|\geqslant n\sqrt[4]{\zeta}/2\text{ }\middle|\text{ }\Theta\right]\leqslant O(\sqrt[4]{\zeta})$. From this and from (64), we can conclude that $\mathop{\mathbb{P}}\left[|V^{\text{incorrect}}|\geqslant n\sqrt[4]{\zeta}/2\right]\leqslant O(\sqrt[4]{\zeta})$. Note that, when $|V^{\text{incorrect}}|\leqslant n\sqrt[4]{\zeta}/2$, the protocol achieves a $\sqrt[4]{\zeta}$-weak reconstruction. Hence, with probability $1-O(\sqrt[4]{\zeta})$, our protocol achieves a $\sqrt[4]{\zeta}$-weak reconstruction of the graph. Since $\zeta=\max\{\sqrt{\varepsilon b/d},1/\log n\}$, this indeed implies the reconstruction property as stated in Theorem 5.3.

Finally, let us next prove (65) and complete our proof of Theorem 5.3. Since $\{\tau_{u}^{\text{s}}\}_{u\in V},\{\tau_{u}^{\text{e}}\}_{u\in V}$ are independent of $\Theta$, we can write the left-hand side of (65) as

[TABLE]

Hence, it suffices for us to show that, assuming that $\Theta$ happens, $\mathop{\mathbb{P}}_{u\in V,\tau_{u}^{\text{s}},\tau_{u}^{\text{e}}}[u\in V^{\text{incorrect}}]\leqslant O(\sqrt{\zeta})$. To prove this, first observe that if $u\in V^{\text{incorrect}}$, then either $u\in\bar{R}^{\eta}_{T_{u}(\tau^{\text{s}}_{u})}$ or $u\in R^{\eta}_{T_{u}(\tau^{\text{e}}_{u})}$ or both. Thus, we have

[TABLE]

Since we assume that $\Theta$ occurs, $\Theta^{\text{uniform},\text{s}}$ also occurs, which means that we can apply Lemma E.4 for the sequence $S_{t}=\bar{R}^{\eta}_{t},a=a^{\text{s}}$ and $b=b^{\text{s}}$. This implies that

[TABLE]

Similarly, applying Lemma E.4 with $S_{t}=R^{\eta}_{t},a=a^{\text{e}}$ and $b=b^{\text{e}}$, we also have

[TABLE]

By combining the above three inequalities, we indeed arrive at the desired bound. ∎

E.2 Bounding $|R^{\eta}_{t}|$ and $|\bar{R}^{\eta}_{t}|$: Proofs of Lemma E.1 and Lemma E.2

Proof of Lemma E.1.

Consider any $t\in[0.4n\tau^{\text{s}},0.6n\tilde{\tau}^{\text{s}}]$. For brevity, let $\Theta$ denote an event that $a_{y}(t)\in[0.5\mu(t),1.5\mu(t)]$ and $\|{\bf z}(t)\|\leqslant\left(0.5\sqrt[4]{\varepsilon b/d}\right)\mu(t)$. Applying Theorem D.1 with $\beta=\sqrt{\frac{d}{\varepsilon b}}$, we have $\mathop{\mathbb{P}}_{\mathcal{E}}[\Theta]\geqslant 1-O(\sqrt{\varepsilon b/d})$.

Now, let us bound the size of $\bar{R}^{\eta}_{t}$ conditioned on $\Theta$ happening. To do so, first recall that, since ${\bf x}(t)=a_{||}\cdot({\mathbf{1}}/\sqrt{n})+a_{y}(t)\cdot(\chi/\sqrt{n})+{\bf z}(t)$, we have ${\bf x}_{u}(t)=a_{||}+\frac{a_{y}(t)\chi_{u}}{\sqrt{n}}+{\bf z}_{u}(t)$. Hence, we have

[TABLE]

where the inequality comes from $a_{y}(t)\geqslant 0.5\mu(t)$ and from $\mu(t)\geqslant\mu(0.6n\tilde{\tau}^{\text{s}})$. This inequality implies that, if $u\in\bar{R}^{\eta}_{t}$, then $|{\bf z}_{u}(t)|>\frac{0.25\mu(t)}{\sqrt{n}}$. As a result, we can conclude that

[TABLE]

where the latter comes from $\|{\bf z}(t)\|\leqslant\left(0.5\sqrt[4]{\varepsilon b/d}\right)\mu(t)$.

Thus, we have

[TABLE]

as desired. ∎

The proof of Lemma E.1 is analogous to the above proof and is presented below.

Proof of Lemma E.2.

Consider any $t\in[0.4n\tau^{\text{e}},0.6n\tilde{\tau}^{\text{e}}]$. First of all, let us note that

[TABLE]

where the second inequality comes from the fact that $1+x\leqslant e^{x}$ for all $x\in{\mathbb{R}}$.

The rest of the proof proceeds similar to the proof of Lemma E.1. Again, let $\Theta$ denote an event that $a_{y}(t)\in[0.5\mu(t),1.5\mu(t)]$ and $\|{\bf z}(t)\|\leqslant\left(0.5\sqrt[4]{\varepsilon b/d}\right)\mu(t)$. From Theorem D.1 with $\beta=\sqrt{\frac{d}{\varepsilon b}}$, we have $\mathop{\mathbb{P}}_{\mathcal{E}}[\Theta]\geqslant 1-O(\sqrt{\varepsilon b/d})$.

Conditioned on $\Theta$, observe that

[TABLE]

where the inequality comes from $a_{y}(t)\leqslant 1.5\mu(t)$ and (66). Hence, if $u\in R^{\eta}_{t}$, then $|{\bf z}_{u}(t)|>\frac{0.5\mu(t)}{\sqrt{n}}$. Since $\|{\bf z}(t)\|\leqslant\left(0.5\sqrt[4]{\varepsilon b/d}\right)\mu(t)$, this implies that $|R^{\eta}_{t}|<\frac{\|{\bf z}(t)\|^{2}}{(0.5\mu(t)/\sqrt{n})^{2}}\leqslant n\sqrt{\varepsilon b/d}$.

Thus, we have

[TABLE]

as desired. ∎

E.3 Most Vertices are Uniform: Proof of Lemma E.3

Proof of Lemma E.3.

Let us fix a vertex $u\in V$. We will compute the probability that $u$ is $(a,b,\zeta)$-uniform. First, we will bound the probability that the first condition is not satisfied. To do so, let us introduce an additional notation; we use $X_{t}$ to denote an indicator variable of the event $u\in\{u_{t},v_{t}\}$. Note each $X_{t}$ is an i.i.d. Bernoulli random variable which is one with probability $2/n$. The probability that $T_{u}(a\log n)<0.4an\log n$ can now be written in terms of $X_{t}$’s as follows.

[TABLE]

where the inequality comes from an application of Chernoff bound. Similarly, we get the following bound for $\mathop{\mathbb{P}}_{\mathcal{E}}[T_{u}(b\log n+1)>0.6b\log n]$:

[TABLE]

Next, we proceed to bound the probability that the second condition fails. Let $Y_{\tau}$ denote an indicator variable of the event $T_{u}(\tau+1)<T_{u}(\tau)+\sqrt{\zeta}n$. Observe that $Y_{\tau}$’s are i.i.d. Moreover, the probability that $Y_{\tau}=1$ can be bounded as follows.

[TABLE]

Hence, by Chernoff bound, we have

[TABLE]

Observe that $\sum_{\tau=a\log n}^{b\log n}Y_{\tau}\leqslant 4\sqrt{\zeta}(b-a)\log n$ is equivalent to

[TABLE]

Thus, (67), (68) and (69) together with the fact that $b=2a$ imply that the probability that $u$ is $(a,b,\zeta)$-standard is at least $1-2^{-\Omega(\sqrt{\zeta}a\log n)}$, which is at least $1-n^{-C\sqrt{\zeta}a}$ for some global constant $C$.

As a result, the expected number of vertices that are not $(a,b,\zeta)$-uniform is at most $n^{1-C\sqrt{\zeta}a}$. Hence, an application of Markov’s inequality implies that, with probability at most $n^{C\sqrt{\zeta}a/2}$, the number of non-uniform vertices is at most $n^{1-C\sqrt{\zeta}a/2}$, which concludes the proof of this lemma. ∎

E.4 From Global to Local Time: Proof of Lemma E.4

In this subsection, we present the proof of Lemma E.4.

Proof of Lemma E.4.

Let $St\subseteq V$ denote the set of $(a,b,\zeta)$-uniform vertices. We can first write the left-handside term so that we separate out the uniform $u$’s from the non-uniform ones as follows.

[TABLE]

where the last inequality comes from our assumption that there are only $\sqrt{\zeta}n$ non-uniform vertices. For each vertex $u\in V$, denote the set of $\tau\in[a\log n,b\log n]$ such that $T_{u}(\tau+1)-T_{u}(\tau)<\sqrt{\zeta}n$ by $R_{u}$. We can further bound the term $\mathop{\mathbb{P}}_{u\in V,\tau\in[a\log n,b\log n]}\left[u\in St\wedge u\in S_{T_{u}(\tau)}\right]$ by

[TABLE]

where the second inequality comes from the fact that, if $u$ is $(a,b,\zeta)$-uniform, then $\mathop{\mathbb{P}}_{\tau\in[a\log n,b\log n]}\left[\tau\in R_{u}\right]\leqslant 4\sqrt{\zeta}$. Hence, to prove the intended inequality, it suffices to show that

[TABLE]

Observe that this probability can be further rearranged as

[TABLE]

Let us fix $u\in St$ and $\tau\in[a\log n,b\log n]\setminus R_{u}$. Recall that, since $\{S_{t}\}_{t\in\mathbb{N}}$ is compatible with $\mathcal{E}$, we have $\mathds{1}[u\in S_{t}]=\mathds{1}[u\in S_{T_{u}(\tau)}]$ for every $t\in[T_{u}(\tau),T_{u}(\tau+1))$. Moreover, because $\tau\notin R_{u}$, we have $T_{u}(\tau+1)-T_{u}(\tau)\geqslant\sqrt{\zeta}n$. Thus, we have

[TABLE]

Plugging the above inequality back into (E.4), the probability

[TABLE]

can be upper bounded by

[TABLE]

Finally, recall from definition of uniform vertices that, if $u$ is $(a,b,\zeta)$-uniform, then $T_{u}(a\log n)>0.4an\log n$ and $T_{u}(b\log n+1)\leqslant 0.6b\log n$. Combining this with the above inequality, we have

[TABLE]

which concludes our proof. ∎