New Notions and Constructions of Sparsification for Graphs and Hypergraphs

TL;DR

This paper introduces new notions of graph and hypergraph sparsification that approximate cut structures with additive error, providing probabilistic and deterministic constructions with near-optimal edge bounds and extending to hypergraphs.

Contribution

It proposes a weaker additive sparsification framework, offers polynomial time algorithms for graphs and hypergraphs, and improves spectral hypergraph sparsifier constructions.

Findings

Probabilistic polynomial time construction of additive sparsifiers with $O(rac{1}{\\epsilon^2} n \\text{polylog}(1/\\epsilon))$ edges.

Deterministic polynomial time construction with $O(rac{1}{\\epsilon^2} n)$ edges.

First non-trivial additive sparsification for hypergraphs with $O(n)$ hyperedges for constant $\ ext ext{epsilon}$ and hyperedge rank.

Abstract

A sparsifier of a graph $G$ (Bencz\'ur and Karger; Spielman and Teng) is a sparse weighted subgraph $\tilde G$ that approximately retains the cut structure of $G$. For general graphs, non-trivial sparsification is possible only by using weighted graphs in which different edges have different weights. Even for graphs that admit unweighted sparsifiers, there are no known polynomial time algorithms that find such unweighted sparsifiers. We study a weaker notion of sparsification suggested by Oveis Gharan, in which the number of edges in each cut $(S,\bar S)$ is not approximated within a multiplicative factor $(1+\epsilon)$, but is, instead, approximated up to an additive term bounded by $\epsilon$ times $d\cdot |S| + \text{vol}(S)$, where $d$ is the average degree, and $\text{vol}(S)$ is the sum of the degrees of the vertices in $S$. We provide a probabilistic polynomial time construction of such sparsifiers for every graph, and our sparsifiers have a near-optimal number of edges $O(\epsilon^{-2} n {\rm polylog}(1/\epsilon))$. We also provide a deterministic polynomial time construction that constructs sparsifiers with a weaker property having the optimal number of edges $O(\epsilon^{-2} n)$. Our constructions also satisfy a spectral version of the ``additive sparsification'' property. Our construction of ``additive sparsifiers'' with $O_\epsilon (n)$ edges also works for hypergraphs, and provides the first non-trivial notion of sparsification for hypergraphs achievable with $O(n)$ hyperedges when $\epsilon$ and the rank $r$ of the hyperedges are constant. Finally, we provide a new construction of spectral hypergraph sparsifiers, according to the standard definition, with ${\rm poly}(\epsilon^{-1},r)\cdot n\log n$ hyperedges, improving over the previous spectral construction (Soma and Yoshida) that used $\tilde O(n^3)$ hyperedges even for constant $r$ and $\epsilon$.

Equations

$$\forall S\subseteq V|e_F(S)-e_E(S)|\le ϵ\cdot e_E(S).$$

$$\forall S\subseteq V|{\mathbf{x}}^T{L}_{\tilde{G}}\mathbf{x}-{\mathbf{x}}^T{L}_G\mathbf{x}|\le ϵ\cdot {\mathbf{x}}^T{L}_G\mathbf{x},$$

$$-ϵL_G⪯L_{\tilde{G}}-L_G⪯ϵL_G.$$

$$\forall S\subseteq V|e_F(S)-e_E(S)|\le ϵe_E(S).$$

$$Q_H(\mathbf{x})=\sum _{e\in E}{w}_e\cdot \underset{a,b\in e}{\max }(x_a-x_b{)}^2$$

$$\forall \mathbf{x}\in {\mathbb{R}}^V|Q_{\tilde{H}}(\mathbf{x})-Q_H(\mathbf{x})|\le ϵ\cdot Q_H(\mathbf{x}).$$

$$\forall S\subseteq V|c\cdot e_F(S)-e_E(S)|\le 2ϵd\cdot |S|,$$

$$\forall S\subseteq V|c\cdot e_F(S)-e_E(S)|\le ϵ\cdot (d_{\mathrm{avg}}\cdot |S|+\mathrm{vol}(S))$$

$$-ϵ\cdot (D_G+d_{\mathrm{avg}}I)⪯c\cdot L_{\tilde{G}}-L_G⪯ϵ\cdot (D_G+d_{\mathrm{avg}}I).$$

$$2c{D}_{\tilde{G}}-2D_G-ϵD_G-ϵdI⪯c{L}_{\tilde{G}}-L_G⪯ϵD_G+ϵdI$$

$$-ϵ|E|\le c{e}_F(S)-e_E(S)\le ϵ\text{vol}(S)+ϵd|S|$$

$${\mathbf{x}}^{T}M\mathbf{x}=\mathrm{trace}(M)$$

$$c{L}_{\tilde{G}}-L_G⪯ϵ\cdot (dI+D_G)$$

$$cS{L}_{\tilde{G}}-S{L}_G⪯ϵ\cdot (dI+D_G)$$

$$-ϵD_G-ϵdI⪯c{L}_{\tilde{G}}-L_G⪯ϵD_G+ϵdI.$$

$$D_G+dI=2dI⪯O({\varphi }^{-2}{L}_G)$$

$$|c{e}_F(S)-e_E(S)|\le ϵd|S|+ϵ\mathrm{vol}(S)\forall S\subseteq V.$$

$$\underset{\mathbf{x}\in {\mathbb{R}}^V:\Vert \mathbf{x}\Vert =1}{\max }\sum _{e\in H}(1-W_e)\cdot \underset{a,b\in e}{\max }(x_a-x_b{)}^2$$

$$\forall \mathbf{x}\in {\mathbb{R}}^n,{\mathbf{x}}^{T}M\mathbf{x}\ge 0$$

$$\Vert M\Vert =\underset{\mathbf{x}:\Vert \mathbf{x}{\Vert }_2=1}{\sup }\Vert Mx{\Vert }_2$$

$$⟨A,B⟩=\sum _{i,j}{A}_{i,j}{B}_{i,j}=\mathrm{trace}(A^{T}B)$$

$$\Vert M\Vert =\underset{X\mathrm{density}\mathrm{matrix}}{\sup }|M\bullet X|$$

$$M=\sum _{i=1}^n{\lambda }_i{\mathbf{v}}_i{\mathbf{v}}_i^T,$$

$$|M|=\sum _{i=1}^n|{\lambda }_i|{\mathbf{v}}_i{\mathbf{v}}_i^T.$$

$${\mathbf{x}}^T{L}_G\mathbf{x}={\mathbf{x}}^{\prime T}{L}_{G^{\prime }}{\mathbf{x}}^{\prime }$$

$${\mathbf{x}}^T{L}_{\tilde{G}}\mathbf{x}={\mathbf{x}}^{\prime T}{L}_{{\tilde{G}}^{\prime }}{\mathbf{x}}^{\prime }$$

$${\mathbf{x}}^{\prime T}(d_{\max }^{\prime }I){\mathbf{x}}^{\prime }\le {\mathbf{x}}^T(\lceil d_{\mathrm{avg}}\rceil +D_G)\mathbf{x}$$

$${\mathbf{x}}^{\prime T}(d_{\max }^{\prime }I){\mathbf{x}}^{\prime }=\sum _{v\in V}\lceil \frac{d_v}{\lceil d_{\mathrm{avg}}\rceil }\rceil \cdot \lceil d_{\mathrm{avg}}\rceil {\mathbf{x}}_v^2\le \sum _{v\in V}(d_v+\lceil d_{\mathrm{avg}}\rceil )\cdot {\mathbf{x}}_v^2$$

$$e_E(S)=e_{E^{\prime }}(S^{\prime })$$

$$e_F(S)=e_{F^{\prime }}(S^{\prime })$$

Full text

We study a weaker notion of sparsification suggested by Oveis Gharan, in which the number of cut edges in each cut $(S,\bar{S})$ is not approximated within a multiplicative factor $(1+\epsilon)$, but is, instead, approximated up to an additive term bounded by $\epsilon$ times $d\cdot|S|+\text{vol}(S)$, where $d$ is the average degree of the graph and $\text{vol}(S)$ is the sum of the degrees of the vertices in $S$. We provide a probabilistic polynomial time construction of such sparsifiers for every graph, and our sparsifiers have a near-optimal number of edges $O(\epsilon^{-2}n{\rm polylog}(1/\epsilon))$. We also provide a deterministic polynomial time construction that constructs sparsifiers with a weaker property having the optimal number of edges $O(\epsilon^{-2}n)$. Our constructions also satisfy a spectral version of the “additive sparsification” property.

Notions of sparsification have also been studied for hypergraphs. Our construction of “additive sparsifiers” with $O_{\epsilon}(n)$ edges also works for hypergraphs, and provides the first non-trivial notion of sparsification for hypergraphs achievable with $O(n)$ hyperedges when $\epsilon$ and the rank $r$ of the hyperedges are constant. Finally, we provide a new construction of spectral hypergraph sparsifiers, according to the standard definition, with ${\rm poly}(\epsilon^{-1},r)\cdot n\log n$ hyperedges, improving over the previous spectral construction (Soma and Yoshida) that used $\tilde{O}(n^{3})$ hyperedges even for constant $r$ and $\epsilon$.

1 Introduction

Benczúr and Karger [5] introduced the notion of a cut sparsifier: a weighted graph $\tilde{G}=(V,F)$ is an $\epsilon$ cut sparsifier of a graph $G=(V,E)$ if, for every cut $(S,V-S)$ of the set of vertices, the weighted number of cut edges in $\tilde{G}$ is the same as the number of cut edges in $G$, up to multiplicative error $\epsilon$, that is,

[TABLE]

where $e_{X}(S)$ denotes the weighted number of edges in $X$ leaving the set $S$. A stronger notion, introduced by Spielman and Teng [13], is that of a spectral sparsifier: according to this notion, a weighted graph $\tilde{G}=(V,F)$ is an $\epsilon$ cut sparsifier of a graph $G=(V,E)$ if

[TABLE]

where $L_{X}$ is the Laplacian matrix of the graph $X$. Note that (1) is implied by (2) by taking ${\bf x}$ to be the 0/1 indicator vector of $S$. A more compact way to express (2) is as

[TABLE]

Batson, Spielman and Srivastava [3] show that, for every graph, an $\epsilon$ spectral sparsifier (and hence also an $\epsilon$ cut sparsifier) can be constructed in polynomial time with $O(n/\epsilon^{2})$ weighted edges, which is best possible up to the constant in the big-Oh. Sparsifiers have several applications to speeding-up graph algorithms.

For some graphs $G$, for example the “barbell” graph (that consists of two disjoint cliques joined by a single edge), it is necessary for a non-trivial sparsifier of $G$ to have edges of different weights. This has motivated the question of whether there are weaker, but still interesting, notion of sparsification that can be achieved, for all graphs, using sparsifiers that are “unweighted” in the sense that all edges have the same weight.

Question 1**.**

Is a non-trivial notion of unweighted sparsification possible for all graphs?

Results on unweighted sparsification have focused on bounding the multiplicative error $\epsilon$ in such cases, allowing it to be superconstant [2, 1]. For graphs such as the barbell example one, however gets, necessarily, very poor bounds. But is there an alternative notion for which one can get arbitrarily good approximation on all graphs using a linear number of edges?

If one restricts this question from all graphs to selected classes of graphs, then a number of interesting results are known, and some major open questions arise.

If $G=(V,E)$ is a $d$-regular graph such that every edge has effective resistance $O(1/d)$, the Marcus-Spielman-Srivastava [10] proof of the Kadison-Singer conjecture (henceforth, we will refer to this result as the MSS Theorem) implies that $G$ can be partitioned into almost-regular unweighted spectral sparsifiers with error $\epsilon$ and average degree $O(\epsilon^{-2})$. An interesting class of such graphs are edge-transitive graphs, such as the hypercube.

Another interesting class of graphs all whose edges have effective resistance $O(1/d)$ is the class of $d$-regular expanders of constant normalized expansion $\phi>0$. Before the MSS Theorem, Frieze and Molloy [7] proved that such graphs can be partitioned into unweighted almost-regular graphs of average degree $O(\epsilon^{-2}\log d)$ and normalized edge expansion at least $\phi-\epsilon$. They also show how to construct such a partition in randomized polynomial time under an additional small-set expansion assumption on $G$. Becchetti et al. [4] present a randomized linear time algorithm that, given a dense regular expander $G$ of degree $d=\Omega(n)$ finds an edge-induced expander in $G$ of degree $O(1)$. While both [7] and [4] find sparse expanders inside dense expanders, the work of Frieze and Molloy does not produce constant-degree graphs and the work of Becchetti et al. only applies to very dense graphs. Furthermore, neither work guarantees that one ends up with a sparse graph that is a good sparsifier of the original one.

Question 2**.**

Is there a polynomial time construction of the unweighted spectral sparsifiers of expanders whose existence follows from the Marcus-Spielman-Srivastava theorem?

Notions of cut sparsifiers [9] and spectral sparsifiers [12] have been defined for hypergraphs, generalizing the analogous definitions for graphs. In a hypergraph $H=(V,E)$, a hyperedge $e\in E$ is cut by a partition $(S,V-S)$ of the vertices if $e$ intersects both $S$ and $V-S$. As for graphs, we can define $e_{E}(S)$ to be the (weighted, if applicable) number of hyperdges in $E$ that are cut by $(S,V-S)$. As before, a weighted subset of edges $F$ defines a hypergraph cut sparsifier with error $\epsilon$ if

[TABLE]

Kogan and Krauthgamer [9] show how to construct such a (weighted) sparsifier in randomized polynomial time using $O(\epsilon^{-2}n\cdot(r+\log n))$ hyperedges where $r$ is the maximum size of the hyperedges which is also called the rank of the hypergraph.

In order to define a notion of spectral sparsification, we associate to a hypergraph $H=(V,E)$ the following analog of the Laplacian quadratic form, namely a function $Q_{H}$ such that

[TABLE]

where $w_{e}$ is the weight (if applicable) of hyperedge $e$. Note that with this definition we have that if ${\bf x}={\bf 1}_{S}$ for some subset $S$ of vertices then $Q_{H}({\bf x})=e_{E}(S)$. Following Soma and Yoshida [12], we say that a weighted hypergraph $\tilde{H}$ is a spectral sparsifier with error $\epsilon$ of $G$ if we have

[TABLE]

Soma and Yoshida [12] provide a randomized polynomial time construction of such sparisifiers, using $\tilde{O}(\epsilon^{-2}n^{3})$ hyperedges.

Question 3**.**

Is it possible, for every hypergraph, to construct a weighted spectral sparsifier with $\tilde{O}_{r,\epsilon}(n)$ hyperedges?

As in the case of graphs, it is also natural to raise the following question.

Question 4**.**

Is a non-trivial notion of unweighted sparsification possible for all hypergraps?

We provide a positive answer to all the above questions.

1.1 Our Results

1.1.1 Sparsification with additive error

Oveis-Gharan suggested the following weakened definition of sparsification: if $G=(V,E)$ is $d$-regular, we say that an unweighted graph $\tilde{G}=(V,F)$ is an additive cut sparsifier of $G$ with error $\epsilon$ if we have

[TABLE]

where $c=|E|/|F|$. Note that this (up to a constant factor change in the error parameter $\epsilon$) is equivalent to the standard notion if $G$ has constant normalized edge expansion, because $e_{E}(S)$ and $d\cdot S$ will then be within a constant factor of each other. On non-expanding graphs, however, this definition allows higher relative error on sparse cuts and a tighter control on expanding cuts. (The factor of 2 has no particular meaning and it is just there for consistency with the definition that we give next for non-regular graphs.)

For non-regular graphs $G$, we say that $\tilde{G}=(V,F)$ is an additive cut sparsifier of $G$ with error $\epsilon$ if we have

[TABLE]

where $c=|E|/|F|$ and $d_{\rm avg}:=2|E|/|V|$ is the average degree of $G$ and ${\rm vol}(S)$ is the volume of $S$ that is, the sum of the degrees of the vertices in $S$. In can be shown that both terms are necessary if one wants a definition of unweighted sparsification that is applicable to all graphs.

This notion has a natural spectral analog, which we state directly in the more general form:

[TABLE]

Note, again, that if $G$ is a regular expander then this definition is equivalent to the standard definition of spectral sparsifier.

In a hypergraph, the degree of a vertex is the number of hyperedges it belongs to, and the volume of a set of vertices is the sum of the degrees of the vertices that belong to it. With these definitions in mind, the notion of additive graph sparsifier immediately generalizes to hypergraphs.

1.1.2 New Graph Sparsification Constructions

Our first result is a deterministic polynomial time construction which achieves a weak form of unweighted additive sparsification.

Theorem 1.1** (Deterministic Polynomial Time Construction).**

Given a graph $G=(V,E)$ and a parameter $\epsilon>0$, in deterministic polynomial time we can find a subset $F\subseteq E$ of size $|F|=O(n/\epsilon^{2})$ such that, if we let $L_{G}=D_{G}-A_{G}$ be the Laplacian of $G$, $L_{\tilde{G}}=D_{\tilde{G}}-A_{\tilde{G}}$ be the Laplacian of the graph $\tilde{G}=(V,F)$, $d=2|E|/|V|$ be the average degree of $G$, and $c=|E|/|F|$, we have

[TABLE]

Note, in particular, that we get that for every set of vertices $S\subseteq V$ we have

[TABLE]

The first inequality follows by computing the quadratic forms of (4) with the $\pm 1$ indicator vector ${\bf x}:={\bf 1}_{S}-{\bf 1}_{\bar{S}}$ of $S$, and noting that ${\bf x}^{T}L_{G}{\bf x}=4e_{E}(S)$, that ${\bf x}^{T}L_{\tilde{G}}{\bf x}=4e_{F}(S)$, that

[TABLE]

for every diagonal matrix $M$, and that ${\rm trace}(D_{G})={\rm trace}(cD_{\tilde{G}})={\rm trace}(dI)=2|E|$. The second inequality follows by computing the quadratic forms of (4) with the $0/1$ indicator vector ${\bf x}={\bf 1}_{S}$ of $S$, and noting that ${\bf x}^{T}L_{G}{\bf x}=e_{E}(S)$, ${\bf x}^{T}L_{\tilde{G}}{\bf x}=e_{F}(S)$, ${\bf x}^{T}I{\bf x}=|S|$ and ${\bf x}^{T}D_{G}{\bf x}={\rm vol}(S)$.

Our proof is based on the online convex optimization techniques of Allen-Zhu, Liao and Orecchia [1]. The construction of [1] involves weights for two reasons: one reason is a change of basis that maps $L_{G}$ to identity, a step that is not necessary in our setting and that could also be avoided in their setting if $G$ is a graph all whose edges have bounded effective resistance. The second reason is more technical, and it is to avoid blowing up the “width” on the online game that they define. The second issue comes up when one wants to prove $cL_{\tilde{G}}-L_{G}\succeq-\epsilon\cdot(D_{G}+dI)$, but is not a problem for the upper bound $cL_{\tilde{G}}-L_{G}\preceq\epsilon\cdot(D_{G}+dI)$.

To sidestep this problem, we set the goals of proving the bounds

[TABLE]

[TABLE]

where $SL_{G}$ denotes the signless Laplacian of a graph $G$, defined as $D_{G}+A_{G}$. Note that the above PSD inequalities are equivalent to (4).

The reasons why, when our goal is the PSD inequalities above, we are able to control the width without scaling (and without weighing the edges) are quite technical, and we defer further discussion to Section 3.

Our next result is a probabilistic construction of sparsifiers with additive error matching the Oveis-Gharan definition.

Theorem 1.2** (Probabilistic Polynomial Time Construction).**

Given an $n$-vertex graph $G=(V,E)$ and a parameter $\epsilon>0$, in probabilistic polynomial time we can find a subset $F\subseteq E$ of size111where $\tilde{O}(\cdot)$ hides $\log(1/\varepsilon)^{O(1)}$ factors $|F|=n\cdot\tilde{O}(1/\epsilon^{2})$ such that, if we let $L_{G}=D_{G}-A_{G}$ be the Laplacian of $G$, $L_{\tilde{G}}=D_{\tilde{G}}-A_{\tilde{G}}$ be the Laplacian of the graph $\tilde{G}=(V,F)$, $d=2|E|/|V|$ be the average degree of $G$, and $c=|E|/|F|$, we have

[TABLE]

When we apply the above result to a $d$-regular expander $G$, we obtain a graph $\tilde{G}$ whose average (and maximum) degree is $\tilde{O}(\epsilon^{-2})$ and which is itself a good expander. More precisely, if $G$ has normalized edge expansion $\phi$ and $\tilde{G}$ is as above, then the normalized edge expansion of $\tilde{G}$ is about $\phi-2\epsilon$. Recall that Frieze and Molloy can find a $\tilde{G}$ as above but with degree $O(\epsilon^{-2}\log d)$ rather than $O(\epsilon^{-2}{\rm polylog}\epsilon^{-1})$. Furthermore, if $G$ is a $d$-regular expander of normalized edge expansion $\phi$, we have222There is some abuse of notation in (7), because (7) only holds in the space orthogonal to ${\bf 1}=(1,1,\cdots,1)$. that

[TABLE]

and so the unweighted sparsifier $\tilde{G}$ of $G$ given by the above theorem is also a spectral sparsifier in the standard sense. This answers Questions 1 and 2 of the previous section.

We briefly discuss the techniques in the proof. Following Frieze and Molloy [7] and Bilu and Linial [6], we apply the Lovász Local Lemma [11] (LLL) to construct an additive cut sparsifier. One difficulty with this approach is that one has to verify that the sparsifier approximates each of the exponentially many cuts. Indeed, if one defines a “bad” event for each one of these cuts, there are too many events that are dependent in order to successfully apply LLL. A key insight in [7] is that it is sufficient to verify those cuts $(S,V-S)$ where $S$ induces a connected subgraph. This makes a big difference in graphs of maximal degree $d\ll n$: for a vertex $v$, there are $\approx n^{\ell-1}$ subsets of $\ell$ vertices containing $v$ whereas one can prove that there are at most $\binom{d(\ell-1)}{\ell-1}$ such subsets of size $\ell$ that induce a connected subgraph. This allows one to manage the exponentially many events and get almost optimal results with LLL. Indeed, we obtain a close to optimal average degree $\tilde{O}(\epsilon^{-2})$. This improves upon the average degree bound in $\epsilon^{-2}\log d$ [7] . We achieve this by an iterative procedure that intuitively halves the number of edges, instead of sparsifying the graph “in one go.”

Another difference is that, in contrast to [7] and [6], we can use recent constructive versions of LLL [8] to give an efficient probabilistic time algorithm for finding the sparsifier. To apply the constructive version of LLL in the presence of exponentially bad events, one needs to find a subset of bad events of polynomial size such that the probability that any other bad event is true is negligible. We show that this can be achieved by selecting the subset of events corresponding to cuts $(S,V-S)$ so that $S$ induces a connected graph and $|S|=O(\log_{d}(n))$. This gives us an efficient probabilistic algorithm for finding a cut sparsifier which we also generalize to hypergraphs (as we state in the next section). For graphs, we then adapt the techniques of Bilu and Linial [6] to go from a cut sparsifier to a spectral one. To do so we need to consider some more bad events in the application of LLL than needed by Bilu-Linial who worked with “signings” of the adjacency matrix. Specifically, in addition to the events that they considered, we need to also bound the degree of vertices.

1.1.3 New Hypergraph Sparsification Constructions

Theorem 1.3** (Hypergraph cut sparsification with additive error).**

Given an $n$-vertex hypergraph $H=(V,E)$ of rank $r$ and a parameter $\epsilon>0$, in probabilistic polynomial time we can find a subset $F\subseteq E$ of size $|F|=O\left(\frac{n}{r}\cdot\frac{1}{\epsilon^{2}}\log\frac{r}{\epsilon}\right)$ such that, if we let $d=r|E|/|V|$ be the average degree of $H$, and $c=|E|/|F|$, the following holds with probability at least $1-n^{-2}$:

[TABLE]

The proof follows the same approach as the first part of our proof of Theorem 1.2, and in fact we present directly the proof for hypergraphs, leaving the result for graphs as a corollary. It might seem strange that the number of hyperedges in our sparsifier is, for fixed $\epsilon$, of the form $O\left(\frac{n}{r}\log r\right)$, since, intuitively, the sparsification problem should only become harder when $r$ grows. The reason is that, even in a regular hypergraph, $d|S|$ overestimates the number of hyperedges incident on $S$ by up to a factor of $r$, and so, in order to have a non-trivial guarantee, one has to set $\epsilon<1/r$.

Theorem 1.4** (Hypergraph sparsification with multiplicative error).**

There is a randomized polynomial time algorithm that, given a hypergraph of rank $r$, finds a weighted spectral sparsifier with multiplicative error $\epsilon$ having $O(\epsilon^{-2}r^{3}n\log n)$ hyperedges.

The above result should be compared with the $O(\epsilon^{-2}n^{3}\log n)$ hyperedges of the construction of Soma and Yoshida [12]. Our approach is to provide an “hypergraph analog” of the spectral graph sparsifier construction of Spielman and Srivastava [14]. Given $H$, we construct an associated graph $G$ (in which each hyperedge of $H$ is replaced by a clique in $G$), we compute the effective resistances of the edges of $G$, and we use them to associate a notion of “effective resistance” to the hyperedges of $H$. Then we sample from the set of hyperdedges of $H$ by letting the sampling probability of each hyperedge be proportional to its “effective resistance” and we weigh them so that the expected weight of each hyperedge in the sample is the same. At this point, to bound the error, Spielman and Srivastava complete the proof by applying a matrix concentration bound for the spectral norm of sums of random matrices. For hypergraphs, we would like to have a similar concentration bound on the error given by,

[TABLE]

where $W_{e}$ is a random variable that is 0 if the hyperedge $e$ is not selected and it is its weight in the sparsifier if it is selected, with things set up so that $1-W_{e}$ has expectation zero. (Actually, this would only lead to a sparsifier with additive error: to achieve multiplicative error we have to study an expression such as the one above but after a change of basis defined in terms of the associated graph. For simplicity we will ignore this point in this overview.)

However, unlike in the graph case, the expression in (9) does not correspond to the spectral norm, or any other standard linear-algebraic norm, due to the $\max$ term, and the key difficulty in all previous approaches to the problem was to get suitable upper bounds on this quantity. Our main idea is to consider the quantity $V_{x}=\sum_{e\in H}(1-W_{e})\cdot\max_{a,b\in e}\ \ (x_{a}-x_{b})^{2}$ and view it as a random process indexed by the set of all unit vectors $x$, and directly argue about its supremum over all such $x$, using the technique of generic chaining. In particular, we relate the metric given by the sub-gaussian norm of the increments of the process $V_{x}$ to another suitably defined Gaussian random process on the associated graph $G$ of $H$, which is much easier to analyze. This allows us to relate the bound on the supremum of $V_{x}$ to a related expression on the graph $G$, for which we can use known matrix concentration bounds.

2 Preliminaries

2.1 Linear Algebra Preliminaries

In this paper all matrices will have real-valued entries.

A matrix $M$ is Positive Semidefinite (abbreviated PSD and written $M\succeq{\bf 0}$) if it is symmetric and all its eigenvalues are non-negative. Equivalently, $M$ is PSD if and only if

[TABLE]

that is, the quadratic form of $M$ is always non-negative. The trace of a matrix is the sum of its diagonal entries. For a symmetric matrix, its trace is equal to the sum of its eigenvalues, counted with multiplicities. A density matrix is a PSD matrix of trace one. The operator norm of a matrix $M$ is

[TABLE]

If $M$ is symmetric, then the above is the largest absolute value of the eigenvalues of $M$ and we also refer to it as the spectral norm or spectral radius of the matrix.

If $A$ and $B$ are matrices of the same size, then their Frobenius inner product is defined as

[TABLE]

and we will also sometimes denote it as $A\bullet B$. Note that if $M$ is a symmetric matrix we have

[TABLE]

If $M$ is a symmetric $n\times n$ matrix with spectral decomposition

[TABLE]

then the “absolute value” of $M$ is the PSD matrix

[TABLE]

2.2 Reduction to bounded-degree case

We show that, in proving Theorem 1.1, Theorem 1.2 and Theorem 1.3, it is enough to prove weaker bounds where $d_{\rm avg}I+D_{G}$ is replaced by $d_{\rm max}\cdot I$, and ${\rm vol}(S)$ is replaced by $d_{\max}|S|$, where $d_{\rm max}$ is the maximum degree.

Consider the following construction: given a graph $G=(V,E)$ of average degree $d_{\rm avg}=2|E|/V$, construct a new graph $G^{\prime}=(V^{\prime}E^{\prime})$ such that

• •

To each node $v\in V$ there corresponds a cloud of $\lceil d_{v}/\lceil d_{\rm avg}\rceil\rceil$ nodes in $V$.

• •

To each edge $(u,v)\in E$ there corresponds an edge in $E^{\prime}$ between the cloud of $u$ and the cloud of $v$.

• •

Each vertex in $G^{\prime}$ has degree at most $d^{\prime}_{\rm max}=\lceil d_{\rm avg}\rceil$.

A construction satisfying the above property can be realized by replacing the vertices of $V$, in sequence, by a cloud as required, and then replacing $v$ in the edges incident to $v$ by vertices in the cloud of $v$, in a balanced way.

Now suppose that $F^{\prime}\subseteq E^{\prime}$ is a subset of the edges of $G^{\prime}$ and that $F\subseteq E$ is the set of edges of $G$ corresponding to the edges of $F^{\prime}$. Let $\tilde{G}$ be the graph $\tilde{G}=(V,F)$ and $\tilde{G}^{\prime}=(V,F^{\prime})$. Let ${\bf x}\in{\mathbb{R}}^{V}$ be any vector, and define ${\bf x}^{\prime}\in R^{V}$ to be the vector such that ${\bf x}^{\prime}_{v^{\prime}}={\bf x}_{v}$ if $v^{\prime}$ is in the cloud of $v$. Then we observe that

[TABLE]

[TABLE]

[TABLE]

The only non-trivial statement is the third one. To verify it, we see that the left-hand side is

[TABLE]

This means that we can start from an arbitrary graph $G$, construct $G^{\prime}$ as above, find an unweighted sparsifier $\tilde{G}^{\prime}=(V^{\prime},F^{\prime})$ of $G^{\prime}$, and then obtain a set $F$ of edges such that $\tilde{G}=(V,F)$ is an unweighted sparsifier for $G$, with the property that any bound dependent on $d^{\prime}_{\rm max}I$ on the quality of the sparsification of $\tilde{G}^{\prime}$ becomes a bound in terms of $(\lceil d_{\rm avg}\rceil+D_{G})$ (and we can drop the ceiling at the cost of a constant factor in the error).

If $H=(V,E)$ is a hypergraph we can similarly construct a hypergraph $H^{\prime}=(V^{\prime},E^{\prime})$ such that

• •

To each node $v\in V$ there corresponds a cloud of $\lceil d_{v}/\lceil d_{\rm avg}\rceil\rceil$ nodes in $V$.

• •

To each edge $(u,v)\in E$ there corresponds an edge in $E^{\prime}$ between the cloud of $u$ and the cloud of $v$.

• •

Each vertex in $H^{\prime}$ has degree at most $d^{\prime}_{\rm max}=\lceil d_{\rm avg}\rceil$.

Similarly to the graph case, for every set $S\subseteq V$ we can define a set $S^{\prime}\subseteq V^{\prime}$ (the union of the clouds of vertices in $S^{\prime}$) and for every set $F^{\prime}\subseteq E^{\prime}$ we can define a set of hyperedges $F\subseteq E$ of the same cardinality such that

[TABLE]

[TABLE]

[TABLE]

We also note that, in both constructions, the maximum degree and the average degree of the new graph (or hypergraph) are within a constant factor.

3 Deterministic Construction

In this section we use the online convex optimization approach of Allen-Zhu, Liao and Orecchia [1] to construct a weak form of unweighted additive spectral sparsifiers, and we prove Theorem 1.1. Given the reduction described in Section 2.2, it is enough to prove the following theorem.

Theorem 3.1**.**

There is a deterministic polynomial time algorithm that given a graph $G=(V,E)$ of maximum degree $d_{\max}$ and a parameter $\epsilon$ outputs a multiset $F$ of $O(|V|/\epsilon^{2})$ edges such that the graph $\tilde{G}=(V,F)$ satisfies

[TABLE]

where $c=|E|/|F|$.

We are interested in the following online optimization setting: at each time $t=1,\ldots$, an algorithm comes up with a solution $X_{t}$, which is an $n\times n$ density matrix, and an adversary comes up with a cost matrix $C_{t}$, which is an $n\times n$ matrix, and the algorithm receives a payoff $X_{t}\bullet C_{t}$. The algorithm comes up with $X_{t}$ based on knowledge of $X_{1},\ldots,X_{t-1}$ and of $C_{1},\ldots,C_{t-1}$, while the adversary comes up with $C_{t}$ based on $X_{1},\ldots,X_{t}$ and on $C_{1},\ldots,C_{t-1}$. The goal of the algorithm is to maximize the payoff. After running this game for $T$ steps, one defines the regret of the algorithm as

[TABLE]

Theorem 3.2** (Allen-Zhu, Liao, Orecchia [1]).**

There is a deterministic polynomial algorithm that, given a parameter $\eta>0$, after running for $T$ steps against an adversary that provides cost matrices $C_{t}$ restricted as described below, achieves a regret bound

[TABLE]

Furthermore, if $C_{t}$ is block-diagonal, then $X_{t}$ is also block-diagonal with the same block structure The restrictions on the adversary are that at each step $t$ the cost function $C_{t}$ is positive semidefinite or negative semidefinite and satisfies

[TABLE]

Remark 3.3**.**

The theorem above is the $q=2$ case of Theorem 3.3 in [1]. The Furthermore part is not stated explicitly in [1, Theorem 3.3] but can be verified by inspecting the proof. Note that what we are calling $C_{t}$ corresponds to $-C_{t}$ in the treatment of [1], which is why their cost minimization problem becomes a maximization problem here, and the condition that $C_{t}$ satisfy $\eta X_{t}^{1/4}C_{t}X_{t}^{1/4}\succeq-\frac{I}{4}$ becomes the condition that we have in the above theorem.

To gain some intuition about the way we will use the above theorem, note that the definition of regret implies that we have

[TABLE]

where $\lambda_{\max}(\cdot)$ denotes the largest eigenvalue of the matrix. Now suppose that we play the role of the adversary against the algorithm of Theorem 3.2, and that, at time $t$, we reply to the solution $X_{t}$ of the algorithm with a cost matrix of the form $mL_{a_{t},b_{t}}-L_{G}$ where $m:=|E|$ and $(a_{t},b_{t})$ is an edge chosen so that

[TABLE]

We know that such an edge $(a_{t},b_{t})$ must exist, because the average of the left-hand side above is zero if we compute it for a uniformly chosen random $(a_{t},b_{t})\in E$. After playing this game for $T$ steps we have

[TABLE]

and, calling $F$ the multiset $\{(a_{t},b_{t}):t=1,\ldots,T\}$, calling $\tilde{G}=(V,F)$ the multigraph of such edges and $c=|E|/|F|=m/T$, and noting that $L_{\tilde{G}}=\sum_{t}L_{a_{t},b_{t}}$ we have

[TABLE]

which, provided that we can ensure that $R_{T}$ is small, is one side of the type of bounds that we are trying to prove.

In order to get a two-sided bound, one would like to use the idea that

[TABLE]

and play the above game using, at step $t$, a cost matrix of the form

[TABLE]

where the edge $(a_{t},b_{t})$ is chosen so that

[TABLE]

Then, if we define $c$ and $\tilde{G}$ as above, we would reach the conclusion

[TABLE]

and what remains to do is to see for what value of $T$ we get a sufficiently small regret bound.

Unfortunately this approach runs into a series of difficulties.

First of all, our cost matrix is neither positive semidefinite nor negative semidefinite.

We could make it positive semidefinite by shifting, that is, by adding a multiple of the identity. This is not a problem for the block $mL_{a_{t},b_{t}}-L_{G}$, whose smallest eigenvalue is at most $2d_{\max}$ in magnitude, but it is a serious problem for the block $L_{G}-mL_{a_{t},b_{t}}$, whose smallest eigenvalue is of the order of $-m$: the shift needed to make this block PSD would be so big that the terms $X_{t}\bullet|C_{t}|$ in the regret bound would be too large to obtain any non-trivial result.

Another approach, which is closer to what happens in [1], is to see that the analysis of Theorem 3.2 applies also to block-diagonal matrices in which each block is either positive semidefinite or negative semidefinite. This way, we can shift the two blocks in different directions by $2d_{\max}I$ and get the cost function in a form to which Theorem 3.2 applies, but then we would still be unable to get any non-trivial bound because the term $X_{t}\bullet|C_{t}|$ could be in the order of $m$, while the analysis requires that term to be of the order of $d_{\max}$ to get the result we are aiming for. To see why, note that if $C_{t}$ is a block-diagonal matrix with a positive semidefinite block and a negative semidefinite block, then $|C_{t}|$ is just the same matrix except that the negative semidefinite block appears negated. Recall that we wanted to select an edge so that $X\bullet C_{t}$ is small: what will happen is that the PSD block gives a positive contribution, the NSD block gives a negative contribution, and $X\bullet|C_{t}|$ is the sum of the absolute values of these contributions, which can both be order of $m$.

We could work around this problem by scaling the matrix in a certain way, but this would make the analysis only work for a weighted sparsifier. This difficulty is the reason why [1] construct a weighted sparsifier even if the effective resistances of all the edges of $G$ are small, a situation in which an unweighted sparsifier is known to exist because of the Marcus-Spielman-Srivastava theorem.

We work around these difficulties by reasoning about the signless Laplacian. If $G$ is a graph with diagonal degree matrix $D_{G}$ and adjacency matrix $A_{G}$, then the signless Laplacian of $G$ is defined as the matrix $D_{G}+A_{G}$. We denote by $SL_{G}$ the signless Laplacian of a graph $G$, and by $SL_{a,b}$ the signless Laplacian of a graph containing only the single edge $(a,b)$. Equation (10) below shows that, in this case, the term $X_{t}\bullet|C_{t}|$ in the regret bound can be bounded in term of $d_{\max}$ and are never order of $m$.

Recall that, like the Laplacian, the signless Laplacian is a PSD matrix whose largest eigenvalue is at most $2d_{\max}$.

To prove Theorem 3.1, we will play the role of the adversary against the algorithm of Theorem 3.2 with the PSD cost matrix

[TABLE]

where the edge $(a_{t},b_{t})$ is chosen so that

[TABLE]

Since $X\bullet I=1$ for every density matrix, we get that, after $T$ steps, if we define $F$ to be the multiset of selected edges, $c=\frac{|E|}{|F|}=\frac{m}{T}$, and $\tilde{G}=(V,F)$, then we have

[TABLE]

[TABLE]

and so it remains to show that we can make $R_{T}\leq\epsilon d_{\max}\cdot T$ by choosing $T=O(n/\epsilon^{2})$.

Let us analyze the quantities that come up in the statement of Theorem 3.2.

Since $C_{t}$ is PSD, we have

[TABLE]

The non-trivial part of the analysis is the following bound.

Claim 1**.**

At every time step $t$ we have

[TABLE]

Proof.

Recall from Theorem 3.2 that matrices $X_{t}$ will have the same block structure as the cost matrices $C_{t}$. We can therefore write the matrix $X_{t}$ as

[TABLE]

Then

[TABLE]

and

[TABLE]

Using the triangle inequality and the fact that all the eigenvalues of $X_{t}$, and hence of $Y_{t}$, of $Z_{t}$, of $Y_{t}^{1/4}$ and $Z_{t}^{1/4}$ are at most one, we have

[TABLE]

[TABLE]

Also recall that we chose $(a_{t},b_{t})$ so that we would have

[TABLE]

which is the same as

[TABLE]

which implies

[TABLE]

[TABLE]

Now let us write

[TABLE]

where $\lambda_{i}$ are the eigenvalues of $Y_{t}$ and ${\bf y}_{i}$ are a orthonormal basis of eigenvectors of $Y_{t}$, and let us also write

[TABLE]

where ${\bf w}$ is the vector $\sqrt{m}\cdot({\bf 1}_{a_{t}}-{\bf 1}_{b_{t}})$ of length $\sqrt{2m}$. Then

[TABLE]

Finally, by Cauchy-Schwarz,

[TABLE]

In a completely analogous way we can prove that

[TABLE]

∎

To conclude the proof, take $\eta$ such that

[TABLE]

which, by the above claim, means that it can be done by choosing $\eta=\epsilon/O(\sqrt{d_{\max}m})$. Then using (10) and that $m\leq d_{max}n$ we have the regret bound

[TABLE]

When $T=O(n/\epsilon^{2})$, the above upper bound is $O(\epsilon\cdot T\cdot d_{\max})$, which means that we have constructed a graph $\tilde{G}$ with $T=O(n/\epsilon^{2})$ edges such that

[TABLE]

[TABLE]

where the second equation is equivalent to

[TABLE]

proving Theorem 3.1.

4 Probabilistic construction of additive sparsifiers

In this section, we give probabilistic algorithms for constructing additive spectral sparsifiers of hypergraphs. Specifically, we prove the following theorem which, by the reduction in Section 2.2, implies Theorem 1.3. That we can choose the normalization constant $c$ to equal $|E|/|F|$ in Theorem 1.3 is because, in the reduction, the following theorem is used for a graph where $d_{\max}$ approximately equals the average degree.

Theorem 4.1**.**

Given an $n$-vertex hypergraph $H=(V,E)$ of rank $r$ and of maximal degree $d_{\max}$ together with a parameter $\epsilon>0$, in probabilistic polynomial time we can find a subset $F\subseteq E$ of size $|F|=O\left(\frac{n}{r}\cdot\frac{1}{\epsilon^{2}}\log\frac{r}{\epsilon}\right)$ such that, if we let $c$ be a normalization constant, the following holds with probability at least $1-n^{-2}$:

[TABLE]

In Section 4.1 we then generalize the techniques for simple graphs to obtain additive spectral sparsifiers as stated in Theorem 1.2.

Our arguments are inspired by those used by Frieze and Molloy [7] and subsequently by Bilu and Linial [6]. They use the Lovász Local Lemma (LLL) [11] with an exponential number of bad events and may at first seem non-constructive. However, rather recent results give efficient probabilistic algorithms even in these applications of LLL. Theorem 3.3 in [8] will be especially helpful for us. To state it we need to introduce the following notation. We let $\mathcal{P}$ be a finite collection of mutually independent random variables $\{P_{1},P_{2},\ldots,P_{n}\}$ and let $\mathcal{A}=\{A_{1},A_{2},\ldots,A_{m}\}$ be a collection of events, each determined by some subset of $\mathcal{P}$. For any event $B$ that is determined by a subset of $\mathcal{P}$ we denote the smallest such subset by $\operatorname{vbl}(B)$. Further, for two events $B$ and $B^{\prime}$ we write $B\sim B^{\prime}$ if $\operatorname{vbl}(B)\cap\operatorname{vbl}(B^{\prime})\neq\emptyset$. In other words, $B$ and $B^{\prime}$ are neighbors in the standard dependency graph considered in LLL. Finally, we say that a subset $\mathcal{A}^{\prime}\subseteq\mathcal{A}$ is an efficiently verifiable core subset if there is a polynomial time algorithm for finding a true event in $\mathcal{A}^{\prime}$ if any. We can now state a (slightly) simplified version of Theorem 3.3 in [8] as follows:

Theorem 4.2**.**

Let $\mathcal{A}^{\prime}\subseteq\mathcal{A}$ be an efficiently verifiable core subset of $\mathcal{A}$. If there is an $\varepsilon\in[0,1)$ and an assignment of reals $x:\mathcal{A}\rightarrow(0,1)$ such that:

[TABLE]

then there exists a randomized polynomial time algorithm that outputs an assignment in which all events in $\mathcal{A}$ are false with probability at least $1-\sum_{A\in\mathcal{A}\setminus\mathcal{A}^{\prime}}x(A)$.

The following lemma says that we can roughly half the degree of vertices without incurring too much loss in the cut structure. Applying this lemma iteratively then yields a sparsifier. We use the following notation: For an edge set $X$ and disjoint vertex subsets $S$ and $T$, we let $\delta_{X}(S,T)$ denote the set of edges with one endpoint in $S$ and one in $T$; for brevity, we also write $\delta_{X}(S)$ for $\delta_{X}(S,\bar{S})$. Also recall that $e_{X}(S,T)=|\delta(S,T)|$ and $e_{X}(S)=|\delta_{X}(S)|$.

Lemma 4.3**.**

There exists a probabilistic polynomial-time algorithm that, given an $n$-vertex hypergraph $H=(V,E)$ of maximal degree $d_{\max}$ and of rank $r$, outputs a subgraph $\tilde{H}=(V,F)$ with $F\subseteq E$ such that the following holds with probability at least $1-n^{-3}$:

[TABLE]

Proof.

Throughout the proof we let $d=d_{\max}$. The proof adapts the arguments in [6] (which in turn are similar to those in [7]) to general hypergraphs. Let $G$ denote the graph obtained from $H=(V,E)$ by replacing each hyperedge $e=(v_{1},\ldots,v_{k})\in E$, by a clique with $\binom{k}{2}$ edges $(v_{i},v_{j})$, $i,j\in[k]$. We say that $G$ is associated to $H$. By construction, the degree of any vertex in $G$ is at most $d(r-1)$.

Graph $G$ will be important due to the following fact: it is enough to prove the inequality for those subsets $S\subseteq V$ that induce a connected subgraph of $G$. To see this, let $G[S]$ denote the subgraph induced by $S$. Suppose $G[S]$ is not connected and let $S_{1},\ldots,S_{k}\subseteq S$ be the vertex sets of the connected subgraphs. If the lemma holds for connected components then $\left|2\cdot e_{F}(S_{i})-e_{E}(S_{i})\right|\leq 10\sqrt{d\log(dr)}\cdot|S_{i}|$ for $i=1,\ldots,k$, and so

[TABLE]

where the first equality holds because there are no edges in $E$ (and $F\subseteq E$) between the sets $S_{1},\ldots,S_{k}$.

It is thus sufficient to prove the inequality for those sets $S$ that induce a connected subgraph $G[S]$. Suppose we select $F$ by including each edge $e\in E$ with probability $1/2$ independently of other edges. That is, in the notation of Theorem 4.2, we have that $\mathcal{P}$ consists of $|E|$ mutually independent variables $\{P_{e}\}_{e\in E}$, where $P_{e}$ indicates whether $e\in F$ and $\Pr[P_{e}]=1/2$. Now for each $S$ such that $G[S]$ is connected, let $A_{S}$ be the “bad” event that $\left|2\cdot e_{F}(S)-e_{E}(S)\right|>10\sqrt{d\log(dr)}\cdot|S|$. Note that $e_{F}(S)$ is the sum of at most $d|S|$ independent variables, attaining values [math] and $1$, and that the expected value of $e_{F}(S)$ equals $e_{E}(S)/2$. Thus by the Chernoff inequality we get

[TABLE]

To apply Theorem 4.2, we analyze the dependency graph on the events: there is an edge between $A_{S}$ and $A_{S^{\prime}}$ if $\operatorname{vbl}(A_{S})\cap\operatorname{vbl}(A_{S^{\prime}})\neq\emptyset\Leftrightarrow\delta_{E}(S)\cap\delta_{E}(S^{\prime})\neq\emptyset$. Consider now a fixed event $A_{S}$ and let $k=|S|$. We bound the number of neighbors, $A_{S^{\prime}}$, of $A_{S}$ with $|S^{\prime}|=\ell$. Since we are interested in only subsets $S^{\prime}$ such that $G[S^{\prime}]$ is connected, this is bounded by the number of distinct subtrees on $\ell$ vertices in the associated graph $G$, with a root in one of the endpoints of an edge in $\delta(S)$ . As $G$ has degree at most $d(r-1)$, there are at most $|S|+d(r-1)|S|=drk$ choices of the root. The number of such trees is known to be at most (see e.g. [7])

[TABLE]

where we used that ${dr(\ell-1)\choose\ell-1}\leq(edr)^{\ell-1}$.

Now to verify condition (13) of Theorem 4.2, we set $x(A_{S})=(dr)^{-3|S|}$ for every bad event $A_{S}$. So if we consider an event $A_{S}$ with $k=|S|$, then

[TABLE]

where we used that $d$ is a sufficiently large constant, which is without loss of generality since if $d\leq 10\sqrt{d\log(dr)}$ then the lemma becomes trivial. In other words, (13) is satisfied with $\epsilon$ set to $1/2$.

It remains to define an efficiently verifiable core subset $\mathcal{A}^{\prime}\subseteq\mathcal{A}$ such that $1-\sum_{A\in\mathcal{A}\setminus\mathcal{A}^{\prime}}x(A)\geq 1-n^{-3}$. We let

[TABLE]

By the same arguments as in (14), there is at most $n\cdot{dr(\ell-1)\choose\ell-1}\leq n(edr)^{\ell-1}$ many events with $|S|=\ell$ (corresponding to connected components in $G$). Therefore, the following properties hold:

$\mathcal{A}^{\prime}$ is efficiently verifiable since it contains $n\cdot\sum_{\ell=1}^{s}(edr)^{\ell-1}=O(n\cdot(edr)^{s})=O(n^{3})$ many events that can be efficiently enumerated by first selecting a vertex $r$ among $n$ choices and the considering all possible trees rooted at $r$ with $\ell\leq s$ vertices. 2. 2.

We have

[TABLE]

where for the first inequality we again used that $d$ is a sufficiently large constant.

We have verified Condition (13) of Theorem 4.2 and we have defined an efficiently verifiable core subset $\mathcal{A}^{\prime}$ such that $\sum_{A_{S}\in\mathcal{A}\setminus\mathcal{A}^{\prime}}x(A_{S})\leq n^{-3}$ and so the lemma follows. ∎

Applying the above lemma iteratively will give us additive cut sparsifiers of constant degree. In particular, the condition in the following lemma will imply that the degree of each vertex in $\tilde{H}$ is at most $O(d_{\max}/2^{k})$ and $k$ can be chosen so that the degree is at most $O(\frac{1}{\varepsilon^{2}}\log(r/\varepsilon))$. The following lemma therefore implies Theorem 4.1.

Lemma 4.4**.**

There is an absolute constant $c$ such that the following holds. There is a probabilistic polynomial-time algorithm that given as input an $n$-vertex hypergraph $H=(V,E)$ of maximal degree $d_{\max}$ and of rank $r$, $\varepsilon>0$, and any $k\in\mathbb{N}$ such that $d_{\max}2^{-k}\geq c\frac{1}{\varepsilon^{2}}\log(r/\varepsilon)$, outputs a subgraph $\tilde{H}=(V,F)$ such that the following holds with probability at least $1-n^{-2}$:

[TABLE]

Proof.

Starting with $H$ we apply Lemma 4.3 $k$ times to obtain $\tilde{H}$. Let $F_{i}$ denote the edge set and let $d_{i}$ denote the maximum degree after round $i$. So $F_{0}=E$ and $d_{0}=d_{\max}$. By the guarantees of Lemma 4.3, we have that with probability $1-n^{-3}$

[TABLE]

and

[TABLE]

As we apply Lemma 4.3 $k$ times with $k\leq\log(n)$, the union bound implies that the above inequalities are true for all invocations of that lemma with probability at least $1-k\cdot n^{-3}\geq 1-n^{-2}$. From now on we assume that the above inequalities hold and show that the conclusion of the statement is always true in that case. Specifically, we now prove by induction on $k$ that

[TABLE]

The claim holds trivially for $k=0$. Assume it holds for all $i<k$, which in particular implies $2^{i}d_{i}\leq 2d_{0}$ for all $i<k$. By the triangle inequality and (15),

[TABLE]

As the terms increase geometrically in $i$, this sum is $O(2^{k}\sqrt{(d_{0}/2^{k})\log((d_{0}/2^{k})r)}$ which is $\varepsilon d_{0}$ by our assumption on $k$ and selection of $c$.

Finally, we note that $\left|2^{k}\cdot e_{F_{k}}(S)-e_{F_{0}}(S)\right|\leq\varepsilon d_{0}\cdot|S|$ follows by the same calculations (using (16) instead of (15)). ∎

4.1 Additive spectral graph sparsifiers

In this section we describe how the proof in the previous section generalizes to spectral additive graph sparsifiers.

Theorem 4.5**.**

Given an $n$-vertex graph $G=(V,E)$ and a parameter $\epsilon>0$, in probabilistic polynomial time we can find a subset $F\subseteq E$ of size $|F|=n\cdot O((\log(1/\epsilon)^{3}/\epsilon^{2})$ such that, if we let $L_{G}=D_{G}-A_{G}$ be the Laplacian of $G$, $L_{\tilde{G}}=D_{\tilde{G}}-A_{\tilde{G}}$ be the Laplacian of the graph $\tilde{G}=(V,F)$, and $c$ a normalization constant, we have

[TABLE]

Similar to before, this implies Theorem 1.2 by the reductions in Section 2.2.

To prove Theorem 4.5, we need the following modification of Lemma 4.3 in the case of simple graphs.

Lemma 4.6**.**

There exists a probabilistic polynomial-time algorithm that, given an $n$-vertex graph $G=(V,E)$ of maximal degree $d$, outputs a subgraph $\tilde{G}=(V,F)$ with $F\subseteq E$ such that the following properties hold with probability at least $1-n^{-3}$:

For every disjoint $S,T\subseteq V$ we have $\left|2\cdot e_{F}(S,T)-e_{E}(S,T)\right|\leq 10\sqrt{d\log d}\cdot\sqrt{|S||T|}$. 2. 2.

For every vertex $v\in V$ we have $\left|2\cdot e_{F}(v)-e_{E}(v)\right|\leq 10\sqrt{d\log d}$.

The above lemma is similar to Lemma 3.2 in [6] with the exception that here we also need the degree constraints (the second condition). Similar to Lemma 4.7 we obtain the following by applying Lemma 4.6 iteratively.

Lemma 4.7**.**

There is an absolute constant $c$ such that the following holds. There is a probabilistic polynomial-time algorithm that on input an $n$-vertex graph $G=(V,E)$ of maximum degree $d$, $\varepsilon>0$, and any $k\in\mathbb{N}$ such that $d2^{-k}\geq c\frac{1}{\varepsilon^{2}}\log(1/\varepsilon)$, outputs a subgraph $\tilde{G}=(V,F)$ such that the following properties hold with probability at least $1-n^{-2}$:

For every disjoint $S,T\subseteq V$ we have $\left|2^{k}\cdot e_{F}(S,T)-e_{E}(S,T)\right|\leq\varepsilon d\cdot\sqrt{|S||T|}$. 2. 2.

For every vertex $v\in V$ we have $\left|2^{k}\cdot e_{F}(v)-e_{E}(v)\right|\leq\varepsilon d$.

The proofs of Lemma 4.6 and Lemma 4.7 are very similar to the proofs of Lemma 4.3 and Lemma 4.4, respectively. We have therefore deferred them to Appendix A. We now explain how Lemma 4.7 implies an additive spectral sparsifier for graphs via the following result of Bilu and Linial [6]:

Lemma 4.8** (Lemma 3.3 in [6]).**

Let $A$ be an $n\times n$ real symmetric matrix such that the $\ell_{1}$ norm of each row in $A$ is at most $\ell$, and all diagonal entries of $A$ are, in absolute value, $O(\alpha\log(\ell/\alpha)+1))$. Assume that for any two vectors, ${\bf u},{\bf v}\in\{0,1\}^{n}$, with $supp({\bf u})\cap supp({\bf v})=\emptyset$:

[TABLE]

Then the spectral radius of $A$ is $O(\alpha(\log(\ell/\alpha)+1))$.

Here $supp({\bf u})=\{i:{\bf u}_{i}\neq 0\}$ denotes the support of a vector ${\bf u}$. Now let $G$ and $\tilde{G}$ be the input and output graph of Lemma 4.7. We set $A=2^{k}L_{\tilde{G}}-L_{G}$. Since the Laplacian of a graph is a symmetric real matrix we have that $A$ is a symmetric $n\times n$ real matix where $n$ is the number of vertices in $G$ and $\tilde{G}$. We now verify that $A$ satisfies the assumptions of the above lemma assuming that the algorithm of Lemma 4.7 was successful (which happens with probability at least $1-n^{-2}$).

• •

The $\ell_{1}$ norm of a row in $A$ is at most the $\ell_{1}$ of that row in $2^{k}L_{\tilde{G}}$ plus the $\ell_{1}$ norm of that row in $L_{G}$. This can be upper bounded as follows. The $\ell_{1}$ norm of a row of a Laplacian matrix corresponding to a vertex $v$ equals twice the (weighted) degree of $v$. As any vertex in $G$ has degree at most $d$, it follows that the $\ell_{1}$ norm of any row in $L_{G}$ is at most $2d$. For a row in $2^{k}L_{\tilde{G}}$ we use Property 2 of Lemma 4.7 to bound the $\ell_{1}$ norm by $2(e_{E}(v)+10\sqrt{d\log d})\leq 2(d+10\sqrt{d\log d})$. We therefore have that $\ell_{1}$ norm of any row in $A$ is bounded by

[TABLE]

• •

For the other two conditions, set $\alpha=\epsilon d$ where $\epsilon$ is selected as in Lemma 4.7. Then we have that the absolute value of any diagonal entry in $A$ corresponding to a vertex $v$ equals

[TABLE]

where the inequality is implied by Property 2 of Lemma 4.7. Similarly, consider any vectors ${\bf u},{\bf v}\in\{0,1\}^{n}$ with $supp({\bf u})\cap supp({\bf v})=\emptyset$. Let $S=supp({\bf u})$ and $T=supp({\bf v})$. Then

[TABLE]

where the last inequality is implied by Property 1 of Lemma 4.7. The second equality is by the identity

[TABLE]

(and similar for $2^{k}L_{\tilde{G}}$).

We thus have that the assumptions of Lemma 4.8 are satisfied with $\ell=O(d)$ and $\alpha=\varepsilon d$. It follows that $A$ has a spectral radius of $O(\varepsilon\log(1/\varepsilon)d)$. Or equivalently:

[TABLE]

for an absolute constant $c^{\prime}$. To summarize, we obtain the following lemma which in turn implies Theorem 4.5 (by selecting $k$ as large as possible):

Lemma 4.9**.**

There are absolute constants $c$ and $c^{\prime}$ such that the following holds. There is a probabilistic polynomial-time algorithm that on input an $n$-vertex graph $G=(V,E)$ of maximum degree $d$, $\varepsilon>0$, and any $k\in\mathbb{N}$ such that $d2^{-k}\geq c\frac{1}{\varepsilon^{2}}\log(1/\varepsilon)$, outputs a subgraph $\tilde{G}=(V,F)$ such that the following holds with probability at least $1-n^{-2}$:

[TABLE]

5 Spectral Hypergraph Sparsification

Let $H=(V,E)$ be a weighted hypergraph on $n$ vertices, with weights $w_{e}\geq 0$ on hyperedges $e\in E$. Let $r=\max_{e\in E}|e|$ be the maximum size of hyperedges in $H$, i.e., the rank of the hypergraph.

For a hyperedge $e$, the hypergraph Laplacian operator $Q_{e}:{\mathbb{R}}^{n}\rightarrow{\mathbb{R}}$, acts on a vector $x\in{\mathbb{R}}^{n}$ as

[TABLE]

where $L_{ab}$ is the standard graph Laplacian for an (unweighted) edge $ab$.

Definition 1**.**

(Hypergraph Laplacian) Given a weighted hypergraph $H$, the hypergraph Laplacian operator $Q_{H}:{\mathbb{R}}^{n}\rightarrow{\mathbb{R}}$ for $H$ is defined as

[TABLE]

Definition 2**.**

(Multiplicative hypergraph spectral sparsifier.) A weighted hypergraph $\tilde{H}=(V,F)$ is a $(1+\varepsilon)$-multiplicative spectral sparsifier of $H$ if

[TABLE]

We show the following result, which generalizes the result of Spielman and Srivastava [14] from graphs to hypergraphs.

Theorem 5.1**.**

For any hypergraph $H$ of rank $r$, and $\varepsilon>0$, there is a $(1+\varepsilon)$-multiplicative spectral sparsifier $\tilde{H}$ of $H$ with $O(\frac{1}{\varepsilon^{2}}r^{3}n\log n)$ edges. Moreover, there is an efficient randomized algorithm that computes $\tilde{H}$ with probability $1-n^{-\Omega(1)}$, and runs in time $\tilde{O}_{r,\varepsilon}(n)$.

Unlike in the graph case, where it can be checked if $F$ satisfies (18) by an eigenvalue computation, we do not know of any efficient way to check Condition (18) for hypergraphs.

The following simple lemma shows that to prove Theorem 5.1, it suffices to consider the case where all hyperedges have size between $r/2$ and $r$. We will make this assumption henceforth.

Lemma 5.2**.**

If Theorem 5.1 holds for hypergraphs where each edge has size between $r/2$ and $r$, then it holds for all rank $r$ hypergraphs.

Proof.

For $i=1,\ldots,\log r$, let $H_{i}$ be $H$ restricted to edges of size $(2^{i-1},2^{i}]$. For each $i$, we apply the claimed algorithm to $H_{i}$ to find a $(1+\epsilon_{i})$-sparsifier $\tilde{H}_{i}$ of $H_{i}$ with $\epsilon_{i}=\varepsilon 2^{(i-\log r)/2}$ and return $\tilde{H}=\cup_{i}\tilde{H_{i}}$.

As $\tilde{H}_{i}$ has $O(\frac{1}{\varepsilon_{i}^{2}}2^{3i}n\log n)=O(\frac{1}{\varepsilon^{2}}2^{\log r+2i}n\log n)$ hyperedges, summing over all $i$ from $1$ to $\log r$ gives that $H$ has $O(\frac{1}{\varepsilon^{2}}r^{3}n\log n)$ hyperedges. Moreover, for any $x\in{\mathbb{R}}^{n}$, $\tilde{H}$ satisfies (18) as

[TABLE]

∎

5.1 Algorithm

The algorithm is a natural generalization of the sampling by effective resistances algorithm for graphs [14].

Definition 3**.**

(Associated graph.) Let $G$ denote the multi-graph obtained by replacing each hyperedge $e=(v_{1},\ldots,v_{k})\in H$, by a clique with $\binom{k}{2}$ edges $(v_{i},v_{j})$, $i,j\in[k]$, each with the same weight as that of $e$. We call $G$ the associated graph of $H$.

To avoid confusion, we will use $(a,b)$ to denote the edges in $G$ and $e$ for the hyperedges in $H$.

Algorithm.

Given the hypergraph $H$, let $G$ be its associated graph, and let $L_{G}=\sum_{(ab)\in E}L_{ab}$ be the (graph) Laplacian of $G$. Let $Y_{ab}=L_{G}^{-1/2}L_{ab}L_{G}^{-1/2}$, where $L_{G}^{-1}$ is the pseudoinverse of $L_{G}$. Then $r_{ab}:=\|Y_{ab}\|$ is the effective resistance of the edge $ab$. For a hypergraph $e\in E(H)$, define

[TABLE]

Let $L=c\varepsilon^{2}/(r^{4}\log n)$, where $c$ is a fixed constant that can be computed explicitly from the analysis described later. For each hyperedge $e$, set

[TABLE]

$\tilde{H}$ is obtained by sampling each $e\in H$ independently with probability $p_{e}$ and scaling its weight by $1/p_{e}$.

5.2 Analysis

Our goal in the next few sections is to prove Theorem 5.1. We first show that $\tilde{H}$ has $O((r^{3}n\log n)/\varepsilon^{2})$ edges with high probability, and then focus on showing that (18) holds with probability $1-n^{\Omega(1)}$.

Bounding the number of edges.

The expected number of edges in $\tilde{H}$ is $\sum_{e}p_{e}$, which is at most $(\sum_{e}r_{e})/L$. So it suffices to bound,

[TABLE]

The effective resistances in a graph satisfy the metric property, $r_{ab}\leq r_{ac}+r_{cb}$ for all $a,b,c$, and so for any $e\in E(H)$ with $k=|e|$, and any $a,b\in e$, summing over all $c\in e$ gives

[TABLE]

As $k\geq r/2$ by our assumption from Lemma 5.2, this gives that

[TABLE]

Without loss of generality we can assume that $G$ is connected, in which case $L_{G}$ has rank exactly $n-1$ and $L_{G}\mathbf{1}=0$. This gives that $\sum_{(ab)\in E(G)}Y_{ab}=I_{n-1}$, which upon taking traces on both sides, and using that $Y_{ab}$ is rank $1$, gives $\sum_{(ab)\in E(G)}r_{ab}=n-1.$

So the expected number of edges is $O(n/rL)=O((nr^{3}\log n)/\varepsilon^{2})$, and as the hyperedges are sampled independently, by standard tail bounds the number of edges is tightly concentrated around the mean.

Proving condition (18).

We now focus on showing that (18) holds. It is useful to first consider the analysis of Spielman and Srivastava [14] for the graph case.

The graph case.

In the graph setting, (18) becomes

[TABLE]

where $L_{\tilde{G}}=\sum_{(ab)\in F}(1/p_{ab})L_{ab}$ is the Laplacian of $\tilde{G}$.

Setting $z=L_{G}^{1/2}x$, and $Y_{ab}=L_{G}^{-1/2}L_{ab}L_{G}^{-1/2}$, this is equivalent to showing that

[TABLE]

where $X_{ab}$ is the random matrix which is $Y_{ab}/p_{ab}$ with probability $p_{ab}$ and is the all-[math] matrix otherwise. So $\operatorname{\mathbb{E}}[X_{ab}]=Y_{ab}$. As $\sum_{(ab)\in G}Y_{ab}=I$ (on the range of $L_{G}$), this reduces to show that $z^{T}(\sum_{(ab)\in G}X_{ab}-I)z\leq\varepsilon\|z\|^{2}$ or equivalently,

[TABLE]

This can be done using standard matrix concentration bounds for the spectral norm such as the following.

Theorem 5.3**.**

*(Matrix Bernstein inequality, [15].)

Let $X_{1},\ldots,X_{m}$ be independent, symmetric $d\times d$ random matrices, and $S=\sum_{i}X_{i}$, $L=\max_{i}\|X_{i}\|$. Then*

[TABLE]

In particular, this gives the following useful corollary.

Corollary 5.4**.**

If $A_{1},\ldots,A_{m}$ are PSD with $\sum_{i}A_{i}\preceq I$, and $X_{i}=A_{i}/p_{i}$ with probability $p_{i}$ and [math] otherwise, then for any $\varepsilon\leq 1$,

[TABLE]

where $L=\max_{i}\|A_{i}\|/p_{i}$.

Applying Corollary 5.4 with $A_{i}=Y_{ab}$ and $p_{i}=p_{ab}$, we have $L=\max_{ab}\|Y_{ab}\|/p_{ab}=\max_{ab}r_{ab}/p_{ab}=O(\varepsilon^{2}/\log n),$ which gives that (20) holds with probability at least $1-n^{-\Omega(1)}$ as desired.

The hypergraph case.

We first reduce the condition (18) for hypergraphs to a simpler form. Let $G$ be the graph associated to $H$ and $L_{G}$ be its Laplacian. We have following simple relation.

Lemma 5.5**.**

For a $k$-edge $e$, let $L_{e}=\sum_{a,b\in e}L_{ab}$. Then, for all $x\in{\mathbb{R}}^{n}$

[TABLE]

If the hyperedges in $H$ have size in $(r/2,r]$, then for all $x\in{\mathbb{R}}^{n}$

[TABLE]

Proof.

Suppose that $x_{1}\leq\ldots\leq x_{k}$. Then $Q_{e}(x)=(x_{k}-x_{1})^{2}$, while $e$ contributes $\sum_{i,j\in[k]}(x_{i}-x_{j})^{2}$ to $x^{T}L_{G}x$. So the lower bound in the first inequality follows directly.

For the upper bound, we observe that $(x_{k}-x_{1})^{2}\leq 2(x_{k}-x_{j})^{2}+2(x_{j}-x_{1})^{2}$ for each $j=2,\ldots,k-1$. Summing these gives $(k-2)(x_{k}-x_{1})^{2}\leq 2\sum_{j=2}^{k-1}((x_{k}-x_{j})^{2}+(x_{j}-x_{1})^{2})$. Adding $2(x_{k}-x_{1})^{2}$ to both sides, and noting that the resulting right side is at most $2x^{T}L_{e}x$, the upper bound follows.

Summing up over all $e\in E(H)$, and using $r/2<k\leq r$ gives the second set of inequalities. ∎

By Lemma 5.5, to show (18) it suffices to show that for $x\in{\mathbb{R}}^{n}$,

[TABLE]

As before, setting $z=L_{G}^{1/2}x$ and $Y_{ab}=L_{G}^{-1/2}L_{ab}L_{G}^{-1/2}$ gives

[TABLE]

Let us define

[TABLE]

and let $X_{e}$ be the random variable that is $1/p_{e}$ with probability $p_{e}$ and [math] otherwise. Then (21) is equivalent to

[TABLE]

As $W_{e}(z)$ scales as $\|z\|^{2}$, it suffices to show that

[TABLE]

where $B_{2}$ is the unit $\ell_{2}$-ball in the subspace restricted to the image of $L_{G}$,

However, unlike in the graph case, it is not immediately clear how to show concentration to prove (22). In particular, as the operator $W_{e}(z)$ involves the $\max$ term, the left hand side does not correspond to any standard linear-algebraic quantity like the spectral norm, for which we can use matrix concentration bounds.

A natural idea might be to replace $W_{e}(z)$ by the larger term $\sum_{(a,b)\in e}z^{T}Y_{ab}z$, and reduce the problem to the graph case, for which we can use matrix Bernstein inequality. But this does not work as the multiplier $(X_{e}-1)$ in (22) can be negative (so $|\sum_{e\in H}(X_{e}-1)W_{e}(z)|$ could be arbitrarily large even though $|\sum_{e\in H}(X_{e}-1)\sum_{a,b\in e}z^{T}Y_{ab}z|$ is [math]).

So our approach will be to directly consider the inequality (18) for each $z$ in the unit $\ell_{2}$-ball, and bound the probability of violation for any $z$ by applying a union bound over all such points $z$ by a careful net argument. More precisely, we view the left hand side of $\eqref{eq:hyp-spars-2}$ as a random process indexed by $z\in B_{2}$, and use generic chaining arguments to bound the supremum of this process.

Summarizing, let $W_{H}(z)=\sum_{e\in E}W_{e}(z)$, and $W_{\tilde{H}}(z)=\sum_{e\in E}X_{e}W_{e}(z)$ be the corresponding operator for $\tilde{H}$. Proving Theorem 5.1 reduces to the following.

Theorem 5.6**.**

With probability $1-n^{-\Omega(1)}$, it holds that

[TABLE]

This will be accomplished in the next few sections.

5.3 Supremum of random processes

We first give some background on the theory of supremum of random processes and mention the results we need. For more details, we refer the reader to Chapters 7 and 8 of the excellent recent text [16].

Definition 4**.**

(Random process) A random process is a collection of random variables $(X_{t})_{t\in T}$ on the same probability space, which are indexed by the elements $t$ of some set $T$.

The random variables $X_{t}-X_{s}$ for $s,t\in T$ are the increments of the random process. A random process is called mean-zero if all $X_{t}$ have mean-zero. We will only consider mean-zero processes in this paper.

Definition 5**.**

(Gaussian process) A random process $(X_{t})_{t\in T}$ is called a Gaussian process, if $X_{t}$ are jointly Gaussian, i.e. if every finite linear combination of the $X_{t}$ is Gaussian.

Any Gaussian process can be written in a canonical way as $X_{t}=\langle g,t\rangle$, where $t\in{\mathbb{R}}^{n}$ and $g\sim N(0,I_{n})$ is the standard normal vector. This gives that for any $s,t\in T$, the increments of a Gaussian process satisfy,

[TABLE]

where $\|t-s\|_{2}$ denotes the Euclidean distance between $t$ and $s$.

As a (mean-zero) Gaussian process is completely determined by its covariance, the supremum $\operatorname{\mathbb{E}}\sup_{t\in T}X_{t}$ of a gaussian process is completely determined by the geometry of the metric space $(T,d)$. In particular, we have the following celebrated result.

Theorem 5.7**.**

(Talagrand’s majorizing measures theorem.)) Let $(X_{t})_{t\in T}$ be a mean-zero Gaussian process on a set $T$, with the canonical metric on $T$, $d(s,t)=\|t-s\|_{2}$. Then for some absolute constants $c,C$

[TABLE]

where $\gamma_{2}(T,d)=\inf_{(T_{k})}\sup_{t\in T}\sum_{k=0}^{\infty}2^{k/2}d(t,T_{k}),$ and where the infimum is over all sets $T_{k}\subset T$, satisfying $|T_{k}|\leq 2^{2^{k}}$ for all $k$.

We now consider sub-gaussian processes (see section 8.1 in [16] for details).

Definition 6**.**

(Sub-gaussian increments.) Consider a random process $(X_{t})_{t\in T}$ on a metric space $(T,d)$. We say that the process has sub-gaussian increments if there exists some $K\geq 0$, such that

[TABLE]

Here $\|\cdot\|_{\psi_{2}}$ is the sub-gaussian norm for real-valued random variable $X$, defined as

[TABLE]

We need the following two basic facts about the $\psi_{2}$-norm (section 2.6 in [16]).

Fact 5.8**.**

For any random variable $X$, $\|X\|_{\psi_{2}}\leq c\|X\|_{\infty}$ (with $c=1/\sqrt{\ln 2}$).

Fact 5.9**.**

For $X_{1},\ldots,X_{n}$ independent $\|\sum_{i=1}^{n}X_{i}\|_{\psi_{2}}^{2}\leq c\sum_{i=1}^{n}\|X_{i}\|^{2}_{\psi_{2}},$ where $c$ is an absolute constant.

The following result follows directly from Theorem 5.7 (see section 8.6 in [16] for details).

Theorem 5.10**.**

(Talagrand’s comparison inequality.) Let $(X_{t})_{t\in T}$ be a mean-zero random process on a set $T$, and let $(Y_{t})_{t\in T}$ be a Gaussian process with the canonical metric $d(s,t)=(\operatorname{\mathbb{E}}[(Y_{s}-Y_{t})^{2}])^{1/2}=\|Y_{s}-Y_{t}\|_{2}$. Assume that for all $s,t\in T$, we have

[TABLE]

Then, for some absolute constant $C$,

[TABLE]

More generally, for every $u\geq 0$,

[TABLE]

where $\mathrm{diam}(T)$ is the diameter of $T$ with respect to the metric $d$.

In other words, if we can find a Gaussian process $Y_{t}$ such that its Gaussian increments upper bound the corresponding sub-gaussian increments of $X_{t}$, then we can bound the supremum of $X_{t}$ by that of $Y_{t}$.

5.4 Random process for hypergraph sparsification

We now consider the relevant random processes arising in our setting of hypergraph sparsification.

Gaussian Process on the associated graph.

Let $G$ be the associated graph of $H$, and consider the random matrix $U=\sum_{(ab)\in E(G)}g_{ab}Y_{ab}$, where $g_{ab}$ are independent $N(0,1)$.

For $z\in{\mathbb{R}}^{n}$, consider the Gassian process $U_{z}=z^{T}Uz=\sum_{(ab)\in E(G)}(z^{T}Y_{ab}z)g_{ab}$. As $\|U\|=\max_{z\in B_{2}}z^{T}Uz$, it follows that $\|U\|=\sup_{z\in T}U_{z}$ with $T=B_{2}$. As,

[TABLE]

the canonical metric induces the distance

[TABLE]

Hypergraph sampling process.

Let us now consider the random process corresponding to (22). We consider the case when $p_{e}=1/2$ (the theory of sub-gaussian does not work well for $p_{e}\ll 1$) (in section 5.5 we will show that the case of general $p_{e}$ reduces to that of $p_{e}=1/2$. For $p_{e}=1/2$, $(X_{e}-1)$ takes value $-1$ or $1$ with probability $1/2$ each. So we define

[TABLE]

where $\varepsilon_{e}$ are independent Rademacher random variables.

The following key Lemma will allow us to bound the (complicated) sub-gaussian process $V_{z}$ by the simpler Gaussian process $U_{z}$.

Lemma 5.11**.**

There is an absolute constant $c$, such that for any $z,z^{\prime}\in B_{2}$,

[TABLE]

Before proving this lemma, we need the following simple fact.

Lemma 5.12**.**

For any numbers $c_{1},\ldots,c_{s}$ and $d_{1},\ldots,d_{s}$,

[TABLE]

Proof.

Let $c_{a}=\max_{i}c_{i}$ and $d_{b}=\max_{i}d_{i}$. If $c_{a}\geq d_{b}$, then

[TABLE]

The other case when $c_{a}\leq d_{b}$ is completely analogous. ∎

We now prove Lemma 5.11.

Proof.

(Lemma 5.11). Fix $z,z^{\prime}\in B_{2}$. For a hyperedge $e$, let $a(e),b(e)\in e$ be the indices that maximize $z^{T}Y_{a(e)b(e)}z$, and $a^{\prime}(e),b^{\prime}(e)\in e$ be those that maximize $z^{\prime T}Y_{a^{\prime}(e)b^{\prime}(e)}z^{\prime}$. Then,

[TABLE]

By Facts 5.8 and 5.9, there is an absolute constant $c$ such that,

[TABLE]

On the other hand, by (24) we have that

[TABLE]

Even though $Y_{a(e)b(e)}$ could be different from $Y_{a^{\prime}(e)b^{\prime}(e)}$, we can use Lemma 5.12 to show that the right hand side of (25) is upper bounded by the right side of (26).

Fix a hyperedge $e\in H$ and let $k=|e|$. Applying Lemma 5.12 to the $s=k^{2}$ pairs $a,b\in[k]$ with $c_{ab}=z^{T}Y_{ab}z$ and $d_{ab}=z^{\prime T}Y_{ab}z^{\prime}$, we get

[TABLE]

Summing over the hyperedges $e\in E(H)$, using (25) and (26), and noting that

[TABLE]

gives the result. ∎

Remark: It might seem that the inequality (27) can be tightened by a factor $O(1/r)$, by using that for any $z_{1}\leq\ldots\leq z_{r}$, we have that $(z_{r}-z_{1})^{2}\leq\frac{2}{r}\sum_{ij}(z_{i}-z_{j})^{2}$ (we used similar ideas in Lemma 5.5). However, this following example shows that this is not possible.

Suppose, $Y_{ab}=L_{ab}$, i.e. $z^{T}Y_{ab}z=(z_{a}-z_{b})^{2}$. Consider $z=(z_{1},\ldots,z_{r})=(-1,0,\ldots,0,M)$ and $z^{\prime}=(0,0,\ldots,0,M)$. The term on the left side of (27) is

[TABLE]

On the other hand, the terms on the right side of (27) correspond to

[TABLE]

However, it is easily verified that for each of the $r^{2}-1$ pairs $(a,b)$ except $(a,b)=(1,r)$, $(z_{a}-z_{b})^{2}-(z^{\prime}_{a}-z^{\prime}_{b})^{2}\leq 1$. Making $M$ arbitrarily large, this shows that we really need the full contribution of each of the $r^{2}$ terms on the right side of (27), and we cannot improve the inequality.

Bounds on the process $V_{z}$.

Lemma 5.11 and Theorem 5.10 will let us bound the supremum of $V_{z}$ by that of $U_{z}$. We can directly bound the latter using the following variant of the matrix Bernstein inequality.

Theorem 5.13**.**

*([15], Theorem 4.1.1.)

If $A_{1},\ldots,A_{m}$ are symmetric $d\times d$ matrices, and $g_{i}$ are independent $N(0,1)$ random variables, then for $Y=\sum_{i}g_{i}A_{i}$*

[TABLE]

In particular, Theorem 5.13 has the following corollary.

Corollary 5.14**.**

If $A_{i}$ are PSD, and satisfy $\|A_{i}\|\leq\delta$ and $\sum_{i}A_{i}\preceq I$, then $\|\sum_{i}A_{i}^{2}\|\leq\delta$ and so for $c\geq 2$

[TABLE]

We now show that a similar tail bound holds for $\sup_{z}V_{z}$.

Theorem 5.15**.**

Let $S\subset E(H)$ be a subset of hyperedges with $r_{e}\leq\delta$ for all $e\in S$. For independent Rademacher $\varepsilon_{e}$, and $z\in{\mathbb{R}}^{n}$, let

[TABLE]

Then $\operatorname{\mathbb{E}}\sup_{z\in B_{2}}V_{z}=O(\sqrt{\delta\log n})$, and for all $u\geq 0$

[TABLE]

Proof.

Let $E(G[S])=\{(ab):a,b\in e,e\in S\}$ be the multi-set of edges in the associated graph $G[S]$ for $H$ restricted to $S$. Consider the process $U_{z}=\sum_{(ab)\in E(G[S])}g_{ab}z^{T}Y_{ab}z$ where $g_{ab}$ are independent $N(0,1)$. Then

[TABLE]

As $r_{e}=\max_{a,b\in e}\|Y_{ab}\|$, we have $\|Y_{ab}\|\leq\delta$ for all $(ab)\in E(G[S])$. Moreover as

[TABLE]

Corollary 5.14 gives that $\|U\|=O(\sqrt{\delta\log n})$. By Lemma 5.11 and the Talagrand comparison inequality Theorem 5.10, we have that for some constant $C$,

[TABLE]

To compute the tail bound on $\sup_{z}V_{z}$, we need to compute the diameter $\mathrm{diam}(T)$ with respect $d_{u}$. By the definiton of $d_{u}(z,z^{\prime})$,

[TABLE]

Using $(c-d)^{2}\leq 2c^{2}+2d^{2}$ for any $c,d\in{\mathbb{R}}$,

[TABLE]

The claimed tail bound on $\sup_{z}V_{z}$ now follows from Theorem 5.10. ∎

5.5 Putting it all together

We now prove Theorem 5.6. Given $H$, we compute $G$ and $p_{e}$ as described earlier. By rounding $p_{e}$ up to nearest integer powers of $2$, we can assume that for each $e\in E(H)$, $p_{e}=2^{-j}$ for some $j\in\{0,\ldots,\ell\}$. This ensures $p_{e}\geq\min(1,r_{e}/L)$, while at most doubling the expected size of $\tilde{H}$. Let $C_{j}=\{e\in E(H):p_{e}=2^{-j}\}$. As $p_{e}=1$ for hyperedges $e$ with $r_{e}\geq L$, the sampling error in $\tilde{H}$ is only due to edges with $r_{e}<L$, and so in the analysis of the sampling error below we will assume that $r_{e}<L$ for all $e\in E(H)$.

We view the process of sampling $\tilde{H}$ in the following iterative way. Let $H_{0}=H$, and for $i=1,\ldots,\ell$, $H_{i}$ is obtained from $H_{i-1}$ by picking each hyperedge $e$ of classes $C_{j}$ for $j\in\{\ell-i+1,\ell\}$, independently with probability $1/2$, and doubling the weight of $e$ if it is picked. Or equivalently, for $i=1,\ldots,\ell$, $H_{i}$ is obtained by picking each edge $e\in C_{j}$ in $H$ independently with probability $\min(1,2^{\ell-j-i})$ and scaling its weight by $\max(1,2^{j+i-\ell})$. So $H_{\ell}=\tilde{H}$, and an edge $e$ in $C_{j}$ survives independently in $H_{\ell}$ with probability $p_{e}=2^{-j}$.

Proof.

(Theorem 5.6.) By the discussion above, for $i=0,\ldots,\ell$,

[TABLE]

and note that $W_{H_{0}}(z)=W_{H}(z)$ and $W_{\tilde{H}}(z)=W_{H_{\ell}}(z)$.

For any $z\in B_{2}$, by triangle inequality

[TABLE]

Taking supremum over all $z$, and taking the $\sup$ inside the summation,

[TABLE]

As $H_{i}$ is obtained by $H_{i-1}$ by sampling each edge of class $j\in[\ell-i+1,\ell]$ with probability $1/2$ and doubling its weight, we have

[TABLE]

For $j\in\{\ell-i+1,\ell\}$ and any $e\in C_{j}$, $W_{e}(z)=\max_{a,b\in e}z^{T}Y_{ab}z$ with $\|Y_{ab}\|\leq r_{e}\leq 2^{-j}L$ for all $a,b\in e$. So $\|2^{i+j-\ell}Y_{ab}\|\leq 2^{i-\ell}L$, and applying Theorem 5.15 with $V_{z}=W_{H_{i}}(z)-W_{H_{i-1}}(z)$ and $u=\sqrt{\log n}$ gives that

[TABLE]

Together with (28), and taking union bound over the $\ell=O(\log n)$ classes, we get that

[TABLE]

with probability $n^{-\Omega(1)}$, as desired. ∎

Appendix A Proofs of Lemma 4.6 and Lemma 4.7

For completeness we give the proofs of Lemma 4.6 and Lemma 4.7 that are very similar to those of Lemma 4.3 and Lemma 4.4, respectively.

See 4.6

Proof.

The proof closely follows that of Lemma 4.3 and is similar to the proof of Lemma 3.2 by Bilu and Linial [6]. We first observe that we only need to verify Property 1 for those disjoint sets $S,T\subseteq V$ such that $G[S\cup T]$ is connected. To see this, let $G[S\cup T]$ denote the subgraph induced by $S\cup T$. Suppose $G[S\cup T]$ is not connected and let $S_{1}\cup T_{1},S_{2}\cup T_{2},\ldots,S_{k}\cup T_{k}$ be the vertex sets of the connected components where $S_{1},\ldots,S_{k}\subseteq S$ and $T_{1},\ldots,T_{k}\subseteq T$. If Property 1 holds for connected components then $\left|2\cdot e_{F}(S_{i},T_{i})-e_{E}(S_{i},T_{i})\right|\leq 10\sqrt{d\log d}\cdot\sqrt{|S_{i}||T_{i}|}$ for $i=1,\ldots,k$, and so

[TABLE]

It is thus sufficient to prove the inequalities for those disjoint vertex sets $S,T$ that induce a connected subgraph $G[S\cup T]$.

Suppose we select $F$ by including each edge $e\in E$ with probability $1/2$ independently of other edges. That is, in the notation of Theorem 4.2, we have that $\mathcal{P}$ consists of $|E|$ mutually independent variables $\{P_{e}\}_{e\in E}$, where $P_{e}$ indicates whether $e\in F$ and $\Pr[P_{e}]=1/2$. Now for each $S$ and $T$ such that $G[S\cup T]$ is connected, let $A_{S,T}$ be the “bad” event that $\left|2\cdot|\delta_{F}(S,T)|-|\delta_{E}(S,T)|\right|>10\sqrt{d\log d}\cdot\sqrt{|S||T|}$. Note that $|\delta_{F}(S,T)|$ is the sum of at most $d\sqrt{|S||T|}$ independent variables, attaining values [math] and $1$, and that the expected value of $|\delta_{F}(S,T)|$ equals $|\delta_{E}(S,T)|/2$. Thus by the Chernoff inequality we get

[TABLE]

Similarly, if we let $D_{v}$ denote the bad event that the degree constraint of $v$ is violated, i.e., $\left|2\cdot e_{F}(v)-e_{E}(v)\right|>10\sqrt{d\log d}$. Then

[TABLE]

To apply Theorem 4.2, we analyze the dependency graph on the events:

• •

There is an edge between $A_{S,T}$ and $A_{S^{\prime},T^{\prime}}$ if $\operatorname{vbl}(A_{S,T})\cap\operatorname{vbl}(A_{S^{\prime},T^{\prime}})\neq\emptyset\Leftrightarrow\delta_{E}(S,T)\cap\delta_{E}(S^{\prime},T^{\prime})\neq\emptyset$.

• •

There is an edge between $A_{S,T}$ and $D_{v}$ if $\operatorname{vbl}(A_{S,T})\cap\operatorname{vbl}(D_{v})\neq\emptyset\Leftrightarrow\delta_{E}(S,T)\cap\delta_{E}(v)\neq\emptyset$.

• •

There is an edge between $D_{u}$ and $D_{v}$ if $\operatorname{vbl}(D_{u})\cap\operatorname{vbl}(D_{v})\neq\emptyset\Leftrightarrow\delta_{E}(u)\cap\delta_{E}(v)\neq\emptyset$.

Consider now a fixed event $A_{S,T}$ and let $k=|S\cup T|$. We bound the number of neighbors, $A_{S^{\prime},T^{\prime}}$, of $A_{S,T}$ with $|S^{\prime}\cup T^{\prime}|=\ell$. Since we are interested in only subsets $S^{\prime},T^{\prime}$ such that $G[S^{\prime}\cup T^{\prime}]$ is connected, this is bounded by the number of distinct subtrees on $\ell$ vertices in the associated graph $G$, with a root in one of the endpoints of an edge in $\delta(S,T)$. There are thus at most $2d\min(|S|,|T|)\leq dk$ many choices of the root and, as $G$ has degree at most $d$, the number of such trees is known to be at most (see e.g. [7])

[TABLE]

where we used that ${d(\ell-1)\choose\ell-1}\leq(ed)^{\ell-1}$. Moreover, it is easy to see that $A_{S,T}$ has at most $2d\min(|S|,|T|)\leq dk$ neighbors $B_{v}$.

Now to verify condition (13) of Theorem 4.2, we set $x(A_{S\cup T})=d^{-3|S\cup T|}$ for every bad event $A_{S,T}$ and $x(B_{v})=d^{-3}$ for every bad event $B_{v}$. So if we consider an event $A_{S,T}$ with $k=|S\cup T|$, then

[TABLE]

where we used that $d$ is a sufficiently large constant, which is without loss of generality since if $d\leq 10\sqrt{d\log(d)}$ then the lemma becomes trivial. In other words, (13) is satisfied for events $A_{S,T}$ with $\epsilon$ set to $1/2$. Let us now consider an event $B_{v}$. Clearly there is at most $d$ other events $B_{u}$ such that $B_{u}\sim B_{v}$. Moreover, by the same arguments as above there are at most $2d(ed)^{\ell-1}$ neighbors $A_{S,T}$ such that $|S\cup T|=\ell$. Hence

[TABLE]

where the second to last inequality follows because of the same simplifications as done above with $k=1$. We have thus verified that (13) is satisfied for all events with $\epsilon$ set to $1/2$.

It remains to define an efficiently verifiable core subset $\mathcal{A}^{\prime}\subseteq\mathcal{A}$ such that $1-\sum_{A\in\mathcal{A}\setminus\mathcal{A}^{\prime}}x(A)\geq 1-n^{-3}$. We let

[TABLE]

By the same arguments as in (29), there is at most $n\cdot{d(\ell-1)\choose\ell-1}\leq n(ed)^{\ell-1}$ many vertex sets $U$ such that $|U|=\ell$ and $G[U]$ is connected. Moreover, for each $U$ there are $2^{\ell}$ possible ways of partitioning it into $S$ and $T$. Therefore, the following properties hold:

$\mathcal{A}^{\prime}$ is efficiently verifiable since it contains $n\cdot\sum_{\ell=1}^{s}(ed)^{\ell-1}2^{\ell}=O(n\cdot(ed\cdot 2)^{s})=O(n^{3})$ many events $A_{S,T}$ that can be efficiently enumerated by first selecting a vertex $r$ among $n$ choices, then considering all possible trees rooted at $r$ with $\ell\leq s$ vertices, and all possible ways of partitioning such a component into $S$ and $T$. Moreover, the remaining $n$ events $B_{v}$ in $\mathcal{A}^{\prime}$ contains $n$ are easy to verify in polynomial time. 2. 2.

We have

[TABLE]

where for the first inequality we again used that $d$ is a sufficiently large constant.

We have verified Condition (13) of Theorem 4.2 and we have defined an efficiently verifiable core subset $\mathcal{A}^{\prime}$ such that $\sum_{A_{S}\in\mathcal{A}\setminus\mathcal{A}^{\prime}}x(A_{S})\leq n^{-3}$ and so the lemma follows.

∎

See 4.7

Proof.

Starting with $G$ we apply Lemma 4.6 $k$ times to obtain $\tilde{G}$. Let $F_{i}$ denote the edge set and let $d_{i}$ denote the maximum degree after round $i$. So $F_{0}=E$ and $d_{0}=d$. By the guarantees of Lemma 4.6, we have that with probability $1-n^{-3}$

[TABLE]

and

[TABLE]

As we apply Lemma 4.6 $k$ times with $k\leq\log(n)$, the union bound implies that the above inequalities are true for all invocations of that lemma with probability at least $1-k\cdot n^{-3}\geq 1-n^{-2}$. From now on we assume that the above inequalities hold and show that the conclusion of the statement is always true in that case. Specifically, we now prove by induction on $k$ that

[TABLE]

The claim holds trivially for $k=0$. Assume it holds for all $i<k$, which in particular implies $2^{i}d_{i}\leq 2d_{0}$ for all $i<k$. By the triangle inequality and (30),

[TABLE]

As the terms increase geometrically in $i$, this sum is $O(2^{k}\sqrt{(d_{0}/2^{k})\log((d_{0}/2^{k}))}$ which is $\varepsilon d_{0}$ by our assumption on $k$ and selection of $c$.

Finally, we note that $\left|2^{k}\cdot e_{F_{k}}(S,T)-e_{F_{0}}(S,T)\right|\leq\varepsilon d_{0}\cdot\sqrt{|S||T|}$ follows by the same calculations (using (31) instead of (30)). ∎