Testing Bipartitness in an Augmented VDF Bounded-Degree Graph Model

TL;DR

This paper develops nearly-linear time property testers for bipartiteness and cycle-freeness in an augmented bounded-degree graph model that includes access to an unknown vertex distribution, extending previous models.

Contribution

It introduces an augmented testing model with an oracle for the unknown distribution and provides almost-linear complexity testers for bipartiteness and cycle-freeness.

Findings

Developed nearly-linear time testers for bipartiteness.

Created testers for cycle-freeness with similar complexity.

Extended the VDF model with an oracle for the distribution.

Abstract

In a recent work (ECCC, TR18-171, 2018), we introduced models of testing graph properties in which, in addition to answers to the usual graph-queries, the tester obtains {\em random vertices drawn according to an arbitrary distribution $D$}. Such a tester is required to distinguish between graphs that have the property and graphs that are far from having the property, {\em where the distance between graphs is defined based on the unknown vertex distribution $D$}. These ("vertex-distribution free" (VDF)) models generalize the standard models in which $D$ is postulated to be uniform on the vertex-set, and they were studies both in the dense graph model and in the bounded-degree graph model. The focus of the aforementioned work was on testers, called {\sf strong}, whose query complexity depends only on the proximity parameter $\epsilon$. Unfortunately, in the standard bounded-degree graph model, some natural properties such as Bipartiteness do not have strong testers, and others (like cycle-freeness) do not have strong testers of one-sided error (whereas one-sided error was shown inherent to the VDF model). Hence, it was suggested to study general (i.e., non-strong) testers of "sub-linear" complexity. In this work, we pursue the foregoing suggestion, but do so in a model that augments the model presented in the aforementioned work. Specifically, we provide the tester with an evaluation oracle to the unknown distribution $D$, in addition to samples of $D$ and oracle access to the tested graph. Our main results are testers for Bipartitness and cycle-freeness, in this augmented model, having complexity that is almost-linear in the square root of the "effective support size" of $D$.

Equations

$${\delta }_{\mathcal{D}}(g,g^{\prime })\stackrel{\mathrm{def}}{=}{\mathrm{Pr}}_{v\leftarrow \mathcal{D},i\in [d]}[g(v,i)\ne g^{\prime }(v,i)].$$

$$\sum _{\{u,v\}\in E^{\prime }}{m}_{u,v}$$

$$\sum _{u,v\in V^{\prime }:{\chi }^{\prime }(u)={\chi }^{\prime }(v)}{m}_{u,v}\le |V^{\prime \prime }|+\sum _{⟨u,i⟩,⟨v,j⟩\in V^{\prime \prime }:{\chi }^{\prime \prime }(⟨u,i⟩)={\chi }^{\prime \prime }(⟨v,j⟩)}{m}_{u,v}^{i,j}.$$

$$\mathrm{Exp}\left[\sum _{u,v\in V^{\prime }}{m}_{u,v}\cdot X_{u,v}\right]$$

$$\sum _{e:P(e)=0}\mathcal{D}(e)$$

Full text

1. 1 Introduction

2. 1.1 The vertex-distribution-free model and its augmentation

3. 1.2 Our results

4. 1.3 Techniques

5. 1.4 Discussion

6. 2 The Bipartiteness Tester

7. 2.1 Proof of Theorem 2.1

8. 2.2 Approximating the effective support size of ${\cal D}$

9. 3 The Cycle-freeness Tester

10. Acknowledgments

1 Introduction

In the last couple of decades, the area of property testing has attracted much attention (see, e.g., a recent textbook [3]). Loosely speaking, property testing typically refers to sub-linear time probabilistic algorithms for deciding whether a given object has a predetermined property or is far from any object having this property. Such algorithms, called testers, obtain local views of the object by making adequate queries; that is, the object is seen as a function and the testers get oracle access to this function (and thus may be expected to work in time that is sub-linear in the size of the object).

A significant portion of the foregoing research was devoted to testing graph properties in two different models: the dense graph model (introduced in [5] and reviewed in [3, Chap. 8]) and the bounded-degree graph model (introduced in [6] and reviewed in [3, Chap. 9]). In both models, it was postulated that the tester can sample the vertex-set uniformly at random111Actually, in all these models, it is postulated that the vertex-set consists of $[n]=\{1,2,...,n\}$, where $n$ is a natural number that is given explicitly to the tester, enabling it to sample $[n]$ uniformly at random. (and, in both models, distances between graphs were defined with respect to this distribution).

In a recent work [4], we considered settings in which uniformly sampling the vertex-set of the graph is not realistic, and asked what happens if the tester can obtain random vertices drawn according to some distribution ${\cal D}$ (and, in addition, obtain answers to the usual graph-queries). The distribution ${\cal D}$ should be thought of as arising from some application, and it is not known a priori to the (application-independent) tester. In this case, it is also reasonable to define the distance between graphs with respect to the distribution ${\cal D}$, since this is the distribution that the application uses.

These considerations led us to introduce models of testing graph properties in which the tester obtains random vertices drawn according to an arbitrary vertex distribution ${\cal D}$ (and, in addition, obtains answers to the usual graph-queries). Such a tester is required to distinguish between graphs that have the property and graphs that are far from having the property, where the distance between graphs is defined based on the unknown vertex distribution ${\cal D}$. These (“vertex-distribution free” (VDF)) models generalize the standard models in which ${\cal D}$ is postulated to be uniform on the vertex-set, and they were studies both in the dense graph model and in the bounded-degree graph model (see [4, Sec. 2] and [4, Sec. 3], respectively).

The focus of [4] was on testers, called strong, whose query complexity depends only on the proximity parameter $\epsilon$. Unfortunately, in the standard bounded-degree graph model, some natural properties such as bipartiteness do not have strong testers, and others (like cycle-freeness) do not have strong testers of one-sided error (whereas one-sided error was shown inherent to the VDF model [4, Thm. 1.1]). Hence, it was suggested in [4, Sec. 5.2] to study general (i.e., non-strong) testers of “sub-linear” complexity, especially for the VDF bounded-degree graph model.

In this work, we pursue the foregoing suggestion, but do so in a model that augments the model presented in [4]. Specifically, we provide the tester with an evaluation oracle to the unknown distribution ${\cal D}$, in addition to samples of ${\cal D}$ and oracle access to the tested graph.

1.1 The vertex-distribution-free model and its augmentation

We start by recalling the vertex-distribution free (VDF) model that generalizes the bounded-degree graph model. Essentially, this model differs from the standard bounded-degree graph model in that the tester cannot obtain uniformly distributed vertices, but rather random vertices drawn according to an arbitrary distribution ${\cal D}$ (which is unknown a priori)). (In addition, the tester obtains answers to the usual graph-queries.) As usual, the tester is required to accept (whp) graphs that have the predetermined property and reject (whp) graph that that are far from the property, but the distance between graphs is defined in terms of the distribution ${\cal D}$. Specifically, when generalizing the standard bounded-degree model, we define the distance between graphs as the sum of the weights of the edges in their symmetric difference, where the weight of an edge is proportional to the sum of the probability weights of its end-points according to ${\cal D}$.

Recall that the bounded-degree model (both in its standard and VDF incarnations) refers to a fixed degree bound, denoted $d$, and to graphs that are represented by their incidence functions; that is, the graph $G=(V,E)$ is represented by the incidence function $g:V\times[d]\to V\cup\{\bot\}$, where $g(v,i)=u$ if $u$ is the $i^{\rm th}$ neighbour of $v$ and $g(v,i)=\bot$ if $v$ has less than $i$ neighbours. Fixing a vertex distribution ${\cal D}$, we say that the graph $G$ or rather its incidence function $g$ is $\epsilon$-far from the graph property $\Pi$ if, for every $g^{\prime}:V\times[d]\to V\cup\{\bot\}$ that represents a graph in $\Pi$, it holds that ${\rm Pr}_{v\leftarrow{\cal D},i\in[d]}[g(v,i)\!\neq\!g^{\prime}(v,i)]>\epsilon$.

Hence, following [4, Def. 3.1], a tester of $\Pi$ in the VDF bounded-degree graph model is given a proximity parameter $\epsilon$, samples drawn from an arbitrary distribution ${\cal D}$, and oracle access to the incidence function of the graph, $G=(V,E)$. It is required that, for every vertex-distribution ${\cal D}$ (and every $\epsilon>0$ and $G$), the tester accepts (whp) if $G$ is in $\Pi$ and rejects (whp) if $G$ is $\epsilon$-far from $\Pi$ (where the distance is defined according to ${\cal D}$).

The augmentation.

Here, we augment the foregoing model by providing the tester also with an evaluation oracle to the vertex distribution; that is, an oracle that on query $v$ returns ${\cal D}(v)={\rm Pr}_{x\leftarrow{\cal D}}[x\!=\!v]$. This augmentation is introduced because we could not obtain our results without it (or without some relaxation of it), but it can be justified as feasible in some settings (see brief discussion in Section 1.4). At this point, we spell out the resulting definition of a tester.

Definition 1.1

(the augmented VDF testing model):*

For a fixed $d\in{\mathbb{N}}$, let $\Pi$ be a property of graphs of degree at most $d$. An augmented VDF tester for the graph property $\Pi$ (in the bounded-degree graph model) is a probabilistic oracle machine $T$ that satisfies the following two conditions (for all sufficiently large $V$), when given access to the following three oracles: an incidence function $g:V\times[d]\to V\cup\{\bot\}$, a device – denoted ${{\tt samp}_{\cal D}}$ – that samples in $V$ according to an arbitrary distribution ${\cal D}$, and an evaluation oracle – denoted ${{\tt eval}_{\cal D}}$ – for ${\cal D}$.*

The tester accepts each $G=(V,E)\in\Pi$ with probability at least $2/3$; that is, for every $g:V\times[d]\to V\cup\{\bot\}$ representing a graph in $\Pi$ and every ${\cal D}$ (and $\epsilon>0$), it holds that ${\rm Pr}[T^{g,{{\tt samp}_{\cal D}},{{\tt eval}_{\cal D}}}(\epsilon)\!=\!1]\geq 2/3$. 2. 2.

Given $\epsilon>0$ and oracle access to any $G=(V,E)$ and ${\cal D}$ such that $G$ is $\epsilon$-far from $\Pi$ according to ${\cal D}$, the tester rejects with probability at least $2/3$; that is, for every $\epsilon>0$ and distribution ${\cal D}$, if $g:V\times[d]\to V\cup\{\bot\}$ satisfies $\delta_{\cal D}^{\Pi}(g)>\epsilon$, then it holds that ${\rm Pr}[T^{g,{{\tt samp}_{\cal D}},{{\tt eval}_{\cal D}}}(\epsilon)\!=\!0]\geq 2/3$, where $\delta_{\cal D}^{\Pi}(g)$ denotes the minimum of $\delta_{\cal D}(g,g^{\prime})$ taken over all incidence functions $g^{\prime}:V\times[d]\to V\cup\{\bot\}$ that represent graphs in $\Pi$, and

[TABLE]

(That is, $\delta_{\cal D}(g,g^{\prime})=\sum_{v\in V}{\cal D}(v)\cdot|\{i\in[d]:g(v,i)\neq g^{\prime}(v,i)\}|/d$.)**

The tester is said to have one-sided error probability if it always accepts graphs in $\Pi$; that is, for every $g:V\times[d]\to V\cup\{\bot\}$ representing a graph in $\Pi$ (and every ${\cal D}$ and $\epsilon>0$), it holds that ${\rm Pr}[T^{g,{{\tt samp}_{\cal D}},{{\tt eval}_{\cal D}}}(\epsilon)\!=\!1]=1$.

At times, we shall identify the incidence function $g$ with the graph $G$ that $g$ represents, and simply say that we provide the tester with oracle access to $G$.

The query complexity of a tester is the maximum number of queries it makes to its (three) oracles as a function of $\epsilon$ and parameters of the vertex-distribution ${\cal D}:V\to[0,1]$. The parameters we have in mind are label-invariant, where a parameter $\psi$ is label invariable if $\psi({\cal D})=\psi({\cal D}^{\prime})$ for any two distributions ${\cal D}$ and ${\cal D}^{\prime}$ that have the same histogram (i.e., for every $p>0$ it holds that $|\{v:{\cal D}(v)\!=\!p\}|=|\{v:{\cal D}^{\prime}(v)\!=\!p\}|$).222Indeed, in this case there exists a permutation $\pi:V\to V$ such that ${\cal D}^{\prime}(v)={\cal D}(\pi(v))$ for every $v\in V$. Recall that in the standard testing model, the complexity could depend on the size of the vertex-set, which is a special case of a parameter of ${\cal D}:V\to[0.1]$. However, we are interested in more refined parameters of ${\cal D}$. The first parameter that comes to mind is the support size of ${\cal D}$, but this parameter is too sensitive to insignificant changes in ${\cal D}$ (e.g., any distribution over $V$ is infinitesimally close to having support size $|V|$). A more robust parameter is the “minimum effective support size”; that is, being “close” to a distribution with the specified support-size (cf., [1]).

Definition 1.2

(effective support size):*

We say that the distribution ${\cal D}$ has $\eta$-effective support of size $n$ if ${\cal D}$ is $\eta$-close to a distribution that has support size at most $n$, where ${\cal D}$ is $\epsilon$-close to ${\cal D}^{\prime}$ if their total variation distance is at most $\epsilon$. The minimal $\eta$-effective support size of ${\cal D}$ is the minimal $n$ such that ${\cal D}$ has $\eta$-effective support of size $n$.*

The notion of effective support size is much more robust that the support size; in particular, if ${\cal D}$ is infinitesimally close to a distribution that has $\eta$-effective support of size $n$, then ${\cal D}$ that has $\eta$-effective support of size $n+1$ (where the additional unit is needed only in pathological cases).333Let ${\cal D}^{\prime}$ be the foregoing distribution that has $\eta$-effective support of size $n$. Then, the typical case is that, for some $\eta^{\prime}<\eta$, the distribution ${\cal D}^{\prime}$ has $\eta^{\prime}$-effective support of size $n$. In this case, any distribution that is $(\eta-\eta^{\prime})$-close to ${\cal D}^{\prime}$ has $\eta$-effective support of size $n$. The pathological case is that ${\cal D}^{\prime}$ has $\eta$-effective support of size $n$, but for every $\eta^{\prime}<\eta$ the minimal $\eta^{\prime}$-effective support size of ${\cal D}^{\prime}$ is larger than $n$. We claim that in this case, for some $\eta^{\prime}<\eta$, the distribution ${\cal D}^{\prime}$ has $\eta^{\prime}$-effective support of size $n+1$ (and it follows that any distribution that is $(\eta-\eta^{\prime})$-close to ${\cal D}^{\prime}$ has $\eta$-effective support of size $n+1$). Suppose that ${\cal D}^{\prime}$ is $\eta$-close to a distribution ${\cal D}^{\prime\prime}$ of support size $n$. We prove the claim by considering two cases.

If the support of ${\cal D}^{\prime}$ is contained in the support of ${\cal D}^{\prime\prime}$, then the claim is trivial (since then ${\cal D}^{\prime}$ has support size $n$).

Otherwise, let $v$ be in the support of ${\cal D}^{\prime}$ but not in the support of ${\cal D}^{\prime\prime}$, and consider modifying ${\cal D}^{\prime\prime}$ by moving a probability mass of ${\cal D}^{\prime}(v)>0$ from $\{u:{\cal D}^{\prime\prime}(u)>{\cal D}^{\prime}(u)\}$ to $v$. Then, the modified distribution ${\cal D}^{\prime\prime\prime}$ has support size $n+1$ and is $(\eta-{\cal D}^{\prime}(v))$-close to ${\cal D}^{\prime}$, and so the claim follows with $\eta^{\prime}=\eta-{\cal D}^{\prime}(v)$.

In general, if ${\cal D}$ is $o(\epsilon)$-close to a distribution that has $\epsilon$-effective support size $n$, then ${\cal D}$ that has $(1+o(1))\cdot\epsilon$-effective support size $n$.

An initial observation and an open problem.

Recall that it was shown in [4, Prop. 3.2] that, without loss of generality, any tester in the VDF model only queries vertices that were provided as answers to prior sample and graph queries. The argument extends to the augmented VDF model. In contrast, it is unclear whether [4, Thm. 3.3], which asserts that strong testability in the VDF model yields strong testability with one-sided error, holds in the augmented VDF model.

Open Problem 1.3

(does one-sided error testing reduce to general testing):*

For $q:(0,1]\to{\mathbb{N}}$, suppose that $\Pi$ is a graph property that can be tested using $q(\epsilon)$ queries in the augmented VDF model, where $\epsilon$ denotes the proximity parameter. Does there exists a function $q^{\prime}:(0,1]\to{\mathbb{N}}$ such that $\Pi$ has a one-sided error tester of query complexity $q^{\prime}(\epsilon)$ in the augmented VDF model.*

(Recall that in the original VDF model, an upper bound of $q^{\prime}(\epsilon)=\exp(O(q(\epsilon)))$ was shown [4, Thm. 3.3].)

1.2 Our results

Our main result is testing Bipartitness in the augmented VDF model within complexity that matches the complexity of the known tester in the standard (bounded-degree graph) model [7], which in turn is almost optimal [6].

Theorem 1.4

(testing bipartitness in the augmented VDF model):*

Bipartiteness can be tested in the augmented VDF testing model (of Definition 1.1) in expected time ${\widetilde{O}}({\sqrt{n}})\cdot{\rm poly}(1/\epsilon)$, where $n$ denotes the minimal $\epsilon/5$-effective support size of the vertex distribution ${\cal D}$, used by the tester. Furthermore, the tester has one-sided error.*

The tester that we use when proving Theorem 1.4 starts by obtaining a rough approximation of the minimal effective support size of ${\cal D}$. In fact, a rough approximation of the effective support size of ${\cal D}$ is implicit in the running-time of the asserted tester. We comment that it is essential to make queries to ${{\tt eval}_{\cal D}}$ in order to obtain such an approximation, since obtaining such an approximation by making queries only to ${{\tt samp}_{\cal D}}$ has complexity $n^{1-o(1)}$ (cf. [9]).444Specifically, at least $n^{1-o(1)}$ queries are necessary to distinguish an $n$-grained distribution of support size $n/11$ from an $n$-grained distribution with support size $n/f$, for any $f=n^{o(1)}$, where a distribution is called $n$-grained if all probabilities are multiples of $1/n$. In fact, using both types of queries, we obtain a good approximation in polylogarithmic time (see Section 2.2).

Next, we extend the known reduction of testing cycle-freeness to testing Bipartiteness, presented in [2] for the standard (bounded degree graph) model, to the (augmented) VDF model. Combining this reduction with Theorem 1.4, we obtain

Theorem 1.5

(testing cycle-freeness in the augmented VDF model):*

Cycle-freeness can be tested in the augmented VDF testing model (of Definition 1.1) in expected time ${\widetilde{O}}({\sqrt{n}})\cdot{\rm poly}(1/\epsilon)$, where $n$ denotes the minimal $\epsilon/5$-effective support size of the vertex distribution ${\cal D}$ used by the tester. Furthermore, the tester has one-sided error.*

A begging open problem is whether the foregoing results can also be obtained in the original VDF model (of [4]), at least when providing the tester with the effective support size.

Open Problem 1.6

(the complexity of testing Bipartiteness in the VDF model):*

What is the query complexity of testing Bipartiteness in the (original) VDF model. In particular, can Bipartite be tested in this model in time ${\widetilde{O}}({\sqrt{n}})\cdot{\rm poly}(1/\epsilon)$, where $n$ denotes the $\Omega(\epsilon)$-effective support size of the vertex distribution ${\cal D}$ used by the tester? The question holds both for the VDF model (as defined in [4]), and in a model in which the tester is provided with the effective support size.555Note that the result of [9] does not rule out the possibility of approximating $n$ to a factor of $n^{c}$ using $n^{0.5+c^{\prime}}$ queries, for some $c,c^{\prime}\in(0,0.5)$.*

Since the reduction of testing cycle-freeness to Bipartiteness does not make queries to ${\cal D}$ (and is in fact oblivious of ${\cal D}$), any upper bound regarding testing Bipartiteness in the (original) VDF model would yield a similar result for testing cycle-freeness. However, the latter problem may be easier.

1.3 Techniques

The testers asserted in Theorems 1.4 and 1.5 constitute testers for the standard (bounded-degree graph) model that meet the best results known in that model. Given that the proofs of the latter results are quite complex (see, e.g., [7]), it is fortunate that we can proceed by reducing the current results to the known ones.

Proving Theorem 1.4.

The Bipartite tester asserted in Theorem 1.4 is obtained by a natural adaptation of the corresponding tester for the standard (bounded-degree graph) model [7]. The latter tester operates by taking many short random walks from few randomly selected vertices, where, in each step of a random walk, the next vertex is selected uniformly among the neighbors of the current vertex. Instead, our tester will select the the next vertex (among the neighbors of the current vertex) with probability that is proportional to the probability weight of the corresponding incident edges (according to ${\cal D}$); that is, being at vertex $v$ we move to a neighbor $w$ with probability proportional to ${\cal D}(v)+{\cal D}(w)$. Here is where we make use of the evaluation oracle ${{\tt eval}_{\cal D}}$. (The start vertices will be selected with probability that is proportional to the ${\cal D}$-weight of their incident edges.)

Since the analysis of the foregoing tester in the standard (bounded-degree graph) model is quite complex, we wish to use this analysis (of [7]) as a black-box. Towards this end, we view the foregoing tester as emulating the tester of [7] on an auxiliary graph in which weighted edges are replaced by a proportional number of parallel edges, while recalling that the analysis of [7] holds also for (non-simple) graphs having parallel edges (see [8]). (We stress that this is a mental experiment performed in the analysis; the actual algorithm is essentially as outlined above.)

Indeed, this strategy may yield an auxiliary graph in which the average degree is much smaller than the maximum degree, whereas the analysis in [7] relies on the two quantities being of the same order of magnitude (see [8]). This issue can be addressed by using the transformation of [8], but doing so will yield a slightly less natural tester (than the one outlined above). So instead of applying the transformation of [8] as a black-box, we use an alternative implementation of its underlying ideas, which capitalizes on the fact that we can set the number of parallel edges (in our mental experiment) to be much larger than the number of vertices in the tested graph. We stress that the complexity of the tester of [7] is dominated by the number of the vertices in the graph, and hardly depends on the number of edges in it.666The complexity is almost-linear in a square root of the number of vertices, and only grows logarithmically with the number of edges.

The forgoing description presumes that we have a good upper bound on the support size of ${\cal D}$. (Note that such an upper bound (along with oracle access to the auxiliary graph) suffices for emulating the tester of [7].) Actually, aiming at complexity bounds that depend on the effective support size of ${\cal D}$ rather than on its support size, we “trim” the graph by ignoring edges of weight $o(\epsilon/n)$, where $n$ is an upper bound on the $\epsilon/4$-effective support size of ${\cal D}$. Lastly, we show how to obtain such a good upper bound by using both the sampling and evaluation oracles of ${\cal D}$.

As for approximating the effective support size of ${\cal D}$, the basic observation is that if $n$ is the minimal integer such that an $\eta$ fraction of the weight of ${\cal D}$ resides on elements of weight at most $\eta/n$, then the $\Theta(\eta)$-support size of ${\cal D}$ is between $\Omega(n)$ and $O(n/\eta)$. Such a rough approximation suffices for the foregoing application, and we obtain it obtained by using a doubling procedure. We actually obtain a better approximation (see Theorem 2.2) by using the rough approximation as a starting point and approximating the total weight assigned to each set $\{v:2^{-i+1}\leq{\cal D}(v)<2^{-i}\}$ for $i\in[\log(n/\eta)+O(1)]$.

Proving Theorem 1.5.

The cycle-freeness tester asserted in Theorem 1.5 is obtained by using the known reduction of testing cycle-freeness to testing Bipartiteness, which was presented and analyzed for the standard (bounded degree graph) model in [2]. Recall that this reduction replaces each edge at random either by a path of length two or by a path of length one (equiv., leaves the edge intact). It was shown in [2, Lem 3.1] that this transformation translates a graph that is far from being cycle-free to a graph that is far from being bipartite, where the distance refers to the number of edges that should be omitted to make the graph satisfy the property. The challenge is to extend this claim to weighted graphs (or rather to distances as measured by the sum of the weights of edges that should be omitted to make the graph satisfy the property).

We meet this challenge by observing that the edges that should be omitted in order to make the original graph cycle-free are those that do not reside on a maximal spanning forest of the graph. We then bucket these edges according to their approximate weight, and consider only the edges that are in the union of the heaviest bucket and said spanning forest. Considering the two-connected components of the corresponding graph, we observe that the remaining edges of the forest (i.e., those that reside inside these two-connected components) cannot be lighter than the edges in the bucket, which reduces the problem of lower-bounding the weight of edges to lower-bounding their number. At this point, we invoke [2, Lem 3.1] and are done.

1.4 Discussion

As admitted upfront, the augmentation of the VDF model captured by Definition 1.1 is made for opportunistic reasons: This augmentation (or some relaxation of it) seems essential to the testers asserted in Theorems 1.4 and 1.5. Nevertheless, one may envision setting in which an evaluation oracle as postulated in the augmentation can be implemented or at least be well-approximated.

Recall that the VDF model was motivated by settings in which some process (or application) of interest refers to (or embeds or emulates) a huge graph; in particular, the process generates random vertices according to some unknown distribution ${\cal D}$, and answers incidence queries regarding the graph. Furthermore, in case the vertices of the graph are real sites, they may maintain a count of the number of times they were visited by the foregoing process (or application). This yields a good approximation of the visiting probabilities that underly the vertex distribution ${\cal D}$ in question.

Indeed, the vertex distribution represents the “importance” of the various vertices from the application’s point of view; that is, the application encounters vertices according to the distribution ${\cal D}$, and the relative “importance” of a vertex (to the application) is captured by the probability that it is encountered (by the application). Hence, the distance of a graph to the property represents the relative importance of the “part of the graph” that violates the property.

A VDF tester offers an application-independent way of determining whether the huge graph (embedded or emulated by an application) has some predetermined property or is far from having the property, where the distance that is relevant here is one that is induced by the vertex-distribution ${\cal D}$ (arising from the application).

2 The Bipartiteness Tester

For starters, we consider a model in which the tester is further augmented by providing it with the effective support size of the vertex distribution ${\cal D}$. Specifically, on input proximity parameter $\epsilon$, we also provide the tester with an upper bound, denoted $n$, on the minimal $\epsilon/4$-effective support size of ${\cal D}$. The complexity of the following tester depends on that upper bound, and at a latter stage we shall show how the tester can obtain a good upper bound by itself.

Theorem 2.1

(a tester that gets an upper bound on the effective support size):*

There exists an oracle machine $T$ that, on input $\epsilon$ and $n$ such that $n$ is an upper bound on the minimal $\epsilon/4$-effective support size of ${\cal D}$, and oracle access to $G,{{\tt samp}_{\cal D}},{{\tt eval}_{\cal D}}$, runs in time ${\widetilde{O}}({\sqrt{n}})\cdot{\rm poly}(1/\epsilon)$ and constitutes a bipartite tester (of one-sided error) as defined above. That is:*

If $G$ is bipartite, then ${\rm Pr}[T^{G,{{\tt samp}_{\cal D}},{{\tt eval}_{\cal D}}}(\epsilon,n)\!=\!1]=1$. 2. 2.

If $G$ is $\epsilon$-far from being bipartite according to ${\cal D}$, then ${\rm Pr}[T^{G,{{\tt samp}_{\cal D}},{{\tt eval}_{\cal D}}}(\epsilon,n)\!=\!0]\geq 2/3$.

The Bipartite tester asserted in Theorem 1.4 is obtained by combining Theorem 2.1 with a procedure that approximates the effective support size, which in turn is provided in Section 2.2.

2.1 Proof of Theorem 2.1

Our starting point is the Bipartiteness tester of [7] that works in the standard (bounded-degree graph) model. On input an $n$-vertex graph, this tester operates by taking many (i.e., ${\widetilde{O}}({\sqrt{n}})$) short (i.e., ${\rm poly}(\log n)$) random walks from few (i.e., $O(1)$) randomly selected vertices (where the $O$-notation hide a polynomial dependence on $1/\epsilon$). In [7], in each step of each random walk, the next vertex is selected uniformly among the neighbors of the current vertex. Our basic idea is to adapt this tester to the current setting by selecting the next vertex among the neighbors of the current vertex with probability that is proportional to the probability weight of the corresponding incident edges (according to ${\cal D}$).

Wishing to use the analysis of [7] as a black-box, we present the foregoing tester as emulating the tester of [7] on an auxiliary graph in which weighted edges are replaced by a proportional number of parallel edges, while recalling that the analysis of [7] holds also for (non-simple) graphs having (many) parallel edges (see [8]). That is, on input $G$, we consider an auxiliary multi-graph $G^{\prime}$ in which the number of edges between vertices is approximately proportional to the weight of the corresponding edge in $G$. Indeed, this may yield a graph with a large discrepancy between the average degree and the maximum degree, but this can be fixed using the technique of [8]. In fact, in order to present a simpler and more natural algorithm, we use an alternative implementation of the foregoing technique, which suffices for our setting.

Fixing a vertex distribution ${\cal D}$, on input $G=(V,E)$, proximity parameter $\epsilon$ and the value $n$ (provided as an effective support size of ${\cal D}$), we consider the corresponding auxiliary multi-graph $G^{\prime}=(V^{\prime},E^{\prime})$. For sake of simplicity, we shall assume first that for each $v\in V$ either ${\cal D}(v)=0$ or ${\cal D}(v)\geq\rho$, where we later set $\rho=\Theta(\epsilon/n)$ and reduce the general case to this special case. Recalling that the weight of the edge $\{u,v\}\in E$ under ${\cal D}$ is $2\cdot({\cal D}(u)+{\cal D}(v))/d$, we place $m_{u,v}\stackrel{{\scriptstyle\rm def}}{{=}}{\lfloor{({\cal D}(u)+{\cal D}(v))\cdot N}\rceil}\gg 1$ parallel edges between $u$ and $v$ in $G^{\prime}$, where $N={\rm poly}(d/\epsilon\rho)$ will be determined later. We keep only the non-isolated vertices in $V^{\prime}$; that is, $v\in V^{\prime}$ if and only if $\sum_{u}m_{u,v}>0$, which holds if and only if either ${\cal D}(v)\!\geq\!\rho$ or ${\cal D}(u)\!\geq\!\rho$ for some neighbor $u$ of $v$. Hence, $|V^{\prime}|\leq d/\rho$.

Indeed, the number of parallel edges between $u$ and $v$ is either zero or at least $\rho\cdot N-1$, and the total the number of edges in $G^{\prime}$ (i.e., $\sum_{\{u,v\}\in E}{\lfloor{({\cal D}(u)+{\cal D}(v))\cdot N}\rceil}$) is roughly $\sum_{w\in V}d_{w}\cdot{\cal D}(w)\cdot N=\Theta(N)$, where $d_{v}$ is the degree of $v$ in $G$. Hence, if $G$ is $\epsilon$-far from being bipartite, under ${\cal D}$, then $G^{\prime}$ is $(\epsilon-o(\epsilon))$-far from being bipartite (in the sense that more than $(\epsilon-o(\epsilon))\cdot|E^{\prime}|$ edges must be omitted from $G^{\prime}$ to make it bipartite). Specifically, we have:

Claim 2.1.1

(weight under ${\cal D}$ versus number of edges in $G^{\prime}$):*

If edges of total ${\cal D}$-weight at least $\delta$ must be removed from $G$ in order to make it bipartite, then at least $\delta\cdot N-d|V^{\prime}|$ edges must be removed from $G^{\prime}$ in order to make it bipartite.*

Indeed, if $N=\omega(|V^{\prime}|/\delta)$, then $\delta\cdot N-d|V^{\prime}|=(\delta-o(\delta))\cdot N$.

Proof:  Removing a weighted edge of $G$ is equivalent to removing the corresponding parallel edges of $G^{\prime}$. Recalling that an edge $\{u,v\}\in E$ of weight $w_{u,v}=2\cdot({\cal D}(u)+{\cal D}(v))/d\leq{\cal D}(u)+{\cal D}(v)$ yields $m_{u,v}={\lfloor{({\cal D}(u)+{\cal D}(v))\cdot N}\rceil}\geq w_{u,v}\cdot N-1$ parallel edges and using $|E|<d|V^{\prime}|$, the claim follows.777For any $E^{\prime}\subseteq E$, we have

$\displaystyle\sum_{\{u,v\}\in E^{\prime}}m_{u,v}$ $\displaystyle\geq$ $\displaystyle\sum_{\{u,v\}\in E^{\prime}}(w_{u,v}\cdot N-1)$

$\displaystyle=$ $\displaystyle\left(\sum_{\{u,v\}\in E^{\prime}}w_{u,v}\right)\cdot N-|E^{\prime}|.$

At this point, we could have emulated an execution of the bipartite tester of [8] on $G^{\prime}=(V^{\prime},E^{\prime})$, which boils down to emulating the tester of [7] on an auxiliary graph $G^{\prime\prime}=(V^{\prime\prime},E^{\prime\prime})$, where $|V^{\prime\prime}|=O(|V^{\prime}|)$. (Such an emulation would have resulted in an algorithm that is slightly different from the one we outlined above.) Instead, we prefer to capitalize on the specific features of $G^{\prime}$, and emulate the tester of [7] on a different auxiliary graph $G^{\prime\prime}=(V^{\prime\prime},E^{\prime\prime})$ that is more closely related to $G^{\prime}$ (and to $G$). This allows us to invoke a minor variant of the tester of [7] (indeed, the variant outlined upfront) directly on $G$. In particular, we shall use the fact that the number of parallel edges (between a pair of connected vertices) in $G^{\prime}$ is large (e.g., much larger than the square of the ratio between the maximum and average degrees in $G^{\prime}$); that is, $M=\omega((N/M)^{2}/\epsilon)$, where $M=\min_{\{u,v\}\in E:m_{u,v}>0}\{m_{u,v}\}\geq\rho\cdot N-1$. (Recall that the maximum degree in $G^{\prime}$ is at most $N$, whereas the average degree is at least $M$, since each vertex in $G^{\prime}$ has degree at least $M$.)

The auxiliary graph $G^{\prime\prime}=(V^{\prime\prime},E^{\prime\prime})$.

As in [8], we wish to transform $G^{\prime}=(V^{\prime},E^{\prime})$ to a graph with a maximum degree that is only a constant factor larger than its average degree, while preserving the relative distance from the set of bipartite graphs. Denoting the degree of vertex $v$ (in $G^{\prime}$) by $d^{\prime}_{v}=\sum_{u}m_{v,u}$ and letting $d^{\prime}=\sum_{v\in V^{\prime}}d^{\prime}_{v}/|V^{\prime}|=2|E^{\prime}|/|V^{\prime}|$ denote the average degree of $G^{\prime}$, we replace each verterx in $G^{\prime}$ by a cloud, denoted $C_{v}$, of $c_{v}\stackrel{{\scriptstyle\rm def}}{{=}}{\lceil{d^{\prime}_{v}/d^{\prime}}\rceil}$ vertices. Hence, $V^{\prime\prime}=\bigcup_{v\in V^{\prime}}C_{v}$ and $|V^{\prime\prime}|=\sum_{v\in V^{\prime}}|C_{v}|<\sum_{v\in V^{\prime}}((d^{\prime}_{v}/d^{\prime})+1)=2|V^{\prime}|$. (Unlike in [8], we don’t augment the graph with bipartite expanders of degree $d^{\prime}$ between $C_{v}$ and an auxiliary set of $|C_{v}|$ vertices, but rather capitalize on the fact that the number of parallel edges between clouds is much larger than the product of the size of the cluds; that is, $m_{u,v}=\omega(|C_{u}|\cdot|C_{v}|/\epsilon)$ for every $u,v\in V^{\prime}$ such that $m_{u,v}>0$.) Next, for each $u,v\in E^{\prime}$ such that $m_{u,v}>0$, we partition the $m_{u,v}$ parallel edges that connect $u$ and $v$ (almost) equally among the pairs of vertices in $C_{u}\times C_{v}$. That is, for every ${\langle{u,i}\rangle}\in C_{u}$ and ${\langle{v,j}\rangle}\in C_{v}$, we connect ${\langle{u,i}\rangle}$ and ${\langle{v,j}\rangle}$ by $m_{u,v}^{i,j}$ parallel edges, where $m^{i,j}_{u,v}\in\{{\lfloor{m_{u,v}/|C_{u}\times C_{v}|}\rfloor},{\lceil{m_{u,v}/|C_{u}\times C_{v}|}\rceil}\}$.

The key observation is that a random walk on $G^{\prime}$ is closely related to a random walk on $G^{\prime\prime}$ in the sense that the walk on $G^{\prime}$ moves from $u$ to $v$ with (almost) the same probability that the walk on $G^{\prime\prime}$ moves from a vertex in $C_{u}$ to a vertex in $C_{v}$. This is the case since for every $i\in[c_{u}]$ it holds that $\frac{m_{u,v}}{\sum_{w}m_{u,w}}\approx\frac{m_{u,v}^{i,.}}{\sum_{w}m_{u,w}^{i,.}}$, where $m_{u,w}^{i,.}=\sum_{j\in[c_{w}]}m_{u,w}^{i,j}$. The approximation holds because $m^{i,.}_{u,w}\approx{m_{u,v}/c_{u}}$ which is due to $c_{u}\leq N/M\ll M$, which in turn is due to $M\geq\rho N-1$ and $N=\omega(1/\rho^{2})$. Actually, to get an approximation factor of $1\pm\rho$, we need $N/M\ll\rho\cdot M$, which holds when $N=\omega(1/\rho^{3})$. (Note that $\frac{m_{u,v}}{\sum_{w}m_{u,w}}\approx\frac{c_{v}\cdot m_{u,v}^{i,j}}{\sum_{w}m_{u,w}^{i,.}}$, holds if $(N/M)^{2}\ll M$, but we don’t use this fact.)

Indeed, $|E^{\prime\prime}|=|E^{\prime}|$, and so the average degree (which is $2|E^{\prime\prime}|/|V^{\prime\prime}|<2|E^{\prime\prime}|/2|V^{\prime}|=d^{\prime}/2$) is at least a third of the maximum degree (which is at most $\max_{{\langle{v,j}\rangle}\in V^{\prime\prime}}\{\sum_{{\langle{u,i}\rangle}\in V^{\prime\prime}}{\lceil{m_{u,v}/c_{u}c_{v}}\rceil}\}\leq d^{\prime}+(d/\rho)<1.5d^{\prime}$, since $d^{\prime}\geq M\geq\rho\cdot N-1=\omega(d/\rho)$).888To see the first inequality, note that $\sum_{{\langle{u,i}\rangle}\in V^{\prime\prime}}m_{u,v}/c_{u}c_{v}$ equals $\sum_{u\in V^{\prime}}m_{u,v}/c_{v}=d^{\prime}_{v}/c_{v}\leq d^{\prime}$, since $c_{v}={\lceil{d^{\prime}_{v}/d^{\prime}}\rceil}\geq d^{\prime}_{v}/d^{\prime}$. Hence, $\sum_{{\langle{u,i}\rangle}\in V^{\prime\prime}}{\lceil{m_{u,v}/c_{u}c_{v}}\rceil}$ is at most $d^{\prime}+|V^{\prime\prime}|\leq d^{\prime}+(d/\rho)$.

We conclude that one can employ the tester of [7] to $G^{\prime\prime}$, since the analysis in [7] presumes a constant ratio between the maximum and the average degrees. Of course, we have to guaranteed that the distance of $G$ to being bipartite is reflected by the distance of $G^{\prime\prime}$ to being bipartite. This follows by combining Claims 2.1.1 and 2.1.2.

Claim 2.1.2

(number of violating edges in $G^{\prime}$ versus $G^{\prime}$):*

If at least $\Delta$ edges must be removed from $G^{\prime}$ in order to make it bipartite, then at least $\Delta-|V^{\prime\prime}|$ edges must be removed from $G^{\prime\prime}$ in order to make it bipartite.*

The slackness is due to the fact that $m_{u,v}^{i,j}$ equals $\frac{m_{u,v}}{|C_{u}\times C_{u}|}$ up to at most one unit, rather than being equal to it. Loosely speaking, this means that 2-partitions of $G^{\prime\prime}$ with fewer inter-part edges are obtained by placing all vertices of each cloud on the same side. Intuitively, this holds because the edges between each pair of clouds are partitioned equally between the corresponding pairs of vertices, and so one does not benefit by treating vertices of the same cloud differently (i.e., placing them on different sides of the 2-partition).

Proof:  For every 2-partition $\chi^{\prime\prime}:V^{\prime\prime}\to\{1,2\}$, we present a 2-partition $\chi^{\prime}:V^{\prime}\to\{1,2\}$ such that the number of $\chi^{\prime}$-monochromatic edges in $G^{\prime}$ is at most $|V^{\prime\prime}|$ units larger than the number of $\chi^{\prime\prime}$-monochromatic edges in $G^{\prime\prime}$; that is,

[TABLE]

This is shown by using the probabilistic method. Specifically, fixing $\chi^{\prime\prime}:V^{\prime\prime}\to\{1,2\}$, we consider assigning each vertex of $G^{\prime}$ a color chosen at random according to the colors of the vertices in the corresponding cloud; that is, $\chi^{\prime}(v)=1$ with probability $|\{i\in[c_{v}]:\chi^{\prime\prime}({\langle{v,i}\rangle})=1\}|/|C_{v}|$, and $\chi^{\prime}(v)=2$ otherwise. Letting $X_{u,v}$ be a random variable indicating whether or not $\chi^{\prime}(u)=\chi^{\prime}(v)$ (i.e., $X_{u,v}=1$ if $\chi^{\prime}(u)=\chi^{\prime}(v)$ and $X_{u,v}=0$ otherwise), we have

[TABLE]

Recalling that the absolute difference between $m_{u,v}^{i,j}$ and $\frac{m_{u,v}}{|C_{u}\times C_{u}|}$ is at most one unit, it follows that the expected number of $\chi^{\prime}$-monochromatic edges in $G^{\prime}$ differs from the number of $\chi^{\prime\prime}$-monochromatic edges in $G^{\prime\prime}$ by at most $|V^{\prime\prime}|$ units.

The actual tester.

Recall that, when given oracle access to an $n$-vertex graph (and proximity parameter $\epsilon$), the tester of [7] selects uniformly $O(1/\epsilon)$ (start) vertices, and starts ${\rm poly}(1/\epsilon)\cdot{\widetilde{O}}({\sqrt{n}})$ random $\ell$-step walks from each vertex, where $\ell={\rm poly}(\epsilon^{-1}\log n)$ and each step in the random walk moves uniformly to one of the neighbors of the current vertex. The tester accepts if and only if the explored subgraph is bipartite (i.e., contains no odd-length cycles). When seeking to test $G$ under the vertex-distribution ${\cal D}$, we shall emulate a testing of $G^{\prime\prime}$ as follows.

• •

The $O(1/\epsilon)$ start vertices will be selected according to the degrees in $G^{\prime}$ (and in $G^{\prime\prime}$), which in turn reflect the weight of the incident edges. Hence, we wish to select $v$ with probability that is proportional to $\sum_{u\in\Gamma(v)}({\cal D}(u)+{\cal D}(v))$, where $\Gamma(v)$ denotes the set of neighbors of $v$ in $G$. We do so by employing “rejection sampling” as follows. First, we obtain a sample of ${\cal D}$; that is, $s\leftarrow{\cal D}$. Next, we output $s$ with probability $|\Gamma(s)|/2d$, and halt with no output with probability $(2d-2|\Gamma(s)|)/2d$, Otherwise (i.e., with probability $|\Gamma(s)|/2d$), we output a uniformly selected neighbor of $s$ (i.e., each neighbor of $s$ is output with probability $1/2d$).

Hence, the probability that $v$ is output (in a single trial) equals $p(v)\stackrel{{\scriptstyle\rm def}}{{=}}{\cal D}(v)\cdot\frac{|\Gamma(v)|}{2d}+\sum_{u\in\Gamma(v)}{\cal D}(u)\cdot\frac{1}{2d}$, where the first term is due to selecting $s=v$ and the second term is due to selecting $s\in\Gamma(v)$. Note that $p(v)=\sum_{u\in\Gamma(v)}({\cal D}(v)+{\cal D}(u))/{2d}$, whereas the degree of $v$ in $G^{\prime}$ is proportional to $\sum_{u\in\Gamma(v)}m_{u,v}\approx\sum_{u\in\Gamma(v)}({\cal D}(u)+{\cal D}(v))\cdot N$. (Indeed, we repeat trying till some vertex is output, while noting that each trial succeeds at least with probability $1/d$.)

• •

A step in a random walk is made by selecting a neighbor of the current vertex with probability that is proportional to the weight of the edge leading to it. Specifically, when we reside at $v\in V$, we move to the neighbor $u$ with probability proportional to ${\cal D}(v)+{\cal D}(u)$.

Note that for implementing the foregoing actions it suffices to be able to sample ${\cal D}$, and have oracle access to the evaluation of ${\cal D}$ (and to the incidence function of $G$). Actually, the evaluator of ${\cal D}$ can be replaced by an oracle that answers $(w_{1},w_{2})$ with ${\cal D}(w_{1})/{\cal D}(w_{2})$, provided ${\cal D}(w_{2})>0$ (and a special symbol otherwise).

The general case.

Recall that the foregoing analysis presumes that for each $v\in V$ either ${\cal D}(v)=0$ or ${\cal D}(v)\geq\rho$. Using $\rho=\Theta(\epsilon/n)$, we now reduce the general case of ${\cal D}$ that has $\epsilon/4$-effective support size $n$ to the foregoing case. We first note that, by the following Claim 2.1.3, the vertex-distribution ${\cal D}$ is $\epsilon/2$-close to a vertex-distribution ${\cal D}^{\prime}$ that satisfiers the foregoing condition with $\rho=\epsilon/4n$. Hence, if $G$ is $\epsilon$-far from being bipartite under ${\cal D}$, then it is $0.5\epsilon$-far from being bipartite under ${\cal D}^{\prime}$, and we can test $G$ by providing the tester with oracle access to ${\cal D}^{\prime}$ (i.e., to ${{\tt samp}_{\cal D}}^{\prime}$ and ${{\tt eval}_{\cal D}}^{\prime}$) and setting the proximity parameter to $\epsilon/2$. Actually, it suffices to provide the foregoing tester with sampling access to ${\cal D}^{\prime}$, which can be emulated by “rejection sampling” via ${\cal D}$, and with an evaluator of the ratios between the ${\cal D}^{\prime}$-values (which, in turn, equal the corresponding the ratios between the ${\cal D}$-values).

Claim 2.1.3

(effective support size and minimal weight):*

Suppose that ${\cal D}$ has an $\eta$-effective support of size $n$. Then, ${\cal D}$ is $2\eta$-close to a distribution ${\cal D}^{\prime}$ that satisfies $\min_{e:{\cal D}^{\prime}(e)>0}\{{\cal D}^{\prime}(e)\}>\eta/n$. Furthermore, ${\cal D}^{\prime}(e)>0$ if and only if ${\cal D}(e)>\eta/n$, and if ${\cal D}^{\prime}(e)>0$ then $\frac{{\cal D}^{\prime}(e^{\prime})}{{\cal D}^{\prime}(e)}=\frac{{\cal D}(e^{\prime})}{{\cal D}(e)}$ for every $e^{\prime}$.*

Combining the foregoing analysis with Claim 2.1.3, Theorem 2.1 follows.

Proof:  We may assume, without loss of generality, that the support of ${\cal D}$ has $s>n$ elements, and arrange these elements according to their ${\cal D}$-value; specifically, let $e_{1},...,e_{s}$ such that ${\cal D}(e_{i})\geq{\cal D}(e_{i+1})>0$ for every $i\in[s-1]$. Then, $\sum_{i=n+1}^{s}{\cal D}(e_{i})\leq\eta$, because the distance of ${\cal D}$ from the class of distributions of support size at most $n$ is at least $\sum_{i=n+1}^{s}{\cal D}(e_{i})$. Letting $P(e)={\cal D}(e)$ if ${\cal D}(e)>\eta/n$ and $P(e)=0$, we define ${\cal D}^{\prime}(e)=\frac{P(e)}{\sum_{i\in[s]}P(e_{i})}$, and notice that

[TABLE]

Noting that ${\cal D}^{\prime}(e)\geq{\cal D}(e)$ for every $e\not\in P^{-1}(0)$, the main claim follows. The furthermore claim follows by the definition of ${\cal D}^{\prime}$.

2.2 Approximating the effective support size of ${\cal D}$

The foregoing Bipartite tester, which establishes Theorem 2.1, presupposes that the tester is given an upper bound on the minimum $\epsilon/4$-effective support size of ${\cal D}$ as auxiliary input. Proving Theorem 1.4 requires getting rid of that auxiliary input, or rather approximating it when using oracle access to ${\cal D}$. Indeed, we shall show that given sampling and evaluation oracles to ${\cal D}$, it is relatively easy to approximate its effective support size. (In contrast, as commented in the introduction, obtaining such an approximation while using only samples of ${\cal D}$ is too expensive for our application.)

Note that the notion of approximating the effective support size is not robust in the sense that, for every $\eta<\eta^{\prime}$, a distribution ${\cal D}$ may be have a minimal $\eta$-effective support size that is much larger than its minimal $\eta^{\prime}$-effective support size (e.g., consider ${\cal D}$ that assigns a total probability mass of $1-\eta$ to very few elements and is otherwise uniform on a huge set). Hence, on input $\eta$, our goal is to approximate the $\Theta(\eta)$-effective support size of ${\cal D}$; say, output a number between the minimal $2\eta$-effective support size of ${\cal D}$ and its minimal $\eta$-effective support size (or a good approximation of such a number).

A simple approximation of the effective support size.

We first present a simple algorithm for obtaining a rather rough (but sufficiently good for our purposes) approximation. Given an “effectiveness” parameter $\eta$, we proceed in iterations such that in the $i^{\rm th}$ iteration we take a sample of $m=O(\eta^{-1}\log i)$ elements of ${\cal D}$, and halt outputting $2^{i}/\eta$ if the number of samples that have ${\cal D}$-value below $\eta/2^{i}$ is at most $3\eta\cdot m$. Observer that (in iteration $i$), with probability at least $1-0.01/i^{2}$, the sample approximates the total weight of the “light elements” (i.e., elements having ${\cal D}$-value below $\eta/2^{i}$) in the sense that if ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<\eta/2^{i}]<2\eta$, (resp., if ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<\eta/2^{i}]\geq 4\eta$), then the number of samples that have ${\cal D}$-value below $\eta/2^{i}$ is at most $3\eta\cdot m$ (resp., is greater than $3\eta\cdot m$).

Now, letting $n$ be an upper bound on the $\eta$-effective support size of ${\cal D}$ (and assuming for simplicity that $n$ is a power of two), observe that, with high (constant) probability, we halt by iteration $i^{*}=\log_{2}n$, because ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<\eta/2^{i^{*}}]<n\cdot\eta/2^{i^{*}}+\eta=2\eta$, where the first (resp., second) term is due to elements that are (resp., are not) in the effective support of ${\cal D}$. Hence, with high (constant) probability, the algorithm outputs a value that is at most $n/\eta$. (Actually, if we reach iteration $i^{*}$, we halt in it with probability at least $1-0.01/(i^{*})^{2}$; it follows that with probability at $1-0.01/\log^{2}n$, the algorithm outputs a value that is at most $n/\eta$.)

On the other hand, assuming that the minimal $4\eta$-effective support size of ${\cal D}$ is at least $n^{\prime}$ (and assuming that $n^{\prime}$ is also a power of two), with high (constant) probability, we do not halt before iteration $i^{+}=\log_{2}(\eta\cdot n^{\prime})$, because otherwise ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<\eta/2^{i^{+}-1}]<4\eta$, which implies that ${\cal D}$ has $4\eta$-effective support size at most $n^{\prime}/2$ (since ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<2/n^{\prime}]<4\eta$ implies that ${\cal D}$ is $4\eta$-close to a distribution of support size at most $n^{\prime}/2$). Hence, with high (constant) probability (i.e., with probability at least $1-\sum_{i<i^{+}}0.01/i^{2}>0.98$), the algorithm outputs a value that is at least $n^{\prime}$.

It follows that, with high (constant) probability, the algorithm outputs a number that lies between $n^{\prime}$ and $n/\eta$ (i.e., between the half the minimal $4\eta$-effective support size of ${\cal D}$ and $2/\eta$ times its minimal $\eta$-effective support size). Furthermore, with high (constant) probability, this algorithm runs for $\sum_{i\leq i^{*}}O(\eta^{-1}\cdot\log i)={\widetilde{O}}(\log n)/\eta$ steps (and its expected number of steps can be similarly bounded). Combining this result with Theorem 2.1, we essentially infer Theorem 1.4 (except that $\epsilon/5$ is replaced by $\epsilon/16$).

Obtaining better approximations of the effective support size.

For starters, we present a tighter analysis of (a minor variant of) the foregoing algorithm. Specifically, for any constant $\beta>1$, in the $i^{\rm th}$ iteration, we take a sample of $m=O(\eta^{-1}\log i)$ elements of ${\cal D}$, and halt outputting $2^{i}/\eta$ if the number of samples that have ${\cal D}$-value below $(\beta-1)\cdot\eta/2^{i}$ is at most $\beta^{2}\cdot\eta\cdot m$. In analyzing the probability that this algorithm halts by iteration $i^{*}=\log_{2}n$, we use the fact that ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<(\beta-1)\cdot\eta/2^{i^{*}}]<n\cdot(\beta-1)\cdot\eta/2^{i^{*}}+\eta=\beta\cdot\eta$, whereas in analyzing the probability that the algorithm does halts before iteration $i^{+}=\log_{2}(\eta\cdot n^{\prime})$ implies ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<(\beta-1)\cdot\eta/2^{i^{+}-1}]<\beta^{3}\cdot\eta$ (where here $n^{\prime}$ is the minimal $\beta^{3}\eta$-effective support size).999Here we use the fact that (in iteration $i$), with probability at least $1-0.01/i^{2}$, the sample approximates the total weight of the “light elements” (i.e., elements having ${\cal D}$-value below $\eta/2^{i}$) in the sense that if ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<(\beta-1)\cdot\eta/2^{i}]<\beta\cdot\eta$, (resp., if ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<(\beta-1)\cdot\eta/2^{i}]\geq\beta^{3}\cdot\eta$), then the number of samples that have ${\cal D}$-value below $(\beta-1)\cdot\eta/2^{i}$ is at most $\beta^{2}\cdot\eta\cdot m$ (resp., is greater than $\beta^{2}\cdot\eta\cdot m$). It follows that, with high (constant) probability, the algorithm outputs a number, denoted ${\widetilde{n}}$, that lies between half the minimal $\beta^{3}\cdot\eta$-effective support size of ${\cal D}$ and $2/\eta$ times its minimal $\eta$-effective support size.

To obtain an even better approximation of the effective support size, we use the rough estimate ${\widetilde{n}}$ obtained above in order to approximate the number of elements that have ${\cal D}$-value approximately $\beta^{i-0.5}$ for every $i\in[O(\log{\widetilde{n}}/\eta)]$. Indeed, our first step is ignoring elements having ${\cal D}$-value below $\eta/{\widetilde{n}}$ or so. Specifically, setting $\eta^{\prime}=\beta^{3}\cdot\eta$, recall that if ${\cal D}$ has an $\eta^{\prime}$-effective support of size ${\widetilde{n}}$, then ${\cal D}(H)\geq 1-\beta\eta^{\prime}$ for $H\stackrel{{\scriptstyle\rm def}}{{=}}\{v:{\cal D}(v)\geq(\beta-1)\cdot\eta^{\prime}/{\widetilde{n}}\}$ (see prior paragraph, while replacing $i^{*}$ by $\log_{2}{\widetilde{n}}$).101010In other words, observe that ${\rm Pr}_{v\leftarrow{\cal D}}[{\cal D}(v)<(\beta-1)\cdot\eta^{\prime}/{\widetilde{n}}]<{\widetilde{n}}\cdot(\beta-1)\cdot\eta^{\prime}/{\widetilde{n}}+\eta^{\prime}=\beta\cdot\eta^{\prime}$, where the first (resp., second) term is due to elements that are (resp., are not) in the effective support of ${\cal D}$. Hence, assuming that ${\cal D}(H)\leq 1-\eta$ and letting $\eta^{\prime\prime}=\beta\cdot\eta^{\prime}=\beta^{4}\cdot\eta$, it holds that $|H|$ lies between the minimal $\eta^{\prime\prime}$-effective support size of ${\cal D}$ and its minimal $\eta$-effective support size, and so providing a good approximation of the “effective size” of $H$ will do. (In the case that ${\cal D}(H)>1-\eta$ additional steps will be needed.)

To (effectively) approximate $|H|$, we let $W_{i}=\{v:\beta^{i-1}\leq{\cal D}(v)<\beta^{i}\}$, and observe that it suffice the approximate ${\cal D}(W_{i})$ for $i=1,...,\ell$, where $\ell\stackrel{{\scriptstyle\rm def}}{{=}}\log_{\beta}({\widetilde{n}}/(\beta-1)\cdot\eta^{\prime})=O((\beta-1)^{-1}\cdot\log({\widetilde{n}}/\eta))$. Actually, letting $I=\{i\in[\ell]:{\cal D}(W_{i})\geq(\beta-1)\eta^{\prime\prime}/\ell\}$, it suffices to approximate ${\cal D}(W_{i})$ for every $i\in I$, which yields approximations of the corresponding $|W_{i}|$’s (using $|W_{i}|\approx{\cal D}(W_{i})/\beta^{i-0.5}$). That is, we do not actaully approximate $|H|$ but rather approximate the size of $H^{\prime}\stackrel{{\scriptstyle\rm def}}{{=}}\bigcup_{i\in I}W_{i}\subseteq H$, while capitalizing on ${\cal D}(H\setminus H^{\prime})\leq(\beta-1)\cdot\eta^{\prime\prime}$. Hence, we will approximate the minimal $\eta^{\prime\prime\prime}$-support size for some $\eta\leq\eta^{\prime\prime\prime}\leq\beta\eta^{\prime\prime}=\beta^{5}\eta$. Specifically, for each $i\in[\ell]$, using a sample of $O(t\ell/(\beta-1)^{2}\cdot\eta^{\prime})$ samples, we obtain (with probability $1-2^{-t}$) a $\beta$-factor approximation of ${\cal D}(W_{i})$ for each $i\in I$, which yields a $\beta^{2}$-factor approximation of $|W_{i}|$ (since $|W_{i}|\in[\beta^{i-1}{\cal D}(W_{i}),\beta^{i}\cdot{\cal D}(W_{i}))$). Note that the rough estimate of the effective support size of ${\cal D}$ (i.e., ${\widetilde{n}}$) is only used in order to determine $\ell$.

Recall that we have assumed that ${\cal D}(H)\leq 1-\eta$, whereas this is not necessarily the case. Needless to say, we can easily estimate $1-{\cal D}(H)$ up to any desired constant factor (using $O(1/\eta)$ samples of ${\cal D}$). In case we are quite sure that ${\cal D}(H)>1-\eta$ (which will happen if ${\cal D}(H)>1-\beta\eta$ but not if ${\cal D}(H)<1-\beta^{-1}\eta$), we can reduce the estimate obtained for $|H|$ by disposing an adequate weight of $H$; that is, we dispose as many as the the lightest elements as possible till reaching a subset of $H^{\prime}$ that has ${\cal D}$-value that is most likely to be below $1-\eta$. Note that the foregoing process is performed without making any samples or queries; it is solely based on the estimated values of ${\cal D}(W_{i})$ for $i\in I$ already obtained. Hence, we get.

Theorem 2.2

(approximating the effective support size):*

There exists an oracle machine $M$ that, on input $\eta$ and $\beta>1$, satisfies the following condition for every distribution ${\cal D}$. Given oracle access to ${{\tt samp}_{\cal D}}$ and ${{\tt eval}_{\cal D}}$, with probability at least $2/3$, the machine outputs a number $n$ after making ${\rm poly}(1/(1-\beta))\cdot{\widetilde{O}}(\eta^{-1}\log n)$ steps such that $n$ is at least the minimal $\beta^{5}\cdot\eta$-effective support size of ${\cal D}$ and is at most $\beta^{2}$ times the minimal $\beta^{-1}\cdot\eta$-effective support size of ${\cal D}$. Furthermore, the expected number of steps performed by $M$ is ${\rm poly}(1/(1-\beta))\cdot{\widetilde{O}}(\eta^{-1}\log n)$.*

Needless to say, by a change in parameters we can make $n$ lie between the minimal $\beta\cdot\eta$-effective support size of ${\cal D}$ and $\beta$ times its minimal $\eta$-effective support size. Combining Theorems 2.1 and 2.2, we infer Theorem 1.4.

The approximation algorithm of Theorem 2.2 provides a rather good approximation of the effective support size, but its complexity depends (mildly) on the effective support size. We wonder whether it is possible to obtain a meaningful approximation of the effective support size within complexity that is independent of it. For starters, can approximations as in Theorem 2.2 be obtained within complexity ${\rm poly}(1/\eta)$? In a forthcoming paper, we farther explore the complexity of approximating the effective support size of distributions. We present several algorithms that exhibit a trade-off between the quality of the approximation and the complexity of obtaining it, and leave open the question of their optimality.

3 The Cycle-freeness Tester

Following [2], we reduce testing cycle-freeness to testing bipartiteness, where in both cases we refer to VDF testing in the bounded degree graph model. While the reduction is identical to the one presented in [2] (for the standard bounded-degree model), our analysis presented in Lemma 3.1 addresses issues that arise only in the VDF model.

Specifically, we use the presentation provided in [2, Sec. 8.1] rather than the one provided in [2, Sec. 3]. The pivot of this presentation is the following generalization of 2-colorability in which edges of the graph are labeled by either $\tt eq$ or $\tt neq$. That is, an instance of this problem is a graph $G=(V,E)$ along with a labeling $\tau:E\to\{{\tt eq},{\tt neq}\}$. We say that $\chi:V\to{\{0,1\}}$ is a legal 2-coloring of this instance if for every $\{u,v\}\in E$ it holds that $\chi(u)=\chi(v)$ if and only if $\tau(\{u,v\})={\tt eq}$. That is, a legal 2-coloring (of the vertices) is one in which every two vertices that are connected by an edge labeled $\tt eq$ (resp., $\tt neq$) are assigned the same color (resp., opposite colors).

Note that the standard notion of 2-colorability corresponds to the case in which all edges are labeled $\tt neq$. We first observe that the Bipartite tester provided in Section 2 can be extended to test this generalization of 2-colorability. This is the case because the (one-sided error) Bipartite tester of [7] (as well as the ones of [8] and [5]) can be extended to test this generalization of 2-colorability.

Next, as in [2, Sec. 8.1], we randomly reduce testing cycle-freeness of a graph $G=(V,E)$ to testing generalized 2-coloring of the instance obtained by coupling $G$ with a uniformly selected labeling $\tau:E\to\{{\tt eq},{\tt neq}\}$, which may be selected on the fly (i.e., whenever we encounter a new edge, we assign it a random label). Clearly, if $G$ is cycle-free, then, for any choice of $\tau:E\to\{{\tt eq},{\tt neq}\}$, the instance $(G,\tau)$ has a legal 2-coloring. We conjecture that, like in [2, Lem. 3.1], it holds that if $G$ is $\epsilon$-far from being cycle-free (under the distribution ${\cal D}$), then in expectation the random instance $(G,\tau)$ is $\Omega(\epsilon)$-far from having a legal 2-coloring (under the distribution ${\cal D}$). We only prove a weaker result, which suffices for our application (since ${\widetilde{O}}({\sqrt{n}})\cdot{\rm poly}(\Omega(\epsilon/\log n)^{-1})={\widetilde{O}}({\sqrt{n}})\cdot{\rm poly}(\epsilon^{-1})$).

Lemma 3.1

(analysis of the randomized reduction):*

Let $G=(V,E)$ be a simple graph (i.e., $G$ has neither parallel edges nor self-loops)111111Alternatively, we consider a pair parallel edges (resp., a self-loop) as constituting a cycle. and ${\cal D}$ be a distribution on $V$. If $G$ is $\epsilon$-far from being cycle-free according to distribution ${\cal D}$, then, with probability at least $\Omega(1)$ over the random choice of $\tau:E\to\{{\tt eq},{\tt neq}\}$, the instance $(G,\tau)$ is $\Omega(\epsilon/\log|V|)$-far from having a 2-legal coloring.*

Hence, we reduce testing cycle-freeness of $n$-vertex graphs with respect to the proximity parameter $\epsilon$ and vertex-distribution ${\cal D}$ to testing generalized 2-coloring of $n$-vertex graphs with respect to the proximity parameter $\Omega(\epsilon/\log n)$ (and vertex-distribution ${\cal D}$). An analogous assertion holds for testing cycle-freeneess with respect to arbitrary graphs and distribution ${\cal D}$ that have $\epsilon/5$-effective support size $n$. Hence, combining Lemma 3.1 and Theorem 1.4, we obtain Theorem 1.5.

**Proof:  **Maintaining ${\cal D}$ unchanged, we first omit from $G$ all edges that have probability weight smaller than $\epsilon/d|V|$ (i.e., we omit the edge $\{u,v\}$ if $2\cdot\frac{{\cal D}(u)+{\cal D}(v)}{d}<\epsilon/d|V|$). Denoting the resulting graph by $G^{\prime}=(V,E^{\prime})$, observe that $G^{\prime}$ is $\epsilon/2$-far from being cycle-free according to distribution ${\cal D}$. Next, we consider a spanning forest of maximal weight of $G^{\prime}$, denote its edges by $F^{\prime}$, and note that the weight of the edges in $E^{\prime}\setminus F^{\prime}$ is at least $\epsilon/4$, because $G^{\prime}$ can be made cycle-free by omitting the edges $E^{\prime}\setminus F^{\prime}$. We partition the edges $E^{\prime}\setminus F^{\prime}$ to buckets according to their weight, letting the bucket $B_{i}\subseteq E^{\prime}\setminus F^{\prime}$ contain the edges of weight in $(2^{-i},2^{-i+1}]$. Letting $\ell={\lfloor{\log_{2}(d|V|/\epsilon)}\rfloor}+1$, we observe that $E^{\prime}\setminus F^{\prime}=\bigcup_{i\in[\ell]}B_{i}$, and it follows that there exists $i\in[\ell]$ such that the total weight of the edges in $B_{i}$ is at least $\epsilon/4\ell$.

Fixing such an $i\in[\ell]$ and focusing on the graph $G^{\prime\prime}=(V,F^{\prime}\cup B_{i})$, we consider the two-connected components of $G^{\prime\prime}$ (i.e., the maximal subgraphs of $G^{\prime\prime}$ in which every two vertices are connected by at least two edge-disjoint paths). We observe that edges that connect such components must be in $F^{\prime}$, by considering two cases.

In the first case, these components are not connected in $G^{\prime}$, and in this case there are also not connected in $G^{\prime\prime}$ (and the claim holds vacuously). Otherwise (which is the second case), these components are connected in $G^{\prime}$ and so there must be a path of edges of $F^{\prime}$ that connects (a pair of vertices in) them, since $F^{\prime}$ spans the connected components of $G^{\prime}$. In fact, in this case, any pair of vertices in these two components is connected by a path of edges in $F^{\prime}$. But, then having an edge of $B_{i}$ that connects these components yields a contradiction to their definition (as two-connected components of $G^{\prime\prime}$), since it yields two enge-disjoint paths between the endpoints of this edge (one being the edge itself and the other being a path of edges in $F^{\prime}$).

Lastly, we omit the edges of $F^{\prime}$ that connect different two-connected components, and obtain a graph $G^{\prime\prime\prime}$ in which each connected component is two-connected.

We next claim that each edge in $G^{\prime\prime\prime}=(V^{\prime\prime\prime},E^{\prime\prime\prime})$ has weight at least $2^{-i}$. Since $E^{\prime\prime\prime}\subseteq F^{\prime}\cup B_{i}$, we have to establish this claim only for edges in $E^{\prime\prime\prime}\cap F^{\prime}$. We first observe that the edges in $E^{\prime\prime\prime}\cap F^{\prime}$ induce a spanning tree of each connected component of $G^{\prime\prime\prime}$, because otherwise we reach a contradiction to their definition (as two-connected components of $G^{\prime\prime}$).121212Specifically, vertices in a component of $G^{\prime\prime\prime}$ that are not connected by edges of $F^{\prime}$ must be connected in $G^{\prime\prime}$ by an edge-disjoint path of edges of $F^{\prime}$ that contain vertices that are not in this component. But this contradicts the hypothesis that this (two-connected) component of $G^{\prime\prime\prime}$ is a two-connected component of $G^{\prime\prime}$ (i.e., it contradicts its maximality in $G^{\prime\prime}$). Suppose towards the contradiction that one of the edges of such a spanning tree has weight smaller than $2^{-i}$. Then, omitting this edge and adding an edge of $B_{i}$ that connects the two resulting sub-trees, where such an edge must exist by two-connectivity of the component, we obtain a spanning tree of this component that has a larger weight than the original one. This yields a spanning forest of $G^{\prime}$ that has weight larger than the forest $F^{\prime}$, in contradiction to the definition of $F^{\prime}$.

Finally, we apply [2, Lem. 3.1] to $G^{\prime\prime\prime}$, while noting that $E^{\prime\prime\prime}\setminus F^{\prime}=B_{i}$ and $|B_{i}|\geq\frac{\epsilon/4\ell}{2^{-i+1}}=\Omega(2^{i}\cdot\epsilon/\ell)$, it follows that (with probability at least $\Omega(1)$ over the choice of $\tau$) at least $\Omega(|B_{i}|)$ edges must be omitted from $G^{\prime\prime\prime}$ such that the resulting instance $(\cdot,\tau)$ has a legal 2-coloring. Recalling that each edge in $G^{\prime\prime\prime}$ have weight at least $2^{-i}$, it follows that the weight of edges that must be omitted from $G^{\prime\prime\prime}$ in order to yield an instance $(\cdot,\tau)$ that has a legal 2-coloring is at least $\Omega(|B_{i}|\cdot 2^{-i})=\Omega(\epsilon/\ell)$. The same holds with respect to $G$, since $G^{\prime\prime\prime}$ is a subgraph of $G$, and the claim follows.

Acknowledgments

This project was partially supported by the Israel Science Foundation (grant No. 1146/18), and has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 819702).