Separation of AC$^0[\oplus]$ Formulas and Circuits

TL;DR

This paper establishes a fundamental separation between formulas and circuits of equal depth in AC0[⊕], demonstrating that certain polynomial-size depth-d circuits cannot be simulated by smaller formulas, with implications for approximate degree and circuit complexity.

Contribution

It provides the first known separation between AC0[⊕] formulas and circuits of the same depth, using new bounds for Approximate Majority functions.

Findings

Polynomial-size depth-d circuits are not equivalent to smaller formulas for certain functions.

AC0[⊕] formula lower bounds via polynomial approximation over GF(2).

Construction of monotone AC0 circuits approximating Majority functions.

Abstract

This paper gives the first separation between the power of {\em formulas} and {\em circuits} of equal depth in the $\mathrm{AC}^0[\oplus]$ basis (unbounded fan-in AND, OR, NOT and MOD$_2$ gates). We show, for all $d(n) \le O(\frac{\log n}{\log\log n})$, that there exist {\em polynomial-size depth-$d$ circuits} that are not equivalent to {\em depth-$d$ formulas of size $n^{o(d)}$} (moreover, this is optimal in that $n^{o(d)}$ cannot be improved to $n^{O(d)}$). This result is obtained by a combination of new lower and upper bounds for {\em Approximate Majorities}, the class of Boolean functions $\{0,1\}^n \to \{0,1\}$ that agree with the Majority function on $3/4$ fraction of inputs. $\mathrm{AC}^0[\oplus]$ formula lower bound: We show that every depth-$d$ $\mathrm{AC}^0[\oplus]$ formula of size $s$ has a {\em $1/8$-error polynomial approximation} over $\mathbb{F}_2$ of degree $O(\frac{1}{d}\log s)^{d-1}$. This strengthens a classic $O(\log s)^{d-1}$ degree approximation for \underline{circuits} due to Razborov. Since the Majority function has approximate degree $\Theta(\sqrt n)$, this result implies an $\exp(\Omega(dn^{1/2(d-1)}))$ lower bound on the depth-$d$ $\mathrm{AC}^0[\oplus]$ formula size of all Approximate Majority functions for all $d(n) \le O(\log n)$. Monotone $\mathrm{AC}^0$ circuit upper bound: For all $d(n) \le O(\frac{\log n}{\log\log n})$, we give a randomized construction of depth-$d$ monotone $\mathrm{AC}^0$ circuits (without NOT or MOD$_2$ gates) of size $\exp(O(n^{1/2(d-1)}))$ that compute an Approximate Majority function. This strengthens a construction of \underline{formulas} of size $\exp(O(dn^{1/2(d-1)}))$ due to Amano.

Equations

$${\mathrm{Pr}}_{x\in \{0,1{\}}^n}[f(x)=P(x)]\ge 1-\epsilon .$$

$$3c_1\cdot 5(\log (s/s_i)+1)\cdot (c_2(\frac{1}{d}\log (s_i)+1){)}^d.$$

$$60c_1\cdot \underset{i}{\max }(\log (s/s_i)+1)\cdot (c_2(\frac{1}{d}\log (s_i)+1){)}^d.$$

$$(\log (s/s_i)+1)\cdot (\frac{1}{d}\log (s_i)+1{)}^d\le (\frac{1}{d+1}\log (s)+1{)}^{d+1}.$$

$$(b+1){\left(\frac{a}{d}+1\right)}^d\le {\left(\frac{a+b}{d+1}+1\right)}^{d+1}.$$

$$p_{a,d}(b):={\left(\frac{a+b}{d+1}+1\right)}^{d+1}-(b+1){\left(\frac{a}{d}+1\right)}^d$$

$$\frac{\partial }{\partial b}{p}_{a,d}(b)={\left(\frac{a+b}{d+1}+1\right)}^d-{\left(\frac{a}{d}+1\right)}^d,$$

$$s\ge \exp (\Omega (d{n}^{1/2(d-1)})-O(d)).$$

$$x\in I_0(\gamma )$$

$$x\in I_1(\gamma )$$

$${\mathrm{Pr}}_S[\underset{j\in S}{\bigvee }{x}_j=0]$$

$$\le \exp (-(1+\gamma )\cdot s)\le \exp (-s)\cdot \exp (-s\gamma ).$$

$${\mathrm{Pr}}_S[\underset{j\in S}{\bigvee }{x}_j=0]$$

$$\ge \exp \left((-\frac{1-\gamma }{e^A}-\frac{1}{e^{2A}})\cdot (e^A\cdot s+1)\right)$$

$$=\exp \left(-s+s\gamma -\frac{s}{e^A}-\frac{1-\gamma }{e^A}-\frac{1}{e^{2A}}\right)$$

$$\ge \exp \left(-s+s\gamma -\frac{2s}{e^A}\right)$$

$$=\exp (-s)\cdot \exp (s\gamma (1-\frac{2e^{-A}}{\gamma })))$$

$$1-\frac{2e^{-A}}{\gamma }$$

$$\ge \exp (-1/n)\ge \exp (-\gamma )\ge \exp (-s\gamma ).$$

$${\gamma }_0$$

$${\gamma }_i$$

$$\ge A^i{\gamma }_0\exp (-2{\gamma }_{0}A(1+A+A^2+\cdots +A^{i-1}))\ge A^i{\gamma }_0\exp (-3A^i{\gamma }_0).$$

$$Y_{\epsilon }$$

$$N_{\epsilon }$$

$$x\in N_{\epsilon }$$

$$x\in Y_{\epsilon }$$

$$x\in N_{\epsilon }$$

$$x\in Y_{\epsilon }$$

$$\mathrm{Pr}[b_i^x=1]$$

$$\frac{{\sum }_i{b}_i^x}{M}$$

$$\le \frac{1}{e^A}(1-{\gamma }_{0}A+\delta )=\frac{1}{e^A}(1-{\gamma }_{0}A(1-\frac{\delta }{{\gamma }_{0}A}))$$

$$\le \frac{1}{e^A}(1-{\gamma }_{0}A\exp (-\frac{2\delta }{{\gamma }_{0}A}))\le \frac{1}{e^A}(1-{\gamma }_{0}A\exp (-{\gamma }_{0}A))$$

$$\le \frac{1}{e^A}(1-{\gamma }_1).$$

$$\mathrm{Pr}[b_i^x=1]$$

$$\ge \frac{1}{2^A}{\left(1+2{\gamma }_0\right)}^A\ge \frac{1}{e^A}(1+{\gamma }_{0}A)$$

$$\ge \frac{1}{e^A}(1+{\gamma }_{0}A).$$

$$\frac{{\sum }_i{b}_i^x}{M}$$

$$\ge \frac{1}{e^A}(1+{\gamma }_{0}A-2\delta )=\frac{1}{e^A}(1+{\gamma }_{0}A(1-\frac{2\delta }{{\gamma }_{0}A}))$$

Full text

AC${}^{\textbf{0}}$[$\boldsymbol{\oplus}$] formula lower bound. We show that every depth-$d$ $\mathrm{AC}^{0}[\oplus]$ formula of size $s$ has a $1/8$-error polynomial approximation over $\mathbb{F}_{2}$ of degree $O(\frac{1}{d}\log s)^{d-1}$. This strengthens a classic $O(\log s)^{d-1}$ degree approximation for circuits due to Razborov [12]. Since the Majority function has approximate degree $\Theta(\sqrt{n})$, this result implies an $\exp(\Omega(dn^{1/2(d-1)}))$ lower bound on the depth-$d$ $\mathrm{AC}^{0}[\oplus]$ formula size of all Approximate Majority functions for all $d(n)\leq O(\log n)$.

Monotone AC${}^{\textbf{0}}$ circuit upper bound. For all $d(n)\leq O({\frac{\log n}{\log\log n}})$, we give a randomized construction of depth-$d$ monotone $\smash{\mathrm{AC}^{0}}$ circuits (without NOT or MOD2 gates) of size $\smash{\exp(O(n^{1/2(d-1)}))}$ that compute an Approximate Majority function. This strengthens a construction of formulas of size $\exp(O(dn^{1/2(d-1)}))$ due to Amano [1].

1 Introduction

The relative power of formulas versus circuits is one of the great mysteries in complexity theory. The central question in this area is whether NC1 (the class of languages decidable by polynomial-size Boolean formulas) is a proper subclass of P/poly (the class of languages decidable by polynomial-size Boolean circuits). Despite decades of efforts, this question remains wide open.222In this paper we focus on non-uniform complexity classes. The question of uniform-NC1 vs. P is wide open as well. In the meantime, there has been progress on analogues of the NC1 vs. P/poly question in certain restricted settings. For instance, in the monotone basis (with AND and OR gates only), the power of polynomial-size formulas vs. circuits was separated by the classic lower bound of Karchmer and Wigderson [8] (on the monotone formula size of st-Connectivity).

The bounded-depth setting is another natural venue for investigating the question of formula vs. circuits. Consider the elementary fact that every depth-$d$ circuit of size $s$ is equivalent to a depth-$d$ formula of size at most $s^{d-1}$, where we measure size by the number of gates. This observation is valid with respect to any basis (i.e. set of gate types). In particular, we may consider the $\mathrm{AC}^{0}$ basis (unbounded fan-in AND, OR, NOT gates) and the $\mathrm{AC}^{0}[\oplus]$ basis (unbounded fan-in MOD2 gates in addition to AND, OR, NOT gates). With respect to either basis, there is a natural depth-$d$ analogue of the NC1 vs. P/poly question (where $d=d(n)$ is a parameter that may depend on $n$), namely whether every language decidable by polynomial-size depth-$d$ circuits is decidable by depth-$d$ formulas of size $n^{o(d)}$ (i.e. better than the trivial $n^{O(d)}$ upper bound).

It is reasonable to expect that this question could be resolved in the sub-logarithmic depth regime ($d(n)\ll\log n$), given the powerful lower bound techniques against $\mathrm{AC}^{0}$ circuits (Håstad’s Switching Lemma [5]) and $\mathrm{AC}^{0}[\oplus]$ circuits (the Polynomial Method of Razborov [12] and Smolensky [15]). However, because the standard way of applying these techniques does not distinguish between circuits and formulas, it is not clear how to prove quantitatively stronger lower bounds on formula size vis-a-vis circuit size of a given function. Recent work of Rossman [13] developed a new way of applying Håstad’s Switching Lemma to $\mathrm{AC}^{0}$ formulas, in order to prove an $\exp(\Omega(dn^{1/(d-1)}))$ lower bound on the formula size of the Parity function for all $d\leq O(\log n)$. Combined with the well-known $\exp(O(n^{1/(d-1)}))$ upper bound on the circuit size of Parity, this yields an asymptotically optimal separation in the power of depth-$d$ $\mathrm{AC}^{0}$ formulas vs. circuits for all $d(n)\leq O(\frac{\log n}{\log\log n})$, as well as a super-polynomial separation for all $\omega(1)\leq d(n)\leq o(\log n)$.

In the present paper, we carry out a similar development for formulas vs. circuits in the $\mathrm{AC}^{0}[\oplus]$ basis, obtaining both an asymptotically optimal separation for all $d(n)\leq O(\frac{\log n}{\log\log n})$ and a super-polynomial separation for all $\omega(1)\leq d(n)\leq o(\log n)$. Our target functions lie in the class of Approximate Majorities, here defined as Boolean functions $\{0,1\}^{n}\to\{0,1\}$ that approximate the Majority function on $3/4$ fraction of inputs. First, we show how to apply the Polynomial Method to obtain better parameters in the approximation of $\mathrm{AC}^{0}[\oplus]$ formulas by low-degree polynomials over $\mathbb{F}_{2}$. This leads to an $\exp(\Omega(dn^{1/2(d-1)}))$ lower bound on the $\mathrm{AC}^{0}[\oplus]$ formula size of all Approximate Majority functions. The other half of our formulas vs. circuits separation comes from an $\exp(O(n^{1/2(d-1)}))$ upper bound on the $\mathrm{AC}^{0}[\oplus]$ circuit size of some Approximate Majority function. In fact, this upper bound is realized by a randomized construction of monotone $\mathrm{AC}^{0}$ circuits (without NOT or MOD2 gates). Together these upper and lower bound give our main result:

Theorem 1**.**

1. (i)

For all $2\leq d(n)\leq O(\frac{\log n}{\log\log n})$, there exist $\mathrm{AC}^{0}[\oplus]$ circuits (in fact, monotone $\mathrm{AC}^{0}$ circuits) of depth $d$ and size $\mathrm{poly}(n)$ that are not equivalent to any $\mathrm{AC}^{0}[\oplus]$ formulas of depth $d$ and size $n^{o(d)}$. 2. (ii)

For all $\omega(1)\leq d(n)\leq o(\log n)$, the class of languages decidable by polynomial-size depth-$d$* $\mathrm{AC}^{0}[\oplus]$ formulas is a proper subclass of the class of languages decidable by polynomial-size depth-$d$ $\mathrm{AC}^{0}[\oplus]$ circuits.*

Separation (i) is asymptotically optimal, in view of the aforementioned simulation of $\mathrm{poly}(n)$-size depth-$d$ circuits by depth-$d$ formulas of size $n^{O(d)}$. Separation (ii) resembles an analogue of NC1 $\neq$ P/poly (or rather NC1 $\neq$ AC1) within the class $\mathrm{AC}^{0}[\oplus]$. In fact, extending separation (ii) from depth $o(\log n)$ to depth $\log n$ is equivalent the separation of NC1 and AC1.

1.1 Proof outline

Improved polynomial approximation.

The lower bound for $\mathrm{AC}^{0}[\oplus]$ formulas follows the general template due to Razborov [12] on proving lower bounds for $\mathrm{AC}^{0}[\oplus]$ circuits using low-degree polynomials over $\mathbb{F}_{2}$. Razborov showed that for any Boolean function $f:\{0,1\}^{n}\rightarrow\{0,1\}$ that has an $\mathrm{AC}^{0}[\oplus]$ circuit of size $s$ and depth $d$, there is a randomized polynomial $\mathbf{P}$ of degree $O(\log s)^{d-1}$ that computes $f$ correctly on each input with probability $\frac{7}{8}$ (we call such polynomials $1/8$-error probabilistic polynomials). By showing that some explicit Boolean function $f$ (e.g. the Majority function or the $\mathrm{MOD}_{q}$ function for $q$ odd) on $n$ variables does not have such an approximation of degree less than $\Omega(\sqrt{n})$ [12, 15, 16], we get that any $\mathrm{AC}^{0}[\oplus]$ circuit of depth $d$ computing $f$ must have size $\exp(\Omega(n^{1/2(d-1)}))$.

In this paper, we improve the parameters of Razborov’s polynomial approximation from above for $\mathrm{AC}^{0}[\oplus]$ formulas. More precisely, for $\mathrm{AC}^{0}[\oplus]$ formulas of size $s$ and depth $d$, we are able to construct $1/8$-error probabilistic polynomials of degree $O(\frac{1}{d}\log s)^{d-1}$. (Since every depth-$d$ circuit of size $s$ is equivalent to a depth-$d$ formula of size at most $s^{d-1}$, this result implies Razborov’s original theorem that $\mathrm{AC}^{0}[\oplus]$ circuits of size $s$ and depth $d$ have $1/8$-error probabilistic polynomials of degree $O(\log s)^{d-1}$.)

We illustrate the idea behind this improved polynomial approximation with the special case of a balanced formula (i.e. all gates have the same fan-in) of fan-in $t$ and depth $d$. Note that the size of the formula (number of gates) is $\Theta(t^{d-1})$ and hence it suffices in this case to show that it has a $1/8$-error probabilistic polynomial of degree $O(\log t)^{d-1}$. We construct the probabilistic polynomial inductively. Given a balanced formula $F$ of depth $d$ and fan-in $t$, let $F_{1},\ldots,F_{t}$ be its subformulas of depth $d-1$. Inductively, each $F_{i}$ has a $1/8$-error probabilistic polynomial ${\mathbf{P}}_{i}$ of degree $O(\log t)^{d-2}$ and by a standard error-reduction [10], it has a $(1/16t)$-error probabilistic polynomial of degree $O(\log t)^{d-1}$ (in particular, at any given input $x\in\{0,1\}^{n}$, the probability that there exists an $i\in[t]$ such that ${\mathbf{P}}_{i}(x)\neq F_{i}(x)$ is at most $1/16$). Using Razborov’s construction of a $1/16$-error probabilistic polynomial of degree $O(1)$ for the output gate of $F$ and composing this with the probabilistic polynomials ${\mathbf{P}}_{i}$, we get the result for balanced formulas. This idea can be extended to general (i.e. not necessarily balanced) formulas with a careful choice of the error parameter for each subformula $F_{i}$ to obtain the stronger polynomial approximation result.

Improved formula lower bounds.

Combining the above approximation result with known lower bounds for polynomial approximation [12, 15, 16], we can already obtain stronger lower bounds for $\mathrm{AC}^{0}[\oplus]$ formulas than are known for $\mathrm{AC}^{0}[\oplus]$ circuits. For instance, it follows that any $\mathrm{AC}^{0}[\oplus]$ formula of depth $d$ computing the Majority function on $n$ variables must have size $\exp(\Omega(dn^{1/2(d-1)}))$ for all $d\leq O(\log n)$, which is stronger than the corresponding circuit lower bound. Similarly stronger formula lower bounds also follow for the $\mathrm{MOD}_{q}$ function ($q$ odd).

Separation between formulas and circuits.

However, the above improved lower bounds do not directly yield the claimed separation between $\mathrm{AC}^{0}[\oplus]$ formulas and circuits. This is because we do not have circuits computing (say) the Majority function of the required size. To be able to prove our result, we would need to show that the Majority function has $\mathrm{AC}^{0}[\oplus]$ circuits of depth $d$ and size $\exp(O(n^{1/2(d-1)}))$ (where the constant in the $O(\cdot)$ is independent of $d$). However, as far as we know, the strongest result in this direction [9] only yields $\mathrm{AC}^{0}[\oplus]$ circuits of size greater than $\exp(\Omega(n^{1/(d-1)})),$333Indeed, this is inevitable with all constructions that we are aware of, since they are actually $\mathrm{AC}^{0}$ circuits and it is known by a result of Håstad [5] that any $\mathrm{AC}^{0}$ circuit of depth $d$ for the Majority function must have size $\exp(\Omega(n^{1/(d-1)})).$ which is superpolynomially larger than the upper bound.

To circumvent this issue, we change the hard functions to the class of Approximate Majorities, which is the class of Boolean functions that agree with Majority function on most inputs. While this has the downside that we no longer are dealing with an explicitly defined function, the advantage is that the polynomial approximation method of Razborov yields tight lower bounds for some functions from this class.

Indeed, since the method of Razborov is based on polynomial approximations, it immediately follows that the same proof technique also yields the same lower bound for computing Approximate Majorities. Formally, any $\mathrm{AC}^{0}[\oplus]$ circuit of depth $d$ computing any Approximate Majority must have size $\exp(\Omega(n^{1/2(d-1)}))$. On the upper bound side, it is known from the work of O’Donnell and Wimmer [11] and Amano [1] that there exist Approximate Majorities that can be computed by monotone $\mathrm{AC}^{0}$ formulas of depth $d$ and size $\exp(O(dn^{1/2(d-1)}))$. (Note that the double exponent $\frac{1}{2(d-1)}$ is now the same in the upper and lower bounds.)

We use the above ideas for our separation between $\mathrm{AC}^{0}[\oplus]$ formulas and circuits. Plugging in our stronger polynomial approximation for $\mathrm{AC}^{0}[\oplus]$ formulas, we obtain that any $\mathrm{AC}^{0}[\oplus]$ formula of depth $d$ computing any Approximate Majority must have size $\exp(\Omega(dn^{1/2(d-1)})).$ In particular, this implies that Amano’s construction is tight (up to the universal constant in the exponent) even for $\mathrm{AC}^{0}[\oplus]$ formulas.

Further, we also modify Amano’s construction [1] to obtain better constant-depth circuits for Approximate Majorities: we show that there exist Approximate Majorities that are computed by monotone $\mathrm{AC}^{0}$ circuits of depth $d$ of size $\exp(O(n^{1/2(d-1)}))$ (the constant in the $O(\cdot)$ is a constant independent of $d$).

Smaller circuits for Approximate Majority.

Our construction closely follows Amano’s, which in turn is related to Valiant’s probabilistic construction [18] of monotone formulas for the Majority function. However, we need to modify the construction in a suitable way that exploits the fact that we are constructing circuits. This modification is in a similar spirit to a construction of Hoory, Magen and Pitassi [6] who modify Valiant’s construction to obtain smaller monotone circuits (of depth $\Theta(\log n)$) for computing the Majority function exactly.

At a high level, the difference between Amano’s construction and ours is as follows. Amano constructs random formulas $F_{i}$ of each depth $i\leq d$ as follows. The formula $F_{1}$ is the AND of $a_{1}$ independent and randomly chosen variables. For even (respectively odd) $i>1$, $F_{i}$ is the OR (respectively AND) of $a_{i}$ independent and random copies of $F_{i-1}$. For suitable values of $a_{1},\ldots,a_{d}\in\mathbb{N}$, the random formula $F_{d}$ computes an Approximate Majority with high probability. In our construction, we build a depth $i$ circuit $C_{i}$ for each $i\leq d$ in a similar way, except that each $C_{i}$ now has $M$ different outputs. Given such a $C_{i-1}$, we construct $C_{i}$ by taking $M$ independent randomly chosen subsets $T_{1},\ldots,T_{M}$ of $a_{i}$ many outputs of $C_{i-1}$ and adding gates that compute either the OR or AND (depending on whether $i$ is even or odd) of the gates in $T_{i}$. Any of the $M$ final gates of $C_{d}$ now serves as the output gate. By an analysis similar to Amano’s (see also [6]) we can show that this computes an Approximate Majority with high probability, which finishes the proof.444This is a slightly imprecise description of the construction as the final two levels of the circuit are actually defined somewhat differently.

2 Preliminaries

Throughout, $n$ will be a growing parameter. We will consider Boolean functions on $n$ variables, i.e. functions of the form $f:\{0,1\}^{n}\rightarrow\{0,1\}$. We will sometimes identify $\{0,1\}$ with the field $\mathbb{F}_{2}$ in the natural way and consider functions $f:\mathbb{F}_{2}^{n}\rightarrow\mathbb{F}_{2}$ instead.

Given a Boolean vector $y\in\{0,1\}^{n}$, we use $|y|_{0}$ and $|y|_{1}$ to denote the number of [math]s and number of $1$s respectively in $y$.

The Majority function on $n$ variables, denoted $\mathrm{MAJ}_{n}$ is the Boolean function that maps inputs $x\in\{0,1\}^{n}$ to $1$ if and only if $|x|_{1}>n/2$.

Definition 2**.**

An $(\varepsilon,n)$-Approximate Majority is a function $f:\{0,1\}^{n}\rightarrow\{0,1\}$ such that $\mathop{\mathrm{Pr}}_{x\in\{0,1\}^{n}}[f(x)\neq\mathrm{Maj}_{n}(x)]\leq\varepsilon$.

As far as we know, the study of this class of functions was initiated by O’Donnell and Wimmer [11]. See also [1, 4].

We refer the reader to [2, 7] for standard definitions of Boolean circuits and formulas. We use $\mathrm{AC}^{0}$ circuits (respectively formulas) to denote circuits (respectively formulas) of constant depth made up of $\mathrm{AND},\ \mathrm{OR}$ and $\mathrm{NOT}$ gates. Similarly, $\mathrm{AC}^{0}[\oplus]$ circuits (respectively formulas) will be circuits (respectively formulas) of constant depth made up of $\mathrm{AND},\ \mathrm{OR},\ \mathrm{MOD}_{2}$ and $\mathrm{NOT}$ gates.

The size of a circuit will denote the number of gates in the circuit and the size of a formula will denote the number of its leaves which is within a constant multiplicative factor of the number of gates in the formula.555We assume here without loss of generality that the formula does not contain a gate of fan-in $1$ feeding into another.

3 Lower Bound

In this section, we show that any $\mathrm{AC}^{0}[\oplus]$ formulas of depth $d$ computing a $(1/4,n)$-Approximate Majority must have size at least $\exp(\Omega(dn^{1/2(d-1)}))$ for all $d\leq O(\log n)$.

We work over the field $\mathbb{F}_{2}$ and identify it with $\{0,1\}$ in the natural way. The following concepts are standard in circuit complexity (see, e.g., Beigel’s survey [3]).

Definition 3**.**

Fix any $\varepsilon\in[0,1]$. A polynomial $P\in\mathbb{F}_{2}[X_{1},\ldots,X_{n}]$ is said to be an $\varepsilon$-approximating polynomial for a Boolean function $f:\{0,1\}^{n}\rightarrow\{0,1\}$ if

[TABLE]

We will use the following result of Smolensky [16] (see also Szegedy’s PhD thesis [17]).

Lemma 4** (Smolensky [16]).**

Let $\varepsilon\in(0,\frac{1}{2})$ be any fixed constant. Any $(\frac{1}{2}-\varepsilon)$-approximating polynomial for the Majority function on $n$ variables must have degree $\Omega(\sqrt{n})$.

Corollary 5**.**

Let $f$ be any $(1/4,n)$-Approximate Majority and $\varepsilon\in(0,1/4)$ an arbitrary constant. Then any $(\frac{1}{4}-\varepsilon)$-approximating polynomial for $f$ must have degree $\Omega(\sqrt{n})$.

Proof.

The proof is immediate from Lemma 4 and the triangle inequality. ∎

Definition 6**.**

An $\varepsilon$-error probabilistic polynomial of degree $D$ for a Boolean function $f:\{0,1\}^{n}\to\{0,1\}$ is a random variable $\mathbf{P}$ taking values from polynomials in $\mathbb{F}_{2}[X_{1},\ldots,X_{n}]$ of degree at most $D$ such that for all $x\in\{0,1\}^{n}$, we have $\Pr[\ f(x)=\mathbf{P}(x)\ ]\geq 1-\varepsilon$.

Definition 7**.**

Let $D_{\varepsilon}(f)$ be the minimum degree of an $\varepsilon$-error probabilistic polynomial for $f$.

We will make use of the following two lemmas concerning $D_{\varepsilon}(\cdot)$.

Lemma 8** (Razborov [12]).**

Let $\mathrm{OR}_{n}$ and $\mathrm{AND}_{n}$ be the OR and AND functions on $n$ variables respectively. Then $D_{\varepsilon}(\mathrm{OR}_{n}),D_{\varepsilon}(\mathrm{AND}_{n})\leq\lceil\log(1/\varepsilon)\rceil$.

Lemma 9** (Kopparty and Srinivasan [10]).**

There is an absolute constant $c_{1}$ such that for any $\varepsilon\in(0,1)$, $D_{\varepsilon}(f)\leq c_{1}\cdot\lceil\log(1/\varepsilon)\rceil\cdot D_{1/8}(f)$ for all Boolean functions $f$.

We now state our main result, which shows that every $\mathrm{AC}^{0}[\oplus]$ formula of size $s$ and depth $d+1$ admits a $1/8$-error approximating polynomial of degree $O(\frac{1}{d}\log s)^{d}$.

Theorem 10**.**

There is an absolute constant $c_{2}$ such that, if $f$ is computed by an $\mathrm{AC}^{0}[\oplus]$ formula $F$ of size $s$ and depth $d+1$, then $D_{1/8}(f)\leq 3(c_{2}({\textstyle\frac{1}{d}}\log(s)+1))^{d}$.

Proof.

The proof is an induction on the depth $d$ of the formula.

The base case $d=0$ corresponds to the case when the formula is a single AND, OR or MOD2 gate and we need to show that $D_{1/8}(f)\leq 3$. In the case that the formula is an AND or OR gate, this follows from Lemma 8. If the formula is a MOD2 gate, this follows from the fact that the MOD2 function is exactly a polynomial of degree $1$.

Let $d\geq 1$. We assume that the formula $F$ is the AND/OR/MOD2 of sub-formulas $F_{1},\ldots,F_{m}$ computing $f_{1},\dots,f_{m}$ where $F_{i}$ has size $s_{i}$ and depth $d+1$. So $F$ has size $s=s_{1}+\dots+s_{m}$ and depth $d+2$. Assume that $D_{1/8}(f_{i})\leq 3(c_{2}(\frac{1}{d}\log(s_{i})+1))^{d}$ for all $i$. We must show that $D_{1/8}(f)\leq 3(c_{2}(\frac{1}{d+1}\log(s)+1))^{d+1}$.

By Lemma 9, each $f_{i}$ has an $s_{i}/(16s)$-error probabilistic polynomial $\mathbf{P}_{i}$ of degree $c_{1}\cdot\lceil\log(16s/s_{i})\rceil\cdot D_{1/8}(f_{i})$, which is at most

[TABLE]

Then $(\mathbf{P}_{1},\dots,\mathbf{P}_{m})$ jointly computes $(f_{1},\dots,f_{m})$ with error $1/16$ ($=\sum_{i=1}^{m}(s_{i}/(16s))$).

By a reasoning identical to the base case, it follows that there exists a $1/16$-error probabilistic polynomial $\mathbf{Q}$ of degree $4$ for the output gate of the formula.

Then $\mathbf{Q}(\mathbf{P}_{1},\dots,\mathbf{P}_{m})$ is a $1/8$-error probabilistic polynomial for $f$ of degree

[TABLE]

So long as $c_{2}\geq 20c_{1}$, it suffices to show that for all $i$,

[TABLE]

Consider any $i$ and let $a,b\geq 0$ such that $s_{i}=2^{a}$ and $s=2^{a+b}$. We must show

[TABLE]

For fixed $a\geq 0$, as a polynomial in $b$, the function

[TABLE]

is nonnegative over $b\geq 0$ with a unique root at $b=a/d$. This follows from

[TABLE]

which is zero iff $b=a/d$; this value is a minimum of $p_{a,d}$ with $p_{a,d}(a/d)=0$. ∎

Corollary 11**.**

Fix any constant $d$ and let $n\in\mathbb{N}$ be a growing parameter. Let $f$ be any $(1/4,n)$-Approximate Majority. Then any $\mathrm{AC}^{0}[\oplus]$ formula of depth $d$ computing $f$ must have size $\exp(\Omega(dn^{1/2(d-1)}))$ for all $d\leq O(\log n)$, where asymptotic notation $O(\cdot)$ and $\Omega(\cdot)$ hide absolute constants (independent of $d$ and $n$).

Proof.

Say that $F$ is an $\mathrm{AC}^{0}[\oplus]$ formula of depth $d$ and size $s$ computing $f$. Then, by Lemma 9, we see that $F$ has a $1/8$-error probabistic polynomial $\mathbf{P}$ of degree $D\leq O(O(\frac{1}{d}\log s+1)^{d-1})$. In particular, by an averaging argument, there is some fixed polynomial $P\in\mathbb{F}_{2}[X_{1},\ldots,X_{n}]$ of degree at most $D$ such that $P$ is a $1/8$-error approximating polynomial for $f$.

Corollary 5 implies that the degree of $P$ must be $\Omega(\sqrt{n})$. Hence, we obtain $O(\frac{1}{d}\log s+1)^{d-1}\geq\Omega(\sqrt{n})$. It follows that

[TABLE]

Observe that $\Omega(dn^{1/2(d-1)})$ dominates $O(d)$ so long as $d\leq\varepsilon\log n$ for some absolute constant $\varepsilon>0$ (depending on the constants in $\Omega(\cdot)$ and $O(\cdot)$). Hence, we get the claimed lower bound $s\geq\exp(\Omega(dn^{1/2(d-1)}))$ for all $d\leq\varepsilon\log n$. ∎

4 Upper Bound

In this section, we show that for any constant $\varepsilon$, there are $(\varepsilon,n)$-Approximate Majorities that can be computed by depth $d$ $\mathrm{AC}^{0}$ circuits of size $\exp(O(n^{1/2(d-1)}))$.

Let $\varepsilon_{0}\in(0,1)$ be a small enough constant so that the following inequalities hold for any $\beta\leq\varepsilon_{0}$

• •

$\exp(-\beta)\leq 1-\beta\exp(-\beta)$,

• •

$1-\beta\geq\exp(-\beta-\beta^{2})\geq\exp(-2\beta)$.

(It suffices to take $\varepsilon_{0}=1/2$.)

We need the following technical lemma.

Lemma 12**.**

Let $A,s$ be positive reals, $M,n\in\mathbb{N}$, and $\gamma\in(\frac{1}{n},\frac{1}{10})$ be such that $e^{A}\geq n^{3}$, $n\geq\frac{1}{\varepsilon_{0}}$, and $s\leq n$. Define $I_{0}(\gamma):=\{y\in\{0,1\}^{M}\ |\ |y|_{1}\leq Me^{-A}(1-\gamma)\}$ and $I_{1}(\gamma):=\{y\in\{0,1\}^{M}\ |\ |y|_{1}\geq Me^{-A}(1+\gamma)\}$. If we choose $S\subseteq[M]$ of size $t:=\lceil e^{A}\cdot s\rceil$ by picking $t$ random elements from $M$ with replacement, then

[TABLE]

Further, if $s\gamma\leq\varepsilon_{0}$, then the above probabilities can be lower bounded and upper bounded by $\exp(-s)\cdot(1+s\gamma\exp(-s\gamma))$ and $\exp(-s)\cdot(1-s\gamma\exp(-s\gamma))$ respectively.

A similar statement can be obtained above for the sets $J_{1}(\gamma):=\{y\in\{0,1\}^{M}\ |\ |y|_{0}\leq Me^{-A}(1-\gamma)\}$ and $J_{0}(\gamma):=\{y\in\{0,1\}^{M}\ |\ |y|_{0}\geq Me^{-A}(1+\gamma)\}$, with the event “$\bigvee_{j\in S}x_{j}=0$” being replaced by the event “$\bigwedge_{j\in S}x_{j}=1$”.

Proof.

We give the proof only for $I_{0}(\gamma)$ and $I_{1}(\gamma)$. The proof for $J_{0}(\gamma)$ and $J_{1}(\gamma)$ is similar.

Consider first the case that $x\in I_{1}(\gamma)$. In this case, we have the following computation.

[TABLE]

The above implies the first upper bound on $\mathop{\mathrm{Pr}}_{S}[\bigvee_{j\in S}x_{j}=0]$ from the lemma statement. When $s\gamma\leq\varepsilon_{0}$, we further have $\exp(-s\gamma)\leq 1-s\gamma\exp(-s\gamma)$, which implies the second upper bound. This proves the lemma when $x\in I_{1}$.

Now consider the case that $x\in I_{0}(\gamma)$. We have

[TABLE]

where for the second inequality we have used the fact that since $e^{-A}\leq\frac{1}{n^{3}}\leq\varepsilon_{0}$, we have $1-\frac{1-\gamma}{e^{A}}\geq\exp(-\frac{1-\gamma}{e^{A}}-\frac{1}{e^{2A}})$. Since $e^{-A}\leq\frac{1}{n^{3}}\leq\frac{1}{4n}\leq\gamma/4$, we can lower bound the right hand side of (2) by $\exp(-s)\cdot\exp(s\gamma/2)$. Also, note that

[TABLE]

This implies that the RHS of (2) can also be lower bounded by $\exp(-s)\exp(s\gamma\exp(-s\gamma))\geq\exp(-s)\cdot(1+s\gamma\exp(-s\gamma))$, which implies the claim about $\mathop{\mathrm{Pr}}_{S}[\bigvee_{j\in S}x_{j}=0]$ assuming that $x\in I_{0}(\gamma)$. ∎

We now prove the main result of this section.

Theorem 13**.**

For any growing parameter $n\in\mathbb{N}$ and $2\leq d\leq O(\frac{\log n}{\log\log n})$ and $\varepsilon>0$, there is an $(\varepsilon,n)$-Approximate Majority $f_{n}$ computable by a monotone $\mathrm{AC}^{0}$ circuit with at most $\exp(O(n^{1/2(d-1)}\log(1/\varepsilon)/\varepsilon))$ many gates, where both $O(\cdot)$’s hide absolute constants (independent of $d,\varepsilon$).

Proof.

We assume throughout that $\varepsilon$ is a small enough constant and that $n$ is large enough for various inequalities to hold. We will actually construct a monotone circuit of depth $d$ and size $\exp(O(n^{1/2(d-1)}\log(1/\varepsilon)/\varepsilon))$ computing a $(4\varepsilon,n)$-Approximate Majority, which also implies the theorem.

Fix parameters $A=\lfloor n^{1/2(d-1)}\rfloor$ and $M=\lceil e^{10A}\rceil$. We assume that $A\geq 10\log n$ (which holds as long as $d\leq\frac{c\log n}{\log\log n}$ for an absolute constant $c>0$) and that $\varepsilon\leq\varepsilon_{0}$.

Define a sequence of real numbers $\gamma_{0},\gamma_{1},\ldots,\gamma_{d-2}$ as follows:

[TABLE]

It is clear that $\gamma_{i}\leq A^{i}\gamma_{0}$ for each $i\in[d-2]$. As a result we also obtain

[TABLE]

Let

[TABLE]

The idea is to define a sequence of circuits $C_{1},C_{2},\ldots,C_{d-2}$ with $n$ inputs and $M$ outputs such that $C_{i}$ has depth $i$ and $iM$ many (non-input) gates. Further, for odd $i$

[TABLE]

and similarly for even $i$

[TABLE]

After this is done, we will add on top a depth-$2$ circuit that will reject most inputs from $I_{0}(\gamma_{d-2})$ or $J_{0}(\gamma_{d-2})$ — depending on whether $d-2$ is odd or even respectively — and accept most inputs from $I_{1}(\gamma_{d-2})$ or $J_{1}(\gamma_{d-2})$.

We begin with the construction of $C_{1},\ldots,C_{d-2}$ which is done by induction.

Construction of $C_{1}$.

The base case of the induction is the construction of $C_{1}$, which is done as follows. We choose $M$ i.i.d. random subsets $T_{1},\ldots,T_{M}\subseteq[n]$ in the following way: for each $i\in[M]$, we sample $A$ random elements of $[n]$ with replacement. Let $b_{i}^{x}=\bigwedge_{j\in T_{i}}x_{j}$.

If $x\in N_{\varepsilon}$, then the probability that $b_{i}^{x}=1$ is given by

[TABLE]

where the last inequality follows from the fact that $(1-z)^{A}\leq(1-zA+\frac{A^{2}z^{2}}{2})$.

Let $\delta=1/n^{3}$. Note in particular that $2\delta/\gamma_{0}A\leq\varepsilon_{0}$ for large enough $n$.

By a Chernoff bound, the probability that $\frac{1}{M}\sum_{i}b_{i}^{x}\geq\frac{1}{e^{A}}(1-\gamma_{0}A)(1+\delta)$ is bounded by $\exp(-\Omega(\delta^{2}M/e^{A}))\leq\exp(-\Omega(e^{9A}/n^{6}))\leq\exp(-n)$, since $e^{A}\geq n^{10}$. Thus, with probability at least $1-\exp(-n)$, we have

[TABLE]

Above, we have used the fact that $(1-\frac{\delta}{\gamma_{0}A})\geq\exp(\frac{-2\delta}{\gamma_{0}A})$ since $\delta/\gamma_{0}A\leq\varepsilon_{0}$ for large enough $n$, as noted above.

If $x\in Y_{\varepsilon}$, then the probability that $b_{i}^{x}=1$ is given by

[TABLE]

As above, we can argue that the probability that $\frac{1}{M}\sum_{i}b_{i}^{x}\leq\frac{1}{e^{A}}(1+\gamma_{0}A)(1-\delta)$ is at most $\exp(-n)$. Thus, with probability $1-\exp(-n)$

[TABLE]

Thus, by a union bound over $x$, we can fix a choice of $T_{1},\ldots,T_{M}$ so that (6) holds for all $x\in N_{\varepsilon}$ and (7) holds for all $x\in Y_{\varepsilon}$. Hence, (4) holds for $i=1$ as required. This concludes the construction of $C_{1}$, which just outputs the values of $\bigwedge_{j\in T_{i}}x_{j}$ for each $i$.

Construction of $C_{i+1}$.

For the inductive case, we proceed as follows. We assume that $i$ is odd (the case that $i$ is even is similar). So by the inductive hypothesis, we know that (4) holds and hence that $C_{i}(x)\in I_{0}(\gamma_{i})$ or $I_{1}(\gamma_{i})$ depending on whether $x\in N_{\varepsilon}$ or $Y_{\varepsilon}$. Let $\gamma:=\gamma_{i}$. Let the output gates of $C_{i}$ be $g_{1},\ldots,g_{M}$.

We choose $T_{1},\ldots,T_{M}\subseteq[M]$ randomly as in the statement of Lemma 12 with $s=A$. Note that the chosen parameters satisfy all the hypotheses of Lemma 12. Further we also have $s\gamma\leq A\cdot A^{i}\gamma_{0}\leq A^{d-1}\cdot\frac{\varepsilon}{\sqrt{n}}\leq\varepsilon_{0}$.

The random circuit $C^{\prime}$ is defined to be the circuit obtained by adding $M$ OR gates to $C_{i}$ such that the $j$th OR gate computes $\bigvee_{k\in T_{j}}g_{k}$. Let $b_{j}^{x}$ be the output of the $j$th OR gate on $C_{i}(x)$.

By Lemma 12, we have

[TABLE]

Let $\delta=\frac{1}{n^{3}}$. Note that $A\gamma\in[\frac{1}{\sqrt{n}},\frac{1}{n^{1/2(d-1)}}]$ and hence for large enough $n$, $\frac{2\delta}{A\gamma\exp(-A\gamma)}\leq\varepsilon_{0}$.

Assume $x\in N_{\varepsilon}$. In this case, the Chernoff bound implies that the probability that $\sum_{j\in[M]}b_{j}^{x}\leq M\exp(-A)\cdot(1+A\gamma\exp(-A\gamma))(1-\delta)$ is at most $\exp(-\Omega(\delta^{2}M/e^{A}))\leq\exp(-n)$. When this event does not occur, we have

[TABLE]

We have used above that for large enough $n$, $\frac{2\delta}{A\gamma\exp(-A\gamma)}\leq\varepsilon_{0}$ and hence $1-\frac{2\delta}{A\gamma\exp(-A\gamma)}\geq\exp(\frac{-4\delta}{A\gamma\exp(-A\gamma)})$.

Similarly when $x\in Y_{\varepsilon}$, the Chernoff bound tells us that the probability that $\sum_{j\in[M]}b_{j}^{x}\geq M\exp(-A)\cdot(1-A\gamma\exp(-A\gamma))(1+\delta)$ is at most $\exp(-n)$. In this case, we get

[TABLE]

By a union bound, we can fix $T_{1},\ldots,T_{M}$ so that (9) and (10) are true for all $x\in N_{\varepsilon}$ and $x\in Y_{\varepsilon}$ respectively. This gives us the circuit $C_{i+1}$ which satisfies all the required properties.

The top two levels of the circuit.

At the end of the above procedure we have a circuit $C_{d-2}$ of depth $d-2$ and at most $(d-2)M$ gates that satisfies one of (4) or (5) depending on whether $d-2$ is odd or even respectively. We assume that $d-2$ is even (the other case is similar).

Define $\gamma:=\gamma_{d-2}$. Recall from (3) that $\gamma\geq A^{d-2}\gamma_{0}\exp(-3A^{d-2}\gamma_{0})\geq A^{d-2}\gamma_{0}/2$.

Let $M^{\prime}=\lceil\exp(\frac{10A\log(1/\varepsilon)}{\varepsilon}+10A)\rceil$. We choose $M^{\prime}$ many subsets $T_{1},\ldots,T_{M^{\prime}}\subseteq[M]$ i.i.d. so that each $T_{j}$ is picked as in Lemma 12 with $s=10A\log(1/\varepsilon)/\varepsilon$. Note that

[TABLE]

Say $g_{1},\ldots,g_{M}$ are the output gates of $C_{d-2}$. We define the random circuit $C^{\prime}$ (with $n$ inputs and $M^{\prime}$ outputs) to be the circuit obtained by adding $M^{\prime}$ AND gates such that the $j$th AND gate computes $\bigwedge_{k\in T_{j}}g_{k}$. Let $b_{j}^{x}$ be the output of the $j$th AND gate on $C_{d-2}(x)$.

By Lemma 12, we have

[TABLE]

Say $x\in N_{\varepsilon}$. By a Chernoff bound, the probability that $\sum_{j}b_{j}^{x}\geq 2\varepsilon^{2}M^{\prime}\exp(-s)$ is at most $\exp(-\Omega(\varepsilon^{2}M^{\prime}\exp(-s)))\leq\exp(-\Omega(\varepsilon^{2}e^{10A}))\leq\exp(-n)$. Similarly, when $x\in Y_{\varepsilon}$, the probability that $\sum_{j}b_{j}^{x}\leq\frac{M^{\prime}\exp(-s)}{2\varepsilon^{2}}$ is also bounded by $\exp(-n)$. By a union bound, we can fix a $T_{1},\ldots,T_{M^{\prime}}$ to get a circuit $C_{d-1}$ such that

[TABLE]

This gives us the depth $d-1$ circuit $C_{d-1}$. Note that $C_{d-1}$ has $M^{\prime}+O(dM)=O(M^{\prime})$ gates.

To get the depth $d$ circuit, we choose a random subset $T\subseteq[M^{\prime}]$ by sampling exactly $\lceil\exp(s)\rceil$ many elements of $[M^{\prime}]$ with replacement. We construct a random depth-$d$ circuit $C^{\prime}_{d}$ by taking the OR of the the output gates of $C_{d-1}$ indexed by the subset $T$.

From (12) it follows that

[TABLE]

The final inequalities in each case above hold as long as $\varepsilon$ is a small enough constant.

It follows from the above that there is a choice for $T$ such that $C^{\prime}_{d}$ makes an error — i.e. $C^{\prime}_{d}(x)=1$ for $x\in N_{\varepsilon}$ or $C^{\prime}_{d}(x)=0$ for $x\in Y_{\varepsilon}$ — on at most a $2\varepsilon$ fraction of inputs from $N_{\varepsilon}\cup Y_{\varepsilon}$. We fix such a choice for $T$ and the corresponding circuit $C$.

We have

[TABLE]

Finally by Stirling’s approximation we get

[TABLE]

Hence we see that the circuit $C$ computes a $(4\varepsilon,n)$-Approximate Majority, which proves Theorem 13.

The circuit has depth $d$ and size $O(M^{\prime})=\exp(O(n^{1/2(d-1)}\log(1/\varepsilon)/\varepsilon))$. ∎

5 Conclusion

Our main results extend straightforwardly to $\mathrm{AC}^{0}[\mathrm{MOD}_{p}]$ for any fixed prime $p$. The proofs are exactly the same except for the fact that the approximating polynomials of degree $O(\frac{1}{d}\log s)^{d-1}$ from Section 3 are constructed over $\mathbb{F}_{p}$.

Using the fact [15] that any $(1/4)$-approximating polynomial over $\mathbb{F}_{p}$ ($p$ odd) for the Parity function on $n$ variables must have degree $\Omega(\sqrt{n})$, we see that any polynomial-sized $\mathrm{AC}^{0}[\mathrm{MOD}_{p}]$ formula computing the Parity function on $n$ variables must have depth $\Omega(\log n)$. This strengthens a result of Rossman [13] which gives this statement for $\mathrm{AC}^{0}$ formulas.

Acknowledgements.

We thank Rahul Santhanam for valuable discussions. We also thank the organizers of the 2016 Complexity Semester at St. Petersburg, where this collaboration began.