TL;DR
This paper constructs the hardest possible halfspace for polynomial and rational approximation, and applies this to achieve nearly optimal separations in communication complexity, advancing understanding of computational hardness.
Contribution
It explicitly constructs the hardest halfspace for approximation, matching trivial bounds, and uses this to improve communication complexity separations in multiple models.
Findings
Established polynomial and rational approximation lower bounds for the hardest halfspace.
Achieved near-optimal separation between sign-rank and discrepancy in communication complexity.
Extended results to the $k$-party number-on-the-forehead model, improving previous bounds.
Abstract
We study the approximation of halfspaces in the infinity norm by polynomials and rational functions of any given degree. Our main result is an explicit construction of the "hardest" halfspace, for which we prove polynomial and rational approximation lower bounds that match the trivial upper bounds achievable for all halfspaces. This completes a lengthy line of work started by Myhill and Kautz (1961). As an application, we construct a communication problem that achieves essentially the largest possible separation, of versus between the sign-rank and discrepancy. Equivalently, our problem exhibits a gap of versus between the communication complexity with unbounded versus weakly unbounded error, improving quadratically on previous constructions and completing a line of work started by Babai, Frankl, and Simon (FOCS…
Click any figure to enlarge with its caption.
Figure 1Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
The Hardest Halfspace· youtube
The Hardest Halfspace
Alexander A. Sherstov
Abstract.
We study the approximation of halfspaces in the infinity norm by polynomials and rational functions of any given degree. Our main result is an explicit construction of the “hardest” halfspace, for which we prove polynomial and rational approximation lower bounds that match the trivial upper bounds achievable for all halfspaces. This completes a lengthy line of work started by Myhill and Kautz (1961).
As an application, we construct a communication problem that achieves essentially the largest possible separation, of versus between the sign-rank and discrepancy. Equivalently, our problem exhibits a gap of versus between the communication complexity with unbounded versus weakly unbounded error, improving quadratically on previous constructions and completing a line of work started by Babai, Frankl, and Simon (FOCS 1986). Our results further generalize to the -party number-on-the-forehead model, where we obtain an explicit separation of versus for communication with unbounded versus weakly unbounded error. This gap is a quadratic improvement on previous work and matches the state of the art for number-on-the-forehead lower bounds.
∗ Computer Science Department, UCLA, Los Angeles, CA 90095. 🖂 [email protected] Supported by NSF CAREER award CCF-1149018 and an Alfred P. Sloan Foundation Research Fellowship.
Contents
1. Introduction
Representations of Boolean functions by real polynomials play a central role in theoretical computer science. The notion of approximating a Boolean function pointwise by polynomials of given degree has been particularly fruitful. Formally, let denote the minimum error in an infinity-norm approximation of by a real polynomial of degree at most :
[TABLE]
This quantity clearly ranges between [math] and for any function . In more detail, we have , where the first equality holds because any such is representable exactly by a polynomial of degree at most . The study of the polynomial approximation of Boolean functions dates back to the pioneering work in the 1960s by Myhill and Kautz [59] and Minsky and Papert [57]. This line of research has grown remarkably over the decades, with numerous connections discovered to other subjects in theoretical computer science. Lower bounds for polynomial approximation have complexity-theoretic applications, whereas upper bounds are a tool in algorithm design. In the former category, polynomial approximation has enabled significant progress in circuit complexity [17, 10, 48, 49, 73, 15], quantum query complexity [13, 1, 7, 23], and communication complexity [20, 65, 22, 73, 75, 66, 52, 26, 70, 15, 79, 78]. On the algorithmic side, polynomial approximation underlies many of the strongest results obtained to date in computational learning [82, 45, 44, 37, 61, 8], differentially private data release [84, 25], and algorithm design in general [55, 36, 72].
1.1. The hardest halfspace
Myhill and Kautz’s work [59] six decades ago, and many of the papers that followed [59, 58, 81, 62, 16, 33, 76, 77, 83], focused on halfspaces. Also known as a linear threshold function, a halfspace is any function representable as for some fixed reals The fundamental question taken up in this line of research is: how well can halfspaces be approximated by polynomials of given degree? An early finding, due to Muroga [58], was the upper bound
[TABLE]
for every halfspace in variables. In words, every halfspace can be approximated pointwise by a linear polynomial to error just barely smaller than the trivial bound of . Many authors pursued matching lower bounds on for specific halfspaces , culminating in an explicit construction by Håstad [33] that matches Muroga’s bound (1.1).
The study of for proved to be challenging. For a long time, essentially the only result was the lower bound due to Beigel [16], where is the so-called odd-max-bit halfspace*. Paturi [62] proved the incomparable lower bound , where is the majority function on bits. Much later, the bound was obtained in [76] for an explicit halfspace. This fragmented state of affairs persisted until the question was resolved completely in [77], with an existence proof of a halfspace such that for This result is clearly as strong as one could hope for, since it essentially matches Muroga’s upper bound for approximation by linear polynomials.* The work in [77] further determined the minimum error, denoted , to which this can be approximated by a degree- rational function, showing that this quantity too is as large for as it can be for any halfspace. Explicitly constructing a halfspace with these properties is our main technical contribution:
Theorem 1.1**.**
There is an algorithm that takes as input an integer runs in time polynomial in and outputs a halfspace with
[TABLE]
where is an absolute constant.
Classic bounds for the approximation of the sign function imply that for any the lower bounds in Theorem 1.1 are essentially the best possible for any halfspace on variables (see Sections 5.1 and 5.2 for details). Thus, the construction of Theorem 1.1 is the “hardest” halfspace from the point of view of approximation by polynomials and rational functions.
Theorem 1.1 is not a de-randomization of the existence proof in [77], which incidentally we are still unable to de-randomize. Rather, it is based on a new and simpler approach, presented in detail at the end of this section. Given the role that halfspaces play in theoretical computer science, we see Theorem 1.1 as answering a basic question of independent interest. In addition, Theorem 1.1 has applications to communication complexity and computational learning, which we now discuss.
1.2. Discrepancy vs. sign-rank
Consider the standard model of randomized communication [50], which features players Alice and Bob and a Boolean function On input Alice and Bob receive the arguments and respectively. Their objective is to compute on any given input with minimal communication. To this end, each player privately holds an unlimited supply of uniformly random bits which he or she can use in deciding what message to send at any given point in the protocol. The cost of a protocol is the total number of bits exchanged by Alice and Bob in a worst-case execution. The* -error randomized communication complexity of *, denoted , is the least cost of a protocol that computes with probability of error at most on every input.
Our interest in this paper is in communication protocols with error probability close to that of random guessing, There are two standard ways to define the complexity of a function in this setting, both inspired by probabilistic polynomial time for Turing machines [31]:
[TABLE]
and
[TABLE]
The former quantity, introduced by Paturi and Simon [63], is called the communication complexity of with unbounded error, in reference to the fact that the error probability can be arbitrarily close to The latter quantity, proposed by Babai et al. [11], includes an additional penalty term that depends on the error probability. We refer to as the communication complexity of with weakly unbounded error. For all functions one has the trivial bounds These two complexity measures give rise to corresponding complexity classes in communication complexity theory, defined in the seminal paper of Babai et al. [11]. Formally, is the class of families of communication problems whose unbounded-error communication complexity is at most polylogarithmic in Its counterpart is defined analogously for the complexity measure .
These two models of large-error communication are synonymous with two central notions in communication complexity: sign-rank and discrepancy, defined formally in Sections 2.8 and 2.9. In more detail, Paturi and Simon [63] proved that the communication complexity of any problem with unbounded error is characterized up to an additive constant by the sign-rank of its communication matrix, Analogously, Klauck [40, 41] showed that the communication complexity of any problem with weakly unbounded error is essentially characterized in terms of the discrepancy of . Discrepancy and sign-rank enjoy a rich mathematical life [54, 71, 74, 56] outside communication complexity, which further motivates the study of and as fundamental complexity classes.
Communication with weakly unbounded error is by definition no more powerful than unbounded-error communication, and for twenty years after the paper of Babai et al. [11] it was unknown whether this containment is proper. Buhrman et al. [22] and the author [71] answered this question in the affirmative, independently and with unrelated techniques. These papers exhibited functions with an exponential gap between communication complexity with unbounded error versus weakly unbounded error: in both works, versus in [22] and in [71]. In complexity-theoretic notation, these results show that . A simpler alternate proof of the result of Buhrman et al. [22] was given in [75] using the pattern matrix method. More recently, Thaler [83] exhibited another, remarkably simple communication problem with communication complexity and
To summarize, the strongest explicit separation of communication complexity with unbounded versus weakly unbounded error prior to our work was the separation of versus from twelve years ago [71]. The existence of a communication problem with a quadratically larger gap, of versus , follows from the work in [77]. This state of affairs parallels other instances in communication complexity, such as the versus question in multiparty communication [14], where the best existential separations are much stronger than the best explicit ones. There is considerable interest in communication complexity in explicit separations because they provide a deeper and more complete understanding of the complexity classes, whereas the lack of a strong explicit separation indicates a basic gap in our knowledge. As an application of Theorem 1.1, we obtain:
Theorem 1.2**.**
There is a communication problem defined by
[TABLE]
for some explicitly given reals such that
[TABLE]
Moreover,
[TABLE]
Theorem 1.2 gives essentially the strongest possible separation of the communication classes and , improving quadratically on previous constructions and matching the previous nonconstructive separation. Another compelling aspect of the theorem is the simple form (1.2) of the communication problem in question. The last two bounds in Theorem 1.2 state that has sign-rank at most and discrepancy , which is essentially the strongest possible separation. The best previous construction [71] achieved sign-rank and discrepancy .
We further generalize Theorem 1.2 to the number-on-the-forehead -party model, the standard formalism of multiparty communication. Analogous to two-party communication, the -party model has its own classes and of problems solvable efficiently by protocols with unbounded error and weakly unbounded error, respectively. Their formal definitions can be found in Section 2.8. In this setting, we prove:
Theorem 1.3**.**
There is a -party communication problem defined by
[TABLE]
for some explicitly given reals such that
[TABLE]
Theorem 1.3 gives essentially the strongest possible explicit separation of the -party communication complexity classes and for up to parties, where is an arbitrary constant. The previous best explicit separation [27, 80] of these classes was quadratically weaker, with communication complexity for unbounded error and for weakly unbounded error. The communication lower bound in Theorem 1.3 reflects the state of the art in the area, in that the strongest lower bound for any explicit communication problem to date is due to Babai et al. [12].
1.3. Computational learning
A sign-representing polynomial for a given function is any real polynomial such that for all The minimum degree of a sign-representing polynomial for is called the threshold degree of denoted Clearly for every Boolean function on variables. The reader can further verify that sign-representation is equivalent to pointwise approximation with error strictly less than, but arbitrarily close to, the trivial error of . Sign-representing polynomials are appealing from a learning standpoint because they immediately lead to efficient learning algorithms. Indeed, any function of threshold degree is by definition a linear combination of monomials and can thus be viewed as a halfspace in dimensions. As a result, can be PAC learned [86] under arbitrary distributions in time polynomial in using a variety of halfspace learning algorithms.
The study of sign-representing polynomials started fifty years ago with the seminal monograph of Minsky and Papert [57], who examined the threshold degree of several common functions. Since then, the threshold degree approach has yielded the fastest known PAC learning algorithms for notoriously hard concept classes, including DNF formulas [45] and AND-OR trees [8]. Conspicuously absent from this list of success stories is the concept class of intersections of halfspaces. While solutions are known to several restrictions of this learning problem [18, 51, 87, 9, 44, 46, 43], no algorithm has been discovered for PAC learning the intersection of even two halfspaces in time faster than Known hardness results, on the other hand, only apply to polynomially many halfspaces or to proper learning, e.g., [19, 3, 47, 39].
This state of affairs has motivated a quest to determine the threshold degree of the intersection of two halfspaces [57, 61, 42, 76, 77]. Prior to our work, the best lower bound was for an explicit intersection of two halfspaces [76], complemented by a tight but highly nonconstructive lower bound [77]. Using Theorem 1.1, we prove:
Theorem 1.4**.**
There is an explicitly given halfspace such that
[TABLE]
The symbol above stands for the intersection of two copies of on disjoint sets of variables. In other words, Theorem 1.4 constructs an explicit intersection of two halfspaces whose threshold degree is asymptotically maximal, While the nonconstructive lower bound of [77] already ruled out the threshold degree approach as a way to learn intersections of halfspaces, we see Theorem 1.4 as contributing a key qualitative piece of the puzzle. Specifically, it constructs a small and simple family of intersections of two halfspaces that are off-limits to all known algorithmic approaches (namely, the family obtained by applying to different subsets of the variables ).
1.4. Proof overview
Our solution has two main components: the construction of a sparse set of integers that appear random modulo and the univariatization of a multivariate Boolean function. We describe each of these components in detail.
Discrepancy of integer sets.
Let be a given integer. Key to our work is the notion of -discrepancy, which quantifies the pseudorandomness or aperiodicity modulo of any given multiset of integers. It is largely unrelated to the notion of discrepancy in communication complexity (Section 1.2). Formally, the -discrepancy of a nonempty multiset is defined as
[TABLE]
where is a primitive -th root of unity. This fundamental quantity arises in combinatorics and theoretical computer science, e.g., [30, 69, 2, 38, 64, 5]. The identity for any -th root of unity implies that the set achieves the smallest possible -discrepancy: Much sparser sets with small -discrepancy can be shown to exist using the probabilistic method (Fact 3.3 and Corollary 3.4). Specifically, one easily verifies for any constant the existence of a set with -discrepancy at most and cardinality an exponential improvement in sparsity compared to the trivial set We are aware of two efficient constructions of sparse sets with small -discrepancy, due to Ajtai et al. [2] and Katz [38]. The approach of Ajtai et al. is elementary except for an appeal to the prime number theorem, whereas Katz’s construction relies on deep results in number theory. Neither work appears to directly imply the kind of optimal de-randomization that we require, namely, an algorithm that runs in time polynomial in and produces a multiset of cardinality with -discrepancy bounded away from 1. We obtain such an algorithm by adapting the approach of Ajtai et al. [2].
The centerpiece of the construction of Ajtai et al. [2] is what the authors call the iteration lemma, stated in this paper as Theorem 3.6. Its role is to reduce the construction of a sparse set with small -discrepancy to the construction of sparse sets with small -discrepancy, for primes Ajtai et al. [2] proved their iteration lemma for prime, but we show that their argument readily generalizes to arbitrary moduli . By applying the iteration lemma in a recursive manner, one reaches smaller and smaller primes. The authors of [2] continue this recursive process until they reach primes so small that the trivial construction can be considered sparse. We proceed differently and terminate the recursion after just two stages, at which point the input size is small enough for brute force search based on the probabilistic method. The final set that we construct has size logarithmic in and -discrepancy a small constant, as opposed to the superlogarithmic size and discrepancy in the work of Ajtai et al. [2].
We note that this modified approach additionally gives the first explicit circulant expander on vertices of degree which is optimal and improves on the previous best degree bound of due to Ajtai et al. [2]. Background on circulant expanders, and the details of our expander construction, can be found in Section 5.6.
Univariatization.
We now describe the second major component of our proof. Consider a halfspace in Boolean variables where the coefficients can be assumed without loss of generality to be integers. Then the linear form ranges in the discrete set , for some integer proportionate to the magnitude of the coefficients. As a result, one can approximate to any given error by approximating the sign function to on This approach works for both rational approximation and polynomial approximation. We think of it as the black-box approach to the approximation of because it uses the linear form rather than the individual bits. There is no reason to expect that the black-box construction is anywhere close to optimal. Indeed, there are halfspaces [76, Section 1.3] that can be approximated to arbitrarily small error by a rational function of degree but require a black-box approximant of degree . Surprisingly, we are able to construct a halfspace with exponentially large coefficients for which the black-box approximant is essentially optimal. As a result, tight lower bounds for the rational and polynomial approximation of follow immediately from the univariate lower bounds for approximating the sign function on . The role of is to reduce the multivariate problem taken up in this work to a well-understood univariate question, hence the term univariatization.
The construction of involves several steps. First, we study the probability distribution of the weighted sum modulo , where are given integers and the bits are chosen uniformly at random. We show that the distribution is exponentially close to uniform whenever the multiset has -discrepancy bounded away from . For the next step, fix any multiset with small -discrepancy and consider the linear map given by At this point in the proof, we know that for uniformly random , the probability distribution of is exponentially close to uniform. This implies that the characteristic functions of have approximately the same Fourier spectrum up to degree , for some constant . We substantially strengthen this conclusion by proving that there are probability distributions , supported on , respectively, such that the Fourier spectra of are exactly the same up to degree Our proof relies on a general tool from [77, Theorem 4.1], proved there using the Gershgorin circle theorem.
As our final step, we use to construct a halfspace in terms of whose approximation by rational functions and polynomials gives corresponding approximants for the sign function on the discrete set . More generally, for any tuple , we define an associated halfspace and prove a lower bound on in terms of the discrepancy of the multiset Combining this result with the efficient construction of an integer set with small -discrepancy for , we obtain an explicit halfspace whose approximation by polynomials and rational functions is equivalent to the univariate approximation of the sign function on . Theorem 1.1 now follows by appealing to known lower bounds for the polynomial and rational approximation of the sign function. To obtain the exponential separation of communication complexity with unbounded versus weakly unbounded error (Theorem 1.2), we use the pattern matrix method [73, 75] to “lift” the lower bound of Theorem 1.1 to a discrepancy bound. Finally, our result on the threshold degree of the intersection of two halfspaces (Theorem 1.4) works by combining the rational approximation lower bound of Theorem 1.1 with a structural result from [76] on the sign-representation of arbitrary functions of the form
A key technical contribution of this paper is the identification of -discrepancy as a pseudorandom property that is weak enough to admit efficient de-randomization and strong enough to allow the univariatization of the corresponding halfspace. The previous, existential result in [77] used a completely different and more complicated pseudorandom property based on affine shifts of the Fourier transform on which we have not been able to de-randomize. Apart from the construction of a low-discrepancy set, our proof is simpler and more intuitive than the existential proof in [77].
2. Preliminaries
We start with a review of the technical preliminaries. The purpose of this section is to make the paper as self-contained as possible, and comfortably readable by a broad audience. The expert reader should therefore skim this section for notation or skip it altogether.
2.1. Notation
There are two common arithmetic encodings for the Boolean values: the traditional encoding and the Fourier-motivated encoding Throughout this manuscript, we use the former encoding for the domain of a Boolean function and the latter for the range. With this convention, Boolean functions are mappings for some For Boolean functions and we let denote the coordinatewise composition of with Formally, is given by
[TABLE]
where the linear map on the right-hand side serves the purpose of switching between the distinct arithmetizations for the domain versus range. A partial function on a set is a function whose domain of definition, denoted is a nonempty proper subset of We generalize coordinatewise composition to partial Boolean functions and in the natural way. Specifically, is the Boolean function given by (2.1), with domain the set of all inputs for which
We use the following two versions of the sign function:
[TABLE]
For a subset we let denote the restriction of the sign function to A *halfspace *for us is any Boolean function given by
[TABLE]
for some reals The majority function is the halfspace defined by
[TABLE]
Some authors define only for odd, in which case the tiebreaker term can be omitted.
The complement and the power set of a set are denoted as usual by and , respectively. The symmetric difference of sets and is Throughout this manuscript, we use brace notation as in to specify multisets rather than sets. The cardinality of a finite multiset is defined as the total number of element occurrences in , with each element counted as many times as it occurs. The equality and subset relations on multisets are defined analogously, with the number of element occurrences taken into account. For example, but . Similarly, but
The infinity norm of a function is denoted For real-valued functions and and a nonempty finite subset of their domain, we write
[TABLE]
We will often use this notation with a nonempty proper subset of the domain of and We let and stand for the natural logarithm of and the logarithm of to base respectively. The binary entropy function is given by and is strictly increasing on The following bound is well known [35, p. 283]:
[TABLE]
For a complex number we denote the real part, imaginary part, and complex conjugate of as usual by and respectively. We typeset the imaginary unit in boldface to distinguish it from the index variable .
For an arbitrary integer and a positive integer , recall that denotes the unique element of that is congruent to modulo For an integer the symbols and refer to the ring of integers modulo and the multiplicative group of integers modulo respectively. For a multiset of integers, we adopt the standard notation
[TABLE]
Note that the multisets in (2.3)–(2.6) each have cardinality the same as the original set . We often use these shorthands in combination, as in \text{(aZ+b)\bmod m}=\{(az_{1}+b)\bmod m,\ldots,(az_{n}+b)\bmod m\}.
For a logical condition we use the Iverson bracket
[TABLE]
The following concentration inequality, due to Hoeffding [34], is well-known.
Fact 2.1** (Hoeffding’s Inequality).**
Let be independent random variables with Let
[TABLE]
Then
[TABLE]
In Fact 2.1 and throughout this paper, we typeset random variables using capital letters.
2.2. Number-theoretic preliminaries
For positive integers and that are relatively prime, denotes the multiplicative inverse of modulo The following fact is well-known and straightforward to verify; cf. [2].
Fact 2.2**.**
For any positive integers and that are relatively prime,
[TABLE]
Proof.
We have and analogously Thus, is divisible by both and Since and are relatively prime, we conclude that is divisible by which is equivalent to (2.7). ∎
Recall that the prime counting function for a real argument evaluates to the number of prime numbers less than or equal to In what follows, it will be clear from the context whether refers to or the prime counting function. The asymptotic growth of the latter is given by the prime number theorem, which states that Many explicit bounds on are known, such as the following theorem of Rosser [68].
Fact 2.3** (Rosser).**
For
[TABLE]
The number of distinct prime divisors of a natural number is denoted . We will need the following first-principles bound on , which is asymptotically tight for infinitely many
Fact 2.4**.**
The number of distinct prime divisors of obeys
[TABLE]
In particular,
[TABLE]
Proof.
An integer has by definition distinct prime divisors. Letting denote the -th prime, we have
[TABLE]
where the second step uses the trivial estimate The second step in this derivation settles (2.8), whereas the last step settles (2.9). ∎
2.3. Matrix analysis
For an arbitrary set such as or the symbol denotes the family of matrices with entries in . The symbols and stand for the order- identity matrix and the matrix of all ones, respectively. When the dimensions of the matrix are clear from the context, we omit the subscripts and write simply or The shorthand refers to the diagonal matrix with entries on the diagonal:
[TABLE]
For a matrix recall that its complex conjugate is given by . The transpose and conjugate transpose of are denoted and respectively. The conjugation, transpose, and conjugate transpose operations apply as a special case to vectors, which we view as matrices with a single column. We use the familiar matrix norms and Again, these definitions carry over to vectors as a special case. A matrix is called unitary if
A circulant matrix is any matrix of the form
[TABLE]
for some Thus, every row of is obtained by a circular shift of the previous row one entry to the right. We let denote the right-hand side of (2.10). In this notation, and The eigenvalues and eigenvectors of a circulant matrix are well-known and straightforward to determine. For the reader’s convenience, we include the short derivation below in Fact 2.5 and Corollary 2.6.
Fact 2.5**.**
Let be a circulant matrix. Then for every -th root of unity the vector
[TABLE]
is an eigenvector of with eigenvalue
Proof.
Let denote the vector in (2.11). Then for
[TABLE]
where the third step uses ∎
As a corollary to Fact 2.5, one recovers the full complement of eigenvalues for any circulant matrix and furthermore learns that is unitarily similar to a diagonal matrix. In the statement below, recall that a primitive -th root of unity is any generator, such as for the multiplicative group of the roots of .
Corollary 2.6**.**
Let be a circulant matrix. Let be a primitive -th root of unity. Then the matrix
[TABLE]
is unitary and satisfies
[TABLE]
In particular, the eigenvalues of counting multiplicities, are
[TABLE]
Proof.
For , we have
[TABLE]
where the second step is valid because is primitive and in particular . We conclude that
[TABLE]
Fact 2.5 implies that
[TABLE]
which in light of (2.13) is equivalent to (2.12). ∎
2.4. Polynomial approximation
Recall that the total degree of a multivariate real polynomial , denoted is the largest degree of any monomial of We use the terms “degree” and “total degree” interchangeably in this paper. Let be a given function with domain For any define
[TABLE]
where the infimum is over real polynomials of degree at most In words, is the least error in a pointwise approximation of by a polynomial of degree no greater than The -approximate degree of is the minimum degree of a polynomial that approximates pointwise within :
[TABLE]
In this overview, we focus on the polynomial approximation of the sign function. We start with an elementary construction of an approximant due to Buhrman et al. [21].
Fact 2.7** (Buhrman et al.).**
For any and the sign function can be approximated on pointwise to within by a polynomial of degree
[TABLE]
The degree upper bound in Fact 2.7 is not tight. Indeed, a quadratically stronger bound of follows in a straightforward manner from Jackson’s theorem in approximation theory [67, Theorem 1.4]. Our applications do not benefit from this improvement, however, and we opt for the construction of Buhrman et al. [21] because of its striking simplicity. For the reader’s convenience, we provide their short proof below.
Proof (adapted from Buhrman et al.).
For a positive integer consider the degree- univariate polynomial
[TABLE]
In words, is the probability of observing at least as many heads as tails in a sequence of independent coin flips, each coming up heads with probability By Hoeffding’s inequality (Fact 2.1) for sufficiently large the polynomial sends and similarly As a result, the shifted and scaled polynomial approximates the sign function pointwise on within ∎
On the lower bounds side, Paturi proved that low-degree polynomials cannot approximate the majority function well. He in fact obtained analogous results for all symmetric functions, but the special case of majority will be sufficient for our purposes.
Theorem 2.8** (Paturi).**
For some constant and all integers
[TABLE]
The constant in Paturi’s theorem can be replaced by any other in His result is of interest to us because along with Fact 2.7, it implies a lower bound for the approximation of the sign function on the discrete set of points for any
Proposition 2.9**.**
For all positive integers and
[TABLE]
Proof.
Abbreviate and fix a polynomial of degree at most that approximates the sign function on within . Fact 2.7 gives a polynomial of degree that sends and Then the composition of these two approximants obeys
[TABLE]
This in turn gives an approximant for the majority function on bits:
[TABLE]
In view of Paturi’s lower bound for the majority function (Theorem 2.8), the approximant must have degree But this composition is a polynomial in of degree We conclude that whence ∎
2.5. Rational approximation
Consider a rational function where and are polynomials on We refer to the degrees of and as the numerator degree and denominator degree, respectively, of . The degree of is, then, the maximum of the numerator and denominator degrees. For a function with domain we define
[TABLE]
where the infimum is over multivariate polynomials and of degree at most and , respectively, such that does not vanish on In words, is the least error in an approximation of by a multivariate rational function with numerator degree and denominator degree at most and respectively. We will be mostly working with in the regimes and . In the former regime, we use the shorthand
[TABLE]
As a limiting case of the latter regime, we have
[TABLE]
The study of the rational approximation of the sign function dates back to the seminal work by Zolotarev [89] in the 1870s. The problem was revisited almost a century later by Newman [60], who proved the following result.
Fact 2.10** (Newman).**
For any and any integer
[TABLE]
For a recent exposition of Newman’s construction, we refer the reader to [76, Theorem 2.4]. As an important special case, Newman’s work gives upper bounds for the rational approximation of the sign function on the discrete set Newman’s upper bounds were sharpened and complemented with matching lower bounds in [76, Eq. (2.2) and Theorem 5.1], to the following effect.
Theorem 2.11** (Sherstov).**
For any positive integers and
[TABLE]
Among other things, Theorem 2.11 implies the following result on the rational approximation of the majority function [76, Eq. (2.2) and Theorems 5.1, 5.9].
Theorem 2.12** (Sherstov).**
For any positive integers and
[TABLE]
2.6. Sign-representation
Let be a given function, where is finite. The threshold degree of denoted is the least degree of a polynomial such that For functions and we let the symbol stand for the function given by Note that in this notation, and are completely different functions, the former having domain and the latter The following ingenious observation, due to Beigel et al. [17], relates the notions of sign-representation and rational approximation for conjunctions of Boolean functions.
Theorem 2.13** (Beigel et al.).**
Let and be given functions, where Let be any integer with
[TABLE]
Then
[TABLE]
Proof (adapted from Beigel et al.)..
Fix arbitrary rational functions and of degree at most such that
[TABLE]
Then
[TABLE]
Multiplying through by the positive quantity gives the desired sign-representing polynomial: ∎
The construction of Theorem 2.13 is somewhat ad hoc, and there is no particular reason to believe that it gives a sign-representing polynomial of asymptotically optimal degree. Remarkably, it does. The following converse to the theorem of Beigel et al. was established in [76, Theorem 3.16].
Theorem 2.14** (Sherstov).**
Let and be given functions, where are arbitrary finite sets. Assume that and are not identically false. Let Then
[TABLE]
2.7. Symmetrization
Let denote the symmetric group on elements. For and , we denote For we define A function is called symmetric if for every and every Equivalently, is symmetric if is uniquely determined by Symmetric functions on are intimately related to univariate polynomials, as borne out by Minsky and Papert’s symmetrization argument [57].
Proposition 2.15** (Minsky and Papert).**
Let be a polynomial of degree Then there is a univariate polynomial of degree at most such that for all
[TABLE]
Minsky and Papert’s result generalizes to block-symmetric functions, as pointed out in [66, Proposition 2.3]:
Proposition 2.16** (Razborov and Sherstov).**
Let be positive integers. Let be a polynomial of degree Then there is a polynomial of degree at most such that for all
[TABLE]
Proposition 2.16 follows in a straightforward manner from Minsky and Papert’s Proposition 2.15 by induction on the number of blocks
2.8. Communication complexity
An excellent reference on communication complexity is the monograph by Kushilevitz and Nisan [50]. In this overview, we will limit ourselves to key definitions and notation. We adopt the randomized number-on-the-forehead model, due to Chandra et al. [24]. The model features communicating players, tasked with computing a (possibly partial) Boolean function on the Cartesian product of some finite sets . A given input is distributed among the players by placing , figuratively speaking, on the forehead of the -th player (for ). In other words, the -th player knows the arguments but not . The players communicate by sending broadcast messages, taking turns according to a protocol agreed upon in advance. Each of them privately holds an unlimited supply of uniformly random bits, which he can use along with his available arguments when deciding what message to send at any given point in the protocol. The protocol’s purpose is to allow accurate computation of everywhere on the domain of . An -error protocol for is one which, on every input produces the correct answer with probability at least . The cost of a protocol is the total bit length of the messages broadcast by all the players in the worst case.111 The contribution of a -bit broadcast to the protocol cost is rather than . The -error randomized communication complexity of denoted is the least cost of an -error randomized protocol for . As a special case of this model for one recovers the original two-party model of Yao [88] reviewed in the introduction.
We focus on randomized protocols with probability of error close to that of random guessing, . There are two natural ways to define the communication complexity of a multiparty problem in this setting. The communication complexity of with unbounded error, introduced by Paturi and Simon [63], is the quantity
[TABLE]
The error probability in this formalism is “unbounded” in the sense that it can be arbitrarily close to . Babai et al. [11] proposed an alternate quantity, which includes an additive penalty term that depends on the error probability:
[TABLE]
We refer to as the communication complexity of with weakly unbounded error. These two complexity measures naturally give rise to corresponding complexity classes and in multiparty communication complexity [11], both inspired by Gill’s probabilistic polynomial time for Turing machines [31]. Formally, let be a family of -party communication problems , where is either a constant or a function. Then if and only if for some constant and all . Analogously, if and only if for some constant and all . By definition,
[TABLE]
It is standard practice to abbreviate and . The following well-known fact, whose proof in the stated generality is available in [80, Fact 2.4], gives a large class of communication problems that are efficiently computable with unbounded error.
Fact 2.17**.**
Let be a -party communication problem such that for some polynomial with monomials. Then
[TABLE]
In the setting of parties, Paturi and Simon [63] showed that unbounded-error communication complexity has a natural matrix-analytic characterization. For a matrix without zero entries, the sign-rank of is denoted and defined as the minimum rank of a real matrix such that for all In words, the sign-rank of is the minimum rank of a real matrix that has the same sign pattern as We extend the notion of sign-rank to communication problems by defining where is the characteristic matrix of The following classic result due to Paturi and Simon [63, Theorem 3] relates two-party unbounded-error communication complexity to sign-rank.
Theorem 2.18** (Paturi and Simon).**
Let be a two-party communication problem. Then
[TABLE]
2.9. Discrepancy
A -dimensional cylinder intersection is a function of the form
[TABLE]
where . In other words, a -dimensional cylinder intersection is the product of functions with range where the -th function does not depend on the -th coordinate but may depend arbitrarily on the other coordinates. Introduced by Babai et al. [12], cylinder intersections are the fundamental building blocks of communication protocols and for that reason play a central role in the theory. For a (possibly partial) Boolean function on and a probability distribution on the discrepancy of with respect to is given by
[TABLE]
where the maximum is over cylinder intersections . The minimum discrepancy over all distributions is denoted
[TABLE]
Upper bounds on a function’s discrepancy give lower bounds on its randomized communication complexity, a classic technique known as the discrepancy method [28, 12, 50].
Theorem 2.19**.**
Let be a possibly partial Boolean function on . Then for
[TABLE]
A proof of Theorem 2.19 in the stated generality is available in [79, Theorem 2.9]. Combining this theorem with the definition of gives the following corollary.
Corollary 2.20**.**
Let be a possibly partial Boolean function on . Then
[TABLE]
2.10. Pattern matrix method
Theorem 2.19 and Corollary 2.20 highlight the role of discrepancy in proving lower bounds on randomized communication complexity. Apart from a few canonical examples [50], discrepancy is a challenging quantity to analyze. The pattern matrix method is a technique that gives tight bounds on the discrepancy and communication complexity for a large class of communication problems. The technique was developed in [73, 75] for two-party communication complexity and has since been generalized by several authors to the multiparty setting. We now review the strongest form [79, 78] of the pattern matrix method, focusing our discussion on discrepancy bounds.
Set disjointness is the -party communication problem of determining whether given subsets of the universe have empty intersection, where, as usual, the -th party knows all the sets except for the -th. Identifying the sets with their characteristic vectors, set disjointness corresponds to the Boolean function given by
[TABLE]
The partial function on , called unique set disjointness, is defined as with domain restricted to inputs such that for at most one coordinate . In set-theoretic terms, this restriction corresponds to requiring that the sets either have empty intersection or intersect in a unique element.
The pattern matrix method pertains to the communication complexity of composed communication problems. Specifically, let be a (possibly partial) Boolean function on representing a -party communication problem, and let be given. The coordinatewise composition is then a -party communication problem on . We are now in a position to state the pattern matrix method for discrepancy bounds [79, Theorem 5.7].
Theorem 2.21** (Sherstov).**
For every Boolean function all positive integers and and all reals
[TABLE]
This theorem makes it possible to prove communication lower bounds by leveraging the existing literature on polynomial approximation. In follow-up work, the author improved Theorem 2.21 to an essentially tight upper bound [78, Theorem 5.7]. However, we will not need this sharper version.
3. Discrepancy of integer sets
Let be an integer modulus. Key to our work is the notion of -discrepancy, which quantifies the pseudorandomness or aperiodicity of any given multiset of integers modulo The -discrepancy of a nonempty multiset of arbitrary integers is defined as
[TABLE]
where is a primitive -th root of unity; the right-hand side is obviously the same for any such . By way of terminology, we emphasize that the notion of -discrepancy just defined is unrelated to the notion of discrepancy from Section 2.9. As a matter of convenience, we define
[TABLE]
The notion of -discrepancy has a long history in combinatorics and theoretical computer science, e.g., [30, 69, 2, 38, 64, 5]. The -discrepancy of an integer multiset has a natural interpretation in terms of the discrete Fourier transform on Specifically, consider the frequency vector of , where is the total number of element occurrences in that are congruent to modulo Applying the discrete Fourier transform to produces the sequence which is a permutation of Summarizing, the -discrepancy of coincides up to a normalizing factor with the largest absolute value of a nonconstant Fourier coefficient of the frequency vector of
3.1. Basic properties
We collect a few elementary properties of -discrepancy. To start with, we quantify the “continuity” of in the first argument. By way of notation, we remind the reader that the cardinality of a multiset is found by summing, for each distinct element the number of times occurs in
Proposition 3.1**.**
Fix a natural number Then for any nonempty finite multisets of integers with
[TABLE]
Proof.
Abbreviate and and fix an enumeration of the elements of such that Then for a primitive -th root of unity
[TABLE]
which directly implies (3.2). ∎
The -discrepancy of is invariant under a variety of operations on , such as shifting the elements of by any given integer or multiplying the elements of by an integer relatively prime to For our purposes, the following observation will be sufficient.
Proposition 3.2**.**
Fix a natural number and a nonempty finite multiset of integers. Then
[TABLE]
Proof.
The claim is immediate from the definition of -discrepancy because is a primitive -th root of unity if and only if is. ∎
3.2. Existential bounds
Since the -discrepancy of a multiset remains unchanged when one reduces its elements modulo we can focus without loss of generality on multisets with elements in The identity for any -th root of unity implies that achieves the smallest possible -discrepancy: The problem of constructing sparse nonempty multisets with small discrepancy has seen considerable work. Their existence is straightforward to verify, as follows.
Fact 3.3**.**
Fix and an integer . Let be a random multiset of size whose elements are chosen independently and uniformly at random from Then
[TABLE]
Fact 3.3 has been proved in one form or another by many authors, e.g., [30, 69, 5]. For the reader’s convenience, we include a short proof below.
Proof of Fact 3.3..
Let be independent random variables, each distributed uniformly in For any -th root of unity we have and for Hence, are independent random variables with range in and expectation and likewise for . As a result,
[TABLE]
where the second step uses Hoeffding’s inequality (Fact 2.1). Applying the union bound across all -th roots of unity we conclude that the probability that is at most . ∎
In some applications, one is restricted to working with subsets of as opposed to arbitrary multisets with possibly repeated elements. We record a version of Fact 3.3 for this setting.
Corollary 3.4**.**
Fix and an integer . Let be a random multiset of size whose elements are chosen independently and uniformly at random from Then with probability at least
[TABLE]
the elements of are nonzero and pairwise distinct, and obey
Proof.
The probability that does not contain [math] or repeated elements is easily seen to be As a result, the claim follows from Fact 3.3. ∎
In all of our applications, the error parameter will be a small constant. In this regime, Corollary 3.4 guarantees the existence of a set with -discrepancy at most and cardinality an exponential improvement in sparsity compared to the trivial set No further improvement is possible: it is well known that any nonempty multiset with -discrepancy bounded away from has cardinality . This classical lower bound has a remarkable variety of proofs, e.g., using random walks [5], sphere packing arguments [29], and diophantine approximation [53]. We include here a particularly simple and self-contained proof, adapted from Leung et al. [53]. Unlike all other technical statements in this paper, Fact 3.5 is not used in the proof of our main result and is provided solely for completeness.
Fact 3.5** (Leung et al.).**
Fix a natural number Let be a multiset of integers. Then
[TABLE]
Proof (adapted from [53])..
The proof is based on a classic technique from simultaneous diophantine approximation. For a nonnegative real number let denote the fractional part of Abbreviate and consider the intervals
[TABLE]
By the pigeonhole principle, there must be a pair of distinct integers such that
[TABLE]
are in the same intervals of (3.4) as
[TABLE]
respectively. Without loss of generality, Then the integer obeys
[TABLE]
for some Now
[TABLE]
where the first step uses the definition of -discrepancy; the second step applies the triangle inequality; the third step is valid by periodicity; the fourth step uses the bound for all real ; and the final step is immediate from (3.6). ∎
3.3. An explicit construction
We now turn to the problem of efficiently constructing sparse sets with small -discrepancy. Two such constructions are known to date, due to Ajtai et al. [2] and Katz [38]. The approach of Ajtai et al. is elementary except for an appeal to the prime number theorem. Katz’s construction, on the other hand, relies on deep results in number theory. Neither work appears to directly imply the kind of optimal de-randomization that we require, namely, an algorithm that runs in time polynomial in and produces a multiset of cardinality with -discrepancy bounded away from 1. We obtain such an algorithm by adapting the approach of Ajtai et al. [2]. The following technical result plays a central role.
Theorem 3.6** (cf. Ajtai et al.).**
Fix an integer and a real number . Let be an integer with Fix a set for each prime with such that all have the same cardinality. Consider the multiset
[TABLE]
Then the elements of are pairwise distinct and nonzero. Moreover,
[TABLE]
for some explicitly given constant independent of
Ajtai et al. [2] proved a special case of Theorem 3.6 for prime, but their argument readily generalizes to arbitrary moduli as just stated. For the reader’s convenience, we provide a complete proof of Theorem 3.6 in Appendix A. The theorem’s purpose is to reduce the construction of a sparse set with small -discrepancy to the construction of sparse sets with small -discrepancy, for primes By applying Theorem 3.6 in a recursive manner, one reaches smaller and smaller primes. The authors of [2] continue this recursive process until they reach primes so small that the trivial construction can be considered sparse. We proceed differently and terminate the recursion after just two stages, at which point the input size is small enough for brute force search based on Corollary 3.4. The final set that we construct has size logarithmic in and -discrepancy a small constant, as opposed to the superlogarithmic size and discrepancy in the work of Ajtai et al. [2]. A detailed exposition of our algorithm follows.
Theorem 3.7**.**
Let be given. Then there is an algorithm that takes as input an integer runs in time polynomial in and outputs a nonempty set with
[TABLE]
where is a constant. Moreover, the constant and the algorithm are given explicitly.
Proof.
Set where is the explicit constant from Theorem 3.6. Define
[TABLE]
We may assume that
[TABLE]
where is the prime counting function and is the number of distinct prime divisors function. Indeed, if any of (3.7)–(3.10) is violated, then by elementary calculus is bounded in terms of and therefore the trivial set satisfies and Analogously, the explicit bounds for and in Facts 2.3 and 2.4 ensure that (3.11) and (3.12) can fail only if is bounded in terms of so that we may again output .
Assuming (3.7)–(3.12), our construction of has three stages. In the first and second stages, we construct sparse sets with small -discrepancy for all primes and respectively. In the final stage, we construct the set in the theorem statement. We ensure that each stage runs in time polynomial in
Stage 1. For every prime Corollary 3.4 along with (3.8) guarantees the existence of a set with
[TABLE]
The primes in can be identified by the trivial algorithm in time polynomial in For each such prime we can find a set with the above properties in time by trying out all candidate sets.
Stage 2. Apply the construction of Theorem 3.6 with parameters and to the sets constructed in Stage to obtain a set for each prime This choice of parameters is legitimate by (3.9). By (3.13), the new sets have the same cardinality, namely,
[TABLE]
The prime number theorem (Fact 2.3) implies that . In view of (3.7), (3.14), and the new sets have
[TABLE]
We now show that Stage runs in time polynomial in To start with, the primes in can be identified by the trivial algorithm in time polynomial in For any such prime the construction of the corresponding set in Theorem 3.6 amounts to arithmetic operations in the field of size and therefore can be carried out in time polynomial in
Stage 3. Apply the construction of Theorem 3.6 with parameters and to the sets constructed in Stage to obtain a set . This choice of parameters is legitimate by (3.10). This new set has cardinality
[TABLE]
which in view of (3.11) and (3.12) guarantees that is nonempty. Simplifying,
[TABLE]
where the second step applies the prime number theorem (Fact 2.3). The multiplicative constant in this asymptotic bound on can be easily recovered from the explicit bounds in Fact 2.3. Using (3.9), (3.15), and we further obtain
[TABLE]
Since , the set satisfies the requirements of the theorem. Finally, the construction of in Stage 3 amounts to arithmetic operations in the ring and therefore can be carried out in time polynomial in ∎
4. Univariatization
Consider a halfspace in Boolean variables where the coefficients can be assumed without loss of generality to be integers. Then the linear form ranges in the discrete set , for some integer proportionate to the magnitude of the coefficients. As a result, one can approximate to any given error by approximating the sign function to on This approach works for both rational approximation and polynomial approximation. Needless to say, there is no reason to expect that the degree of the approximant in this naïve construction is anywhere close to optimal. Perhaps the most dramatic example is the odd-max-bit function, defined by . A moment’s thought reveals that can be approximated to any given error by a rational function of degree whereas the naïve construction produces an approximant of degree
Surprisingly, we are able to construct a halfspace with exponentially large coefficients for which the naïve construction is essentially optimal. Specifically, we show that a rational approximant for with given error and given numerator and denominator degrees implies an analogous univariate rational approximant for the sign function on As a result, tight lower bounds for the rational and polynomial approximation of follow immediately from the univariate lower bounds for the sign function. The construction of , carried out in this section, is the centerpiece of our paper. The role of is to reduce the multivariate problem taken up in this work to a well-understood univariate question, whence the title of this section. We have broken down the proof into four steps, corresponding to subsections 4.1–4.4 below.
4.1. Distribution of a linear form
modulo m
We start by studying the probability distribution of the weighted sum modulo , where are given integers and are chosen uniformly at random. We will show that the distribution is close to uniform whenever the multiset has small -discrepancy. This result uses the following classical fact on linear forms modulo .
Fact 4.1** (cf. Gould [32]; Thathachar [85]).**
Fix a natural number and a multiset of integers. Let be a primitive -th root of unity. Then
[TABLE]
Proof (adapted from [85, Lemma 13]).
The fraction of vectors that satisfy the equation can be computed directly, as follows:
[TABLE]
This implies (4.1) because for all . ∎
In the original version of this manuscript, we proved (4.1) using a different, matrix-analytic argument, which we include as Appendix B. The short and elegant proof above was pointed out to us by T. S. Jayram, who kindly allowed us to include it.
We now simplify the right-hand side of (4.1) and relate it to -discrepancy.
Lemma 4.2**.**
Fix a natural number and a multiset of integers. Then for all
[TABLE]
Proof.
Let be a primitive -th root of unity. For we have
[TABLE]
where the second step uses , and the third step follows by convexity since Maximizing over we arrive at
[TABLE]
In view of Fact 4.1, the proof is complete. ∎
4.2. Fooling distributions
Let be a multiset with -discrepancy bounded away from Consider the linear map given by We have shown that for uniformly random , the probability distribution of is exponentially close to uniform. This implies, for some constant , that the sets cannot be reliably distinguished by a real polynomial of degree up to . More precisely, the characteristic functions of have approximately the same Fourier spectrum up to degree . We will now substantially strengthen this conclusion by proving that there are probability distributions , supported on , respectively, such that the Fourier spectra of are exactly the same up to degree To use a technical term, these distributions fool any polynomial of degree up to , in that Our proof relies on the following technical result [77, Theorem 4.1].
Theorem 4.3** (Sherstov).**
Let be given functions on a finite set Suppose that
[TABLE]
By way of notation, we remind the reader that for any real-valued functions and and a nonempty subset of their domain. In words, Theorem 4.3 states that if each have small correlation with and, in addition, have small pairwise correlations, then a distribution exists with respect to which is completely uncorrelated with We are now in a position to prove the existence of the promised fooling distributions. In the statement that follows, recall that is the binary entropy function.
Lemma 4.4**.**
Fix and a nonempty multiset of integers. Let be an integer with
[TABLE]
Define
[TABLE]
Then each is nonempty. Moreover, there is a probability distribution on for each such that
[TABLE]
for all and all real polynomials of degree at most
Proof.
For a subset , define by The centerpiece of the proof is the following claim.
Claim 4.5*.*
For every and every nonempty proper subset
[TABLE]
We will proceed with the main proof and settle the claim after we are finished. Fix arbitrarily. Let denote the family of nonempty subsets of of cardinality at most Recall from (2.2) that
[TABLE]
As a result,
[TABLE]
where the second step uses (4.9) and Claim 4.5; the third step is valid because by (4.4); and the final step is immediate from (4.4). An analogous calculation shows that for every
[TABLE]
where the second step follows from (4.9) and Claim 4.5, and the last step uses (4.4).
Recall from Claim 4.5 that each is nonempty. Applying Theorem 4.3 with (4.10) and (4.11) to the functions and , we infer the existence of a probability distribution on such that
[TABLE]
Now that the probability distributions have been constructed for each consider an arbitrary polynomial of degree at most Then for some reals . As a result, (4.12) implies that for all thereby settling (4.6). ∎
Proof of Claim 4.5..
By symmetry, we may assume that for some Let be a random variable with uniform distribution on . Then
[TABLE]
where the last two steps follow from Lemma 4.2 and (4.4), respectively. This settles (4.7). Moreover,
[TABLE]
where the third step uses ; the next-to-last step is legitimate by Lemma 4.2; and the last step applies Proposition 3.1. Now (4.8) is immediate from (4.13) and (4.14). ∎
4.3. The univariate reduction
At last, we present a generic construction of a halfspace whose approximation by rational functions and polynomials gives corresponding approximants for the sign function on the discrete set . In more detail, let be given integers. For any such -tuple, we define an associated halfspace and prove a lower bound on in terms of the discrepancy of the multiset The following first-principles calculation will be helpful.
Proposition 4.6**.**
Let and . Then
[TABLE]
Proof.
Abbreviate and Since each is positive, we obtain . Taking a weighted sum of these inequalities, we arrive at which is equivalent to (4.15). ∎
We have:
Theorem 4.7**.**
Fix and a nonempty multiset of integers. Let be an integer with
[TABLE]
Define by
[TABLE]
Then
[TABLE]
for all
Proof.
Fix arbitrarily for the remainder of the proof, and suppose that for some Our goal is to show that
[TABLE]
The proof is algorithmic and involves three steps. Given any approximant for , we will first manipulate it to control the sign behavior in the numerator and denominator, then symmetrize it with respect to and finally—the arduous part of the proof—symmetrize it with respect to The result of these manipulations will be a univariate approximant for the sign function.
Step 1: Original approximant. Since there are polynomials and of degree at most and respectively, with
[TABLE]
for all This inequality is equivalent to
[TABLE]
Observe that for all we have and where the former is a consequence of and the latter follows from the definition of a rational approximant. As a result, (4.18) gives
[TABLE]
Step 2: Symmetrization on The fractions in (4.19) and (4.20) have positive numerators and denominators. Therefore, Proposition 4.6 implies that
[TABLE]
Minsky and Papert’s symmetrization technique (Proposition 2.15) ensures the existence of polynomials of degree at most and , respectively, such that for all
[TABLE]
Moreover,
[TABLE]
for all where is given by
[TABLE]
Now (4.21) and (4.22) simplify to
[TABLE]
for all and The numerators and denominators of these fractions are again positive, being averages of positive numbers.
Step 3: Symmetrization on We have reached the most demanding part of the proof, where we symmetrize the approximants obtained so far with respect to For let be given by (4.5). Then Lemma 4.4 guarantees that each is nonempty, and additionally provides a probability distribution on (for each ) such that for every polynomial
[TABLE]
Now fix an integer On the support of we have
[TABLE]
where the second step is valid because by (4.16). It follows that on the support of the linear form
[TABLE]
ranges in forcing . Now (4.23) and (4.24) imply that
[TABLE]
for all integers and all in the support of Since the numerators and denominators of these fractions are positive, Proposition 4.6 allows us to pass to expectations with respect to to obtain
[TABLE]
or equivalently
[TABLE]
for all integers
Consider the univariate polynomials
[TABLE]
Equations (4.26) and (4.27) show that and approximate pointwise on to error less than Moreover, (4.25) ensures that the degrees of are at most the degrees of respectively. We conclude that
[TABLE]
These complementary bounds force (4.17) and thereby complete the proof. ∎
4.4. The master theorem
We now combine Theorem 4.7 with the efficient construction, in Theorem 3.7, of an integer set with small -discrepancy for . The result is an explicit halfspace whose approximation by polynomials and rational functions is asymptotically equivalent to the univariate approximation of the sign function on . We refer to this result as our master theorem since all our main theorems are derived from it.
Theorem 4.8**.**
For some constant there is an algorithm that takes as input an integer runs in time polynomial in and outputs a halfspace with
[TABLE]
for all Moreover, the constant and the algorithm are given explicitly.
Proof.
Let
[TABLE]
where is the constant defined in Theorem 3.7. On input , the construction of is as follows. For the sought property (4.28) amounts to , which is in turn equivalent to and holds trivially for the halfspace
We now turn to the nontrivial case, Abbreviate Then the algorithm of Theorem 3.7 constructs, in time polynomial in a nonempty multiset with -discrepancy
[TABLE]
and cardinality Observe that for any integer the union of copies of is a multiset with -discrepancy and cardinality . Therefore, we may assume without loss of generality that
[TABLE]
We let
[TABLE]
where denote the elements of the multiset Taking we have from (4.29) and (4.31) that
[TABLE]
Moreover,
[TABLE]
where the second step applies (4.29), and the third step uses (4.30), (4.31), and As a result, Theorem 4.7 implies (4.28) for all . In view of (4.32), the proof is complete. ∎
5. Main results
Using the halfspace constructed in our master theorem, we will now establish the main results of this paper.
5.1. Polynomial approximation
Prior to our work, the strongest lower bound for the approximation of an explicit halfspace by polynomials was for an absolute constant , proved in [76, 77]. The result that we are about to prove is a quadratic improvement on previous work, with respect to both degree and error. As we will discuss shortly, this new result is essentially the best possible.
Theorem 5.1** (Polynomial approximation).**
Let be the halfspace constructed in Theorem 4.8. Then for some constant and all
[TABLE]
Proof.
Let be the constant in Theorem 4.8. Then
[TABLE]
where the first step corresponds to taking and in Theorem 4.8, and the second step is immediate from Proposition 2.9. This implies (5.1) for small enough. ∎
Theorem 5.1 is essentially as strong as one could hope for. First of all, any function in Boolean variables can be approximated to zero error by a polynomial of degree at most i.e., at most a constant factor larger than what is assumed in (5.1). Moreover, a classic result due to Muroga [58] implies that for every halfspace, the error bound in (5.1) is almost achieved by polynomials of degree :
Fact 5.2**.**
There is an absolute constant such that for every and every halfspace
[TABLE]
Proof.
Muroga [58] showed that every halfspace can be represented as for some integers whose absolute values sum to It follows that
[TABLE]
5.2. Rational approximation
We now show that the halfspace constructed in our master theorem cannot be approximated pointwise to any small constant except by rational functions of degree . This degree lower bound matches the trivial upper bound and is a quadratic improvement on the previous best construction [76, 77]. More generally, we derive a lower bound on the approximation of by rational functions of any given degree , and this lower bound too is essentially the best possible for any halfspace. Details follow.
Theorem 5.3** (Rational approximation).**
Let be the halfspace constructed in Theorem 4.8. Then for some constant and all
[TABLE]
Proof.
Let be the constant in Theorem 4.8. Then for we have
[TABLE]
where the first step corresponds to taking in Theorem 4.8, and the second step is immediate from Theorem 2.11. This implies (5.2) for small enough. ∎
We now show that the lower bounds on the approximation error in Theorem 5.3 are essentially the best possible for any halfspace.
Fact 5.4**.**
There exists an absolute constant such that for every and every halfspace
[TABLE]
Proof.
As already mentioned, Muroga [58] showed that for some linear polynomial that ranges in where for some absolute constant . This makes it possible to obtain a rational approximant for by taking any rational approximant for the sign function on and composing it with . We conclude that for any integer ,
[TABLE]
where the second step uses Newman’s rational approximation (Fact 2.10). ∎
5.3. Threshold degree
Here, we use the halfspace constructed in our master theorem to study the degree required to sign-represent intersections of halfspaces. Our result is a lower bound of for the intersection of two independent copies of This result improves quadratically on the previous best construction [76, 77] and matches the trivial upper bound of for sign-representing any Boolean function in variables.
Theorem 5.5**.**
Let be the halfspace constructed in Theorem 4.8. Then
[TABLE]
Proof.
Abbreviate Taking in Theorem 2.14 shows that which by Theorem 5.3 forces ∎
Theorem 5.5 should be contrasted with the result of Beigel et al. [17] that the conjunction of any constant number of majority functions on has threshold degree We now derive a lower bound of on the threshold degree of the intersection of an explicitly given halfspace and a majority function, improving quadratically on the previous best construction [76, 77]. As we discuss shortly, the new construction is optimal up to a logarithmic factor.
Theorem 5.6**.**
Let be the halfspace constructed in Theorem 4.8. Then
[TABLE]
Proof.
Abbreviate Then by Theorem 2.14. The lower bounds for the rational approximation of and in Theorems 2.12 and 5.3 now imply that ∎
Remark 5.7*.*
The construction of Theorem 5.6 is essentially the best possible, in that
[TABLE]
for every halfspace Indeed, taking in Theorem 2.12 and Fact 5.4 for a large enough constant yields which in turn implies (5.4) in view of Theorem 2.13.
5.4. Threshold density
In addition to threshold degree, several other complexity measures are of interest when sign-representing Boolean functions by real polynomials. One such complexity measure is threshold density, defined as the least for which a given function can be sign-represented by a linear combination of parity functions. Formally, for a given function its threshold density is the minimum size of a family such that
[TABLE]
for some reals It is clear from the definition that for all functions and we will now construct a pair of halfspaces whose intersection has threshold density Prior to our work, the best construction [76] had threshold density
To proceed, we recall a technique due to Krause and Pudlák [48] that transforms Boolean functions with high threshold degree into Boolean functions with high threshold density. Their transformation works in a black-box manner and sends a function to the function defined by
[TABLE]
The threshold degree of and the threshold density of are related as follows [48, Proposition 2.1].
Theorem 5.8** (Krause and Pudlák).**
For every function
[TABLE]
We are now in a position to obtain the claimed density results.
Theorem 5.9**.**
There is an explicit algorithm that takes as input an integer runs in time polynomial in and outputs a halfspace such that
[TABLE]
Proof.
For any function standard arithmetization gives
[TABLE]
where denotes as usual the XOR of and . Similarly, one has
[TABLE]
where the XOR and complement operations are applied bitwise.
Let be the halfspace from Theorem 5.5, so that has threshold degree By Theorem 5.8, the function has threshold density Observe from (5.7) that is the result of starting with the intersection of two explicitly given halfspaces in variables each, and replacing their input variables with appropriately chosen parity functions. This replacement cannot increase the threshold density because the parity of several parity functions is another parity function. We conclude that This completes the proof of (5.5).
The proof of (5.6) is closely analogous. Specifically, recall from Theorem 5.6 that has threshold degree By Theorem 5.8, the function has threshold density It follows from (5.7) and (5.8) that is the result of starting with the intersection for an explicit halfspace in variables, and replacing the input variables with appropriately chosen parity functions or their negations. This replacement cannot increase the threshold density because the parity of several parity functions is another parity function. We conclude that This completes the proof of (5.6). ∎
Both lower bounds in Theorem 5.9 are essentially the best possible for any halfspace . Indeed, the first lower bound is tight by definition, while the second lower bound nearly matches the upper bound of that follows from Remark 5.7.
5.5. Communication complexity
Using the pattern matrix method, we will now “lift” the approximation lower bound of Theorem 5.1 to communication complexity. As a result, we will obtain an explicit separation of -party communication complexity with unbounded and weakly unbounded error (which for is equivalent to a separation of sign-rank and discrepancy). Our application of the pattern matrix method is based on the fact that the unique set disjointness function has an exact representation on its domain as a polynomial with a small number of monomials; cf. [75, Section 10], [83, Section 4.2.3], and [80, Section 3.1]. Specifically, define by
[TABLE]
Then
[TABLE]
Theorem 5.10**.**
For some constant and all positive integers and there is an explicitly given -party communication problem such that
[TABLE]
Moreover,
[TABLE]
for some explicitly given reals .
Proof.
Let be the halfspace constructed in Theorem 4.8. Then by definition, for a linear polynomial . Moreover, Theorem 5.1 ensures that
[TABLE]
for some constant independent of . Abbreviate and consider the -party communication problem given by
[TABLE]
where the right-hand side features the coordinatewise composition of the polynomial with independent copies of the polynomial . The identity (5.9) implies that coincides with on the domain of the latter. Therefore,
[TABLE]
where the second step uses (5.14) and the pattern matrix method (Theorem 2.21). Applying the discrepancy method (Corollary 2.20), we obtain
[TABLE]
To complete the proof, define the functions for any positive integers and by
[TABLE]
Then (5.11)–(5.13) are immediate from (5.15)–(5.17), whereas (5.10) is a consequence of (5.13) and Fact 2.17. ∎
Theorem 5.10 gives an explicit separation for up to parties, where is an arbitrary constant. The special case can be equivalently stated as an explicit separation of sign-rank and discrepancy:
Corollary 5.11**.**
There is an explicitly given family of communication problems with
[TABLE]
Moreover,
[TABLE]
for some explicitly given reals .
Proof.
Equations (5.19)–(5.22) result from setting in Theorem 5.10. The new item, (5.18), is immediate from (5.22). ∎
Theorem 5.10 and Corollary 5.11 settle Theorems 1.3 and 1.2, respectively, from the introduction.
5.6. A circulant expander
Consider a -regular undirected graph on vertices, with adjacency matrix Since is symmetric, it has real eigenvalues (counting multiplicities). We denote these eigenvalues by and define . It is well known and straightforward to verify that and for We say that is an -expander if This spectral notion is intimately related to key graph-theoretic and stochastic properties of , such as vertex expansion and the convergence rate of a random walk on to the uniform distribution. One is typically interested in -expanders that are -regular for as small as possible, where is a constant. The existence of expanders with strong parameters can be verified using the probabilistic method [6], and explicit constructions are known as well.
In this section, we study the problem of constructing *circulant *expanders. Formally, a graph is circulant if its adjacency matrix is circulant. It is clear that a circulant graph is -regular for some , meaning that every vertex has out-degree and in-degree We focus on circulant graphs that are undirected and have no self-loops, which corresponds to adjacency matrices that are symmetric and have zeroes on the diagonal. It is well known [5] that for any and all large enough , there exists a circulant -expander on vertices of degree . This degree bound is asymptotically optimal [5, 29, 53], and the problem of constructing such circulant expanders explicitly has been studied by several authors [4, 2, 5]. The best construction prior to our work, due to Ajtai et al. [2], achieves degree . In this section, we construct a circulant -expander of optimal degree, , for any constant . By way of terminology, recall that the adjacency matrix of a circulant graph on vertices is for some subset With this in mind, we say that an algorithm constructs a circulant graph on vertices in time if the algorithm outputs in time the elements of the associated subset . The formal statement of our result follows.
Theorem 5.12**.**
Let be given. Then there is an explicitly given algorithm that takes as input an integer and constructs in time polynomial in an undirected simple -regular circulant graph on vertices, where
[TABLE]
Proof.
Let be the constant from Theorem 3.7. We first consider the trivial case when which means that is bounded by an explicit constant. In this case, we take to be the complete graph on vertices. It is clear that is a -regular circulant graph for The adjacency matrix of is , whose eigenvalues by Corollary 2.6 are . In particular, This settles (5.24), whereas (5.23) holds trivially because and are bounded by a constant.
We now turn to the nontrivial case when The algorithm of Theorem 3.7 constructs, in time polynomial in a set with
[TABLE]
For any the linear congruence has at most two solutions Recalling that in the case under consideration, we conclude that there exists with
[TABLE]
Moreover, such can clearly be found by brute force search in time polynomial in Equation (5.27) now implies that no two elements of the multiset are congruent modulo and in particular no element of is congruent to [math] modulo
We define to be the undirected graph with vertex set in which is an edge if and only if is congruent modulo to an element of The roles of and in this definition are symmetric, making an undirected graph. It is obvious that the adjacency matrix of is circulant. Furthermore, has no self-loops because by construction no element of is congruent to [math] modulo . Since the elements of are pairwise distinct modulo the degree of is Now (5.23) follows from (5.26). To settle the remaining property (5.24), observe that the first row of the adjacency matrix of is the characteristic vector of the set . As a result, Corollary 2.6 implies that the eigenvalues of the adjacency matrix of are
[TABLE]
where is a primitive -th root of unity. Setting yields the largest eigenvalue, . The other eigenvalues are bounded by
[TABLE]
Along with (5.25), this proves (5.24). ∎
Acknowledgments
I am thankful to Mark Bun, T. S. Jayram, Ryan O’Donnell, Rocco Servedio, and Justin Thaler for valuable comments on this work. Special thanks to T. S. Jayram for allowing me to include his short and elegant proof of Fact 4.1.
Appendix A The iteration lemma of Ajtai et al.
The purpose of this appendix is to provide a detailed and self-contained proof of Theorem 3.6, which we restate below for the reader’s convenience.
Theorem**.**
Fix an integer and a real number . Let be an integer with
[TABLE]
Fix a set for each prime with such that all have the same cardinality. Consider the multiset
[TABLE]
Then the elements of are pairwise distinct and nonzero. Moreover,
[TABLE]
for some explicitly given constant independent of
This result is a slight generalization of the *iteration lemma *of Ajtai et al. [2], which corresponds to the special case for prime. We closely follow their proof but provide ample detail to make it more accessible. We have structured the presentation around five key milestones, corresponding to Sections A.1–A.5 below. Before proceeding, the reader may wish to review the number-theoretic preliminaries in Section 2.2.
A.1. Shorthand notation
In the remainder of this manuscript, we adopt the shorthand
[TABLE]
where is the imaginary unit. We will need the following bounds, illustrated in Figure A.1:
[TABLE]
To verify these bounds, write and apply elementary calculus.
We let denote the set of prime numbers with In this notation, the multiset is given by
[TABLE]
There are precisely primes in of which at most are prime divisors of Therefore,
[TABLE]
A.2. Elements of S are
nonzero and distinct
As our first step, we verify that the elements of are nonzero modulo . Consider any , any prime with and any Then This means that which in turn implies that .
We now show that the multiset contains no repeated elements. For this, consider any any primes and any and such that
[TABLE]
Our goal is to show that To this end, multiply (A.5) through by to obtain
[TABLE]
The left-hand side and right-hand side of (A.6) are integers in whence
[TABLE]
This implies that which in view of and the primality of and forces Now (A.7) simplifies to
[TABLE]
which in turn yields . Recalling that we arrive at Finally, substituting in (A.8) gives
A.3. Correlation for k
small
So far, we have shown that the elements of are distinct and nonzero. Recall that our objective is to bound the -discrepancy of this set. Put another way, we must bound the exponential sum
[TABLE]
for all This subsection and the next provide two complementary bounds on (A.9). The first bound, presented below, is preferable when is close to zero modulo
Claim A.1*.*
Let be given. Then
[TABLE]
Proof.
Let be the set of those primes in that do not divide or Then clearly
[TABLE]
We have
[TABLE]
We proceed to bound the two summations in (A.11). Bounding the second summation is straightforward:
[TABLE]
where the first step is valid because all sets have the same cardinality, and the last step uses (A.10).
The other summation in (A.11) requires more work. For and we have
[TABLE]
where the second step uses Fact 2.2 and the relative primality of and ; the third step applies the triangle inequality; the fourth step follows from , and the last step is valid by (A.2) and . We have shown that
[TABLE]
for Summing over
[TABLE]
By (A.11)–(A.13), the proof of the claim is complete. ∎
A.4. Correlation for
large
We now present an alternative bound on the exponential sum (A.9), which is preferable to the bound of Claim A.1 when is far from zero modulo
Claim A.2*.*
Let be given. Then
[TABLE]
Proof:.
[TABLE]
where the last two steps use (A.3) and , respectively. ∎
A.5. Finishing the proof
[TABLE]
where is a constant independent of Moreover, can be easily calculated from the explicit bounds in Facts 2.3 and 2.4. We will show that the theorem conclusion (A.1) holds with We may assume that
[TABLE]
since otherwise the right-hand side of (A.1) exceeds and the theorem is trivially true. By (A.4), (A.14), (A.15), and (A.17), we obtain
[TABLE]
which along with (A.15) gives
[TABLE]
Claims A.1 and A.2 ensure that for every
[TABLE]
Substituting the estimate from (A.18), we conclude that
[TABLE]
This conclusion is equivalent to (A.1). The proof of Theorem 3.6 is complete.
Appendix B An alternate proof of Fact 4.1
The purpose of this appendix is to give an alternate, matrix-analytic proof of Fact 4.1.
Fact** (restatement of Fact 4.1).**
Fix a natural number and a multiset of integers. Let be a primitive -th root of unity. Then
[TABLE]
Proof.
For any integer consider the circulant matrix
[TABLE]
By Corollary 2.6, the matrix obeys
[TABLE]
In particular,
[TABLE]
where the first two steps use (B.2) and (B.3), respectively. Applying (B.2) yet again, we arrive at
[TABLE]
where denote the last columns of Since the components of each are bounded in absolute value by we conclude that
[TABLE]
We are now in a position to prove (B.1). Let be a random variable distributed uniformly in Consider the random variables given by The sequence has a natural interpretation in terms of an -step random walk in Specifically, the random walk starts at and evolves according to
[TABLE]
In particular, the -th step of the random walk has transition probability matrix
[TABLE]
where the unspecified entries are zero, and the rows and columns correspond in the usual manner to the values . In the notation of the opening paragraph of the proof, this matrix is precisely Letting be the -dimensional vectors that represent the probability distributions of respectively, we obtain the recursive relations . Therefore,
[TABLE]
Now
[TABLE]
where the next-to-last step uses (B.4). This conclusion is obviously equivalent to (B.1). ∎
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] S. Aaronson and Y. Shi , Quantum lower bounds for the collision and the element distinctness problems , J. ACM, 51 (2004), pp. 595–605, doi:10.1145/1008731.1008735 . · doi ↗
- 2[2] M. Ajtai, H. Iwaniec, J. Komlós, J. Pintz, and E. Szemerédi , Construction of a thin set with small Fourier coefficients , Bulletin of the London Mathematical Society, 22 (1990), pp. 583–590, doi:10.1112/blms/22.6.583 . · doi ↗
- 3[3] M. Alekhnovich, M. Braverman, V. Feldman, A. R. Klivans, and T. Pitassi , The complexity of properly learning simple concept classes , J. Comput. Syst. Sci., 74 (2008), pp. 16–34, doi:10.1016/j.jcss.2007.04.011 . · doi ↗
- 4[4] N. Alon , Eigenvalues, geometric expanders, sorting in rounds, and Ramsey theory , Combinatorica, 6 (1986), pp. 207–219, doi:10.1007/BF 02579382 . · doi ↗
- 5[5] N. Alon and Y. Roichman , Random Cayley graphs and expanders , Random Struct. Algorithms, 5 (1994), pp. 271–285, doi:10.1002/rsa.3240050203 . · doi ↗
- 6[6] N. Alon and J. Spencer , The Probabilistic Method , John Wiley & Sons, 3rd ed., 2008.
- 7[7] A. Ambainis , Polynomial degree and lower bounds in quantum complexity: Collision and element distinctness with small range , Theory of Computing, 1 (2005), pp. 37–46, doi:10.4086/toc.2005.v 001a 003 . · doi ↗
- 8[8] A. Ambainis, A. M. Childs, B. Reichardt, R. Špalek, and S. Zhang , Any AND-OR formula of size N 𝑁 N can be evaluated in time N 1 / 2 + o ( 1 ) superscript 𝑁 1 2 𝑜 1 N^{1/2+o(1)} on a quantum computer , SIAM J. Comput., 39 (2010), pp. 2513–2530, doi:10.1137/080712167 . · doi ↗
