On the $\text{AC}^0[\oplus]$ complexity of Andreev's Problem

TL;DR

This paper establishes an b0[igoplus] lower bound for Andreev's Problem, linking it to list recovery in Reed-Solomon codes over finite fields, with implications for complexity theory.

Contribution

It provides the first b0[igoplus] complexity lower bound for Andreev's Problem and explores its relation to Reed-Solomon list recovery with random subsets.

Findings

Proves b0[igoplus] lower bound for Andreev's Problem

Connects Andreev's Problem to Reed-Solomon list recovery

Analyzes list recovery with random subsets of finite fields

Abstract

Andreev's Problem states the following: Given an integer $d$ and a subset of $S \subseteq \mathbb{F}_q \times \mathbb{F}_q$, is there a polynomial $y = p(x)$ of degree at most $d$ such that for every $a \in \mathbb{F}_q$, $(a,p(a)) \in S$? We show an $\text{AC}^0[\oplus]$ lower bound for this problem. This problem appears to be similar to the list recovery problem for degree $d$-Reed-Solomon codes over $\mathbb{F}_q$ which states the following: Given subsets $A_1,\ldots,A_q$ of $\mathbb{F}_q$, output all (if any) the Reed-Solomon codewords contained in $A_1\times \cdots \times A_q$. For our purpose, we study this problem when $A_1, \ldots, A_q$ are random subsets of a given size, which may be of independent interest.

Equations

$$\mathrm{RS}[q,d]=\{(p(a_1),\dots ,p(a_q))|p\in {\mathcal{P}}_d^q\}$$

$$\mathcal{C}\cap (A_1\times \cdots \times A_n)\ne \varnothing ?$$

$$S=\bigcup _{i\in [q]}\{(a_i,z)|z\in A_i\}\subseteq {\mathbb{F}}_q^2$$

$$f(z_1,\dots ,z_n)=\underset{i\in [\ell ]}{\bigvee }\left(\underset{j\in Z_i}{\bigwedge }{z}_j\right).$$

$$X:=|(A_1\times \cdots \times A_q)\cap \mathcal{C}|.$$

$$\mathrm{I}\mathrm{E}[X]$$

$$=|\mathcal{C}|\cdot (q^{-r}(1-ϵ){)}^q$$

$$=q^{rq+1}\left(q^{-r}(1-ϵ\right){)}^q$$

$$=q\cdot (1-ϵ{)}^q$$

$$\le q\cdot e^{-ϵq}$$

$$=o(1).$$

$$\mathrm{I}\mathrm{E}[X]=q\cdot (1+ϵ{)}^q\ge q.$$

$$\mathbb{P}({\text{AP}}_r(q^{-r}(1-ϵ))=1)\le \exp \left(-\Omega (ϵq)\right).$$

$$\mathbb{P}((A_1\times \cdots \times A_q)\cap \mathrm{RS}[q,rq]=\varnothing )=o(1).$$

$$X=|\mathcal{C}\cap (A_1\times \cdots \times A_q)|=\sum _{(x_1,\dots ,x_q)\in {\mathbb{F}}^q}f(x)\left(\prod _{i\in [q]}{g}_i(x_i)\right).$$

$$\frac{q^q}{|\mathcal{C}|}\cdot X$$

$$\ge \prod _{i\in [q]}\widehat{g_i}(0)-\left|\sum _{({\alpha }_1,\dots ,{\alpha }_q)\in {\mathcal{C}}^{\perp }}\left(\prod _{i\in [q]}\widehat{g_i}({\alpha }_i)\right)\right|.$$

$$\mathrm{I}\mathrm{E}[|R{|}^2]=\sum _{({\alpha }_1,\dots ,{\alpha }_q)\in {\mathcal{C}}^{\perp }\setminus \{\overline{0}\}}\prod _{i\in [q]}\mathrm{I}\mathrm{E}[|\widehat{g_i}({\alpha }_i){|}^2].$$

$$\mathrm{I}\mathrm{E}[|\widehat{g_i}(0){|}^2]\sim {\left(\frac{t}{q}\right)}^2$$

$$\mathrm{I}\mathrm{E}[|\widehat{g_i}({\alpha }_i){|}^2]\sim \frac{t}{q^2}$$

$$\mathrm{I}\mathrm{E}[|R{|}^2]$$

$$\approx q^{q-rq}{\left(\frac{p}{q}\right)}^{q-o(q)}$$

$$\approx q^{-rq}{p}^{q-o(q)}.$$

$$\prod _{i\in [q]}\widehat{g_i}(0)$$

$$\approx p^q.$$

$$\underset{w\in \mathrm{RS}[q,d]}{\max }|w\cap S|=d{\log }_{\frac{1}{p}}q-O\left(\frac{q}{\log \left(\frac{1}{p}\right)}\right).$$

$$\mathbb{P}(\text{C}(X_1,\dots ,X_n)=0)\ge 0.99$$

$$\mathbb{P}(\text{C}(X_1,\dots ,X_n)=1)\ge 0.99$$

$$\mathbb{P}\left(F_p(X_1,\dots ,X_t)=1\right)=p(1+\Omega (ϵL))$$

$$\mathbb{P}\left(F_p(X_1,\dots ,X_t)=1\right)=p\left(1+\frac{1}{2^{\Omega (\sqrt{t})}}-\Omega (ϵL)\right)$$

$${\text{AP}}_r\notin \text{C}\left(\exp \left\{\tilde{\Omega }\left(h{q}^{\frac{r^2}{h-1}}\right)\right\},h\right).$$

$$\mathbb{P}((A_1\times \cdots \times A_n)\cap \mathcal{C}=\varnothing )=o(1).$$

$$\mathbb{P}({\mathcal{E}}_{q-\ell })\le \left(\genfrac{}{}{0ex}{}{q}{\ell }\right)q^{rq+1}{p}^{q-\ell }.$$

$$S_i:=\{j|(i,j)\in S\}.$$

$$\mathcal{C}=\{(i,p_w(i))|i\in [q-\ell ])\}$$

Full text

This problem appears to be similar to the list recovery problem for degree $d$-Reed-Solomon codes over $\mathbb{F}_{q}$ which states the following: Given subsets $A_{1},\ldots,A_{q}$ of $\mathbb{F}_{q}$, output all (if any) the Reed-Solomon codewords contained in $A_{1}\times\cdots\times A_{q}$. For our purpose, we study this problem when $A_{1},\ldots,A_{q}$ are random subsets of a given size, which may be of independent interest.

1 Introduction

For a prime power $q$, let us denote by $\mathbb{F}_{q}$, the finite field of order $q$. Let us denote the elements of $\mathbb{F}_{q}=\{a_{1},\ldots,a_{q}\}$. One can think of $a_{1},\ldots,a_{q}$ as some ordering of the elements of $\mathbb{F}_{q}$. Let $\mathcal{P}_{d}=\mathcal{P}_{d}^{q}$ be the set of all univariate polynomials of degree at most $d$ over $\mathbb{F}_{q}$. Let us define the problem which will be the main focus of this paper:

• Input: A subset $S\subset\mathbb{F}_{q}^{2}$, and integer $d$.
• Output: Is there a $p\in\mathcal{P}_{q}^{d}$ such that $\{(a_{i},p(a_{i}))\;|\;i\in[q]\}\subseteq S$?

The problem of proving NP-hardness of the above function seems to have been first asked in [Joh86]. It was called ‘Andreev’s Problem’ and still remains open. One may observe that above problem is closely related to the List Recovery of Reed-Solomon codes. In order to continue the discussion, we first define Reed-Solomon codes:

Definition 1.1** (Reed-Solomon code).**

The degree $d$ Reed-Solomon over $\mathbb{F}_{q}$, abbreviated as $\operatorname{RS}[q,d]$ is the following set:

[TABLE]

Reed-Solomon codes are one of the most widely (if not the most widely) studied families of error-correcting codes. It can be checked that $\operatorname{RS}[q,d]$ is a $d+1$-dimensional subspace of $\mathbb{F}_{q}^{q}$ such that every non-zero vector has at least $q-d$ non-zero coordinates. In coding theoretic language, we say that $\operatorname{RS}[q,d]$ is a linear code of block length $q$, dimension $d+1$ and distance $q-d$. The rate of the code is given by $\frac{d}{q}$. The set of relevant facts about Reed-Solomon codes for this paper may be found in Appendix A.1

The List Recovery problem for a code $\mathcal{C}\subset\mathbb{F}_{q}^{n}$ is defined as follows:

Definition 1.2** (List Recovery problem for $\mathcal{C}$).**

• Input:* Sets $A_{1},\ldots,A_{n}\subseteq\mathbb{F}_{q}$.*
• Output:* $\mathcal{C}\cap(A_{1}\times\cdots\times A_{n})$*

Given the way we have defined these problems, one can see that Andreev’s Problem is essentially proving NP-hardness for the List Recovery of Reed-Solomon codes where one just has to output a Boolean answer to the question

[TABLE]

Indeed, let us consider a List Recovery instance where the code $\mathcal{C}$ is $\operatorname{RS}[q,d]$, and the input sets are given by $A_{1},\ldots,A_{q}$. Let us identify $(A_{1},\ldots,A_{q})$ with the set

[TABLE]

and let us identify every codeword $w=(w_{1},\ldots,w_{q})\in\mathcal{C}$, with a set $w_{\text{set}}=\{(a_{i},w_{i})\;|\;i\in[q]\}$. Clearly, we have that $w\in A_{1}\times\cdots\times A_{q}$ if and only if $w_{\text{set}}\subseteq S$. Often, we will drop the subscript on $w_{\text{set}}$ and refer to $w$ both as a codeword, and as the set of points it passes through. Further identifying $\mathbb{F}_{q}^{2}$ with $[q^{2}]$, and and parameterizing the problem by $r=\frac{d}{q}$, we view Andreev’s Problem as a Boolean function $\text{AP}_{r}:\{0,1\}^{q\times q}\rightarrow\{0,1\}$.

The main challenge here is to prove (or at least conditionally disprove) NP-hardness for Andreev’s Problem, which has been open for over $30$ years. Another natural problem one could study is the circuit complexity for $\text{AP}_{r}$. This is the main motivation behind this paper, and we will study the $\text{AC}^{0}[\oplus]$ complexity of $\text{AP}_{r}$. We shall eventually see that even this problem needs relatively recent results about the power of $\text{AC}^{0}[\oplus]$ in our proof. Informally, $\text{AC}^{0}$ is the class of Boolean functions computable by circuits of constant depth, and polynomial size, using $\land$, $\lor$, and $\lnot$ gates. $\text{AC}^{0}[\oplus]$ is the class of Boolean functions computable by circuits of constant depth, and polynomial size, using $\land$, $\lor$, $\lnot$, and $\oplus$ ($\text{MOD}_{2}$) gates. The interested and unfamiliar reader is referred to [AB09] (Chapter $14$) for a more formal definition and further motivation behind this class. We show that $\text{AP}_{r}$ cannot be computed by $\text{AC}^{0}$ circuits for a constant $r$. This type of result is essentially motivated by a similar trend in the study of the complexity of Minimum Circuit Size Problem. Informally, the Minimum Size Circuit Problem (or simply MCSP) takes as input a truth table of a function on $m$ bits, and an integer $s$. The output is $1$ if there is a Boolean circuit that computes the function with the given truth table and has size at most $s$. It is a major open problem to show the NP-hardness of MCSP. A lot of effort has also gone into understanding the circuit complexity of MCSP. Allender et al. [ABK*+*06] proved a superpolynomial $\text{AC}^{0}$ lower bound, and Hirahara and Santanam [HS17] proved an almost-quadratic formula lower bound for MCSP. A recent result by Golonev et al. [GII*+*19] extends [ABK*+*06] and proves an $\text{AC}^{0}[\oplus]$ lower bound for MCSP. Thus one can seek to answer the same question about $\text{AP}_{r}$.

We now state our main theorem:

Theorem 1.3** (Main Theorem).**

For any prime power $q$, and $r\in(0,1)$, we have that any depth $h$ circuit with $\land$, $\lor$, $\lnot$, and $\oplus$ gates that computes $\text{AP}_{r}$ on $q^{2}$ bits must have size at least $\exp\left(\tilde{\Omega}\left(hq^{\frac{c^{2}}{h-1}}\right)\right)$.

We make a couple of comments about the theorem. The first, most glaring aspect is that $r\in(0,1)$ is more or less a limitation of our proof technique. Of course, as $r$ gets very small, i.e., $r=O\left(\frac{1}{q}\right)$, one can find depth $2$ circuits of size $q^{O(rq)}=q^{O(1)}$. But, we do not know that the case where, for example, $r=\Theta\left(\frac{1}{\log q}\right)$ is any easier for $\text{AC}^{0}$. Secondly, we are not aware of a different proof of an $\text{AC}^{0}$ (as opposed to an $\text{AC}^{0}[\oplus]$) lower bound.

1.1 Some notation and proof ideas

For a $p\in(0,1)$, let $X_{1},X_{2},\ldots$ denote independent $\operatorname{Ber}(p)$ random variables. For a family of Boolean functions $f:\{0,1\}^{n}\rightarrow\{0,1\}$, we use $f^{(n)}(p)$ to denote the random variable $f(X_{1},\ldots,X_{n})$.

Definition 1.4** (Sharp threshold).**

For a monotone family of functions $f$, we say that $f$ has a sharp threshold at $p$ if for every $\epsilon>0$, there is an $n_{0}$ such that for every $n>n_{0}$, we have that $\mathbb{P}(f^{(n)}(p(1-\epsilon))=0)\geq 0.99$, and $\mathbb{P}(f^{(n)}(p(1+\epsilon))=1)\geq 0.99$.

Henceforth, we shall assume that $q$ is a very large prime power. So, all the high probability events and asymptotics are as $q$ grows. Where there is no ambiguity, we also just use $f(p)$ to mean $f^{(n)}(p)$ and $n$ growing.

One limitation of $\text{AC}^{0}[\oplus]$ that is exploited when proving lower bounds (including in [GII*+*19]) for monotone functions is that $\text{AC}^{0}[\oplus]$ cannot compute functions with ‘very’ sharp thresholds. For a quantitative discussion, let us call the smallest $\epsilon$ in the definition above the threshold interval. It is known that $\text{AC}^{0}$ (and therefore, $\text{AC}^{0}[\oplus]$) can compute (some) functions with threshold interval of $O\left(\frac{1}{\log n}\right)$, for example, consider the following function on Boolean inputs $z_{1},\ldots,z_{n}$: Let $Z_{1}\sqcup\cdots\sqcup Z_{\ell}$ be an equipartition of $[n]$, such that each $|Z_{i}|\approx\log n-\log\log n$. Consider the function given by

[TABLE]

This is commonly known as the tribes function and is known to have a threshold interval of $O\left(\frac{1}{\log n}\right)$. This is clearly computable by an $\text{AC}^{0}$ circuit. A construction from [LSS*+*18] gives an $\text{AC}^{0}$ circuit (in $n$ inputs) of size $n^{O(h)}$ and depth $h$ that has a threshold interval $\tilde{O}\left(\frac{1}{(\log n)^{h-1}}\right)$. A remarkable result from [LSS*+*18] and [CHLT19] (Theorem 13 from [CHLT19] and Lemma $3.2$ in [Agr19]) says that this is in some sense, tight. Formally,

Theorem 1.5** ( [LSS*+*18] [CHLT19]).**

Let $n$ be any integer and $f:\{0,1\}^{n}\rightarrow\{0,1\}$ be a function with threshold interval $\delta$ at $\frac{1}{2}$. Any depth $h$ circuit with $\land$, $\lor$, $\lnot$, and $\oplus$ gates that computes $f$ must have size at least $\exp\left(\Omega\left(h\left(1/\delta\right)^{\frac{1}{h-1}}\right)\right)$.

In [LSS*+*18], this was studied as the Coin Problem, which we will also define in Section 2. Given the above theorem, a natural strategy suggests itself. If we could execute the following two steps, then we would be done:

Establish Theorem 1.5 for functions with thresholds at points other than $\frac{1}{2}$.

• 2.

Show that $\text{AP}_{r}$ has a sharp threshold at $q^{-r}$ with a suitably small threshold interval, i.e., $\frac{1}{\operatorname{poly}q}$.

The first fact essentially reduces to approximating $p$-biased coins by unbiased coins in constant depth. Though we are unable to find a reference for this, this is relatively straightforward, and is postponed to Appendix D.2. Understanding the second part, naturally leads us to study $\text{AP}_{r}(p)$ for some $p=p(q)$. Let $A_{1},\ldots,A_{q}\subset\mathbb{F}_{q}$ be independently chosen random subsets where each element is included in $A_{i}$ with probability $p$. Let $\mathcal{C}$ be the $\operatorname{RS}[q,rq]$ code. We have $|\mathcal{C}|=q^{rq+1}$. Let us denote

[TABLE]

For $w\in\mathcal{C}$, let $X_{w}$ denote the indicator random variable for the event $\{w\in A_{1}\times\cdots\times A_{q}\}$. Clearly, $X=\sum_{w\in\mathcal{C}}X_{w}$, and for every $w\in\mathcal{C}$, we have $\mathbb{P}(X_{w}=1)=p^{q}$. We first note that for $\epsilon=\omega\left(\frac{\log q}{q}\right)$, and $p=q^{-r}(1-\epsilon)$, we have, using linearity of expectation,

[TABLE]

When $p=q^{-r}(1+\epsilon)$, using a similar calculation as above, we have

[TABLE]

To summarize, for $\epsilon=\omega\left(\frac{\log q}{q}\right)$, and $p=q^{-r}(1-\epsilon)$, ${\rm I\kern-3.00003ptE}[X]\rightarrow 0$, and for $p=q^{-r}(1+\epsilon)$, ${\rm I\kern-3.00003ptE}[X]\rightarrow\infty$.

Lemma 1.6**.**

For $\epsilon=\omega\left(\frac{\log q}{q}\right)$, we have

[TABLE]

Proof.

This is just Markov’s inequality. We have $\mathbb{P}(\text{AP}_{r}(p(1-\epsilon))=0)=\mathbb{P}(X\geq 1)\leq{\rm I\kern-3.00003ptE}[X]\leq q\cdot e^{-\epsilon q}=\exp\left(-\Omega(\epsilon q)\right)$. ∎

This counts for half the proof of the sharp threshold for $\text{AP}_{r}$. The other half forms the main technical contribution of this work. We show the following:

Theorem 1.7**.**

Let $q$ be a prime power, $r=r(q)$ and $\epsilon=\epsilon(q)$ be real numbers such that $q^{-r}\geq\frac{\log q}{q}$ and $\epsilon=\omega\left(\max\left\{q^{-r},\sqrt{q^{r-1}\log\left(q^{1-r}\right)}\right\}\right)$.

Let $A_{1},\ldots,A_{q}$ be independently chosen random subsets of $\mathbb{F}_{q}$ with each point picked independently with probability $q^{-r}(1+\epsilon)$. Then

[TABLE]

There is a technical condition on $\epsilon$ that can be ignored for now, and will be addressed before the proof. The only relevant thing to observe is that when $r$ is bounded away from [math] and $1$, then $\epsilon=\frac{1}{\operatorname{poly}(q)}$ suffices. The condition to focus on here is that $q^{-r}\geq\frac{\log q}{q}$. Indeed, one can see that this condition is necessary to ensure that w.h.p, all the $A_{i}$’s are nonempty. So, for example, if the dimension of $\mathcal{C}$ is $q-1$, then setting $p=q^{-1}(1+\epsilon)$ is enough for ${\rm I\kern-3.00003ptE}[X]=\omega(1)$ but this does not translate to there almost surely being a codeword in $A_{1}\times\cdots\times A_{q}$.

Lemma 1.6 and Theorem 1.7 together give us that $\text{AP}_{r}$ has a sharp threshold at $\max\left\{q^{-r},\frac{\log q}{q}\right\}$ whenever $1-\frac{1}{q}\geq r\gg\frac{1}{\log p}$. For the sake of completeness one could ask if $\text{AP}_{r}$ has a threshold for all feasible values of $r$, and we show that the answer is yes. More formally,

Theorem 1.8** (Sharp threshold for list recovery).**

For every $r=r(q)$, there is a critical $p=p(r,q)$ such that for every $\epsilon>0$,

$\mathbb{P}\left(\text{AP}_{r}(p(1-\epsilon))=1\right)=o(1)$. 2. 2.

$\mathbb{P}\left(\text{AP}_{r}(p(1+\epsilon))=1\right)=1-o(1)$.

The case that is not handled by Theorem 1.7 is when $r=O\left(\frac{1}{\log q}\right)$ (since in this case, Theorem 1.7 requires $\epsilon=\Omega(1)$). This corresponds to the case where $q^{-r}$ is a number bounded away from [math] and $1$.

1.2 What doesn’t work, and why

One obvious attempt to prove Theorem 1.7 is to consider the second moment of $X(=|\mathcal{C}\cap(A_{1}\times\cdots\times A_{q})|)$ and hope that ${\rm I\kern-3.00003ptE}[X^{2}]=(1+o(1)){\rm I\kern-3.00003ptE}^{2}[X]$. Unfortunately, ${\rm I\kern-3.00003ptE}[X^{2}]$ is too large. Through a very careful calculation using the weight distribution of Reed Solomon codes which we do not attempt to reproduce here, we have ${\rm I\kern-3.00003ptE}[X^{2}]=\Omega\left(e^{\frac{1}{p}}{\rm I\kern-3.00003ptE}^{2}[X]\right)$. So in the regime where, for example, $p=q^{-\Omega(1)}$, this approach is unviable.

To understand this (without the aforementioned involved calculation) in an informal manner, let us fix $p=q^{-r}$ for some fixed constant $r$. Let us identify the tuple of sets $(A_{1},\ldots,A_{q})$ with the single set $S=\cup_{i\in[q]}\{(a_{i},z)\;|\;z\in A_{i}\}$. So, we are choosing a random subset $S\subset\mathbb{F}_{q}^{2}$ of size $\approx q^{2-r}$. On the other hand, the objects we are looking for, i.e., codewords, have size $q$. This is much larger than the standard deviation of $|S|$, which is of the order of $q^{1-(r/2)}$. Thus, conditioning on the existence of some codeword $w\subset\mathbb{F}_{q}^{2}$, the distribution of $S$ changes significantly. One way to see this is the following: Using standard Chernoff bounds, one can check that the size of $S$ is almost surely $q^{2-r}\pm O\left(q^{1-(r/2)}\log q\right)$. However, conditioned on $w\in A_{1}\times\cdots\times A_{q}$, the size of $S$ is almost surely $q+q^{-r}(q^{2}-q)\pm O\left(q^{1-(r/2)}\log q\right)$ (the additional $q$ comes from the points that make up $w$). This is much larger than before when $r$ is relatively large. On the other hand, the main point behind (successful) applications of the second moment method is that the distribution does not significantly change after such a conditioning.

One possible way to circumvent the above problem is to pick a uniformly random set $S\subset\mathbb{F}_{q}^{2}$ of size $q^{2-r}$, instead of every point independently with probability $q^{-r}$. This is a closely related distribution, and it is often the case that Theorems in this model are also true in the above ‘i.i.d.’ model. This fact can be also be made formal (see, for example [JuR00] Corollary $1.16$). Here, when one conditions on the existence of some codeword $w$, at least $|S|$ does not change. Thus the second moment method is not ruled out right at the start. However, it seems to be much more technically involved and it is unclear if it is possible to obtain the relatively small threshold interval that is required for Theorem 1.3 in this way.

1.3 What works and how

Here, we sketch the proofs of the Theorem 1.7 and Theorem 1.8, which can be considered the two main technical contributions of this work.

1.3.1 Proof sketch of Theorem 1.7

The key idea in the proof of this theorem is to count the number of polynomials in the ‘Fourier basis’. Let us consider $f:\mathbb{F}_{q}^{q}\rightarrow\{0,1\}$ to be the indicator of $\mathcal{C}$. For $i\in[q]$, let $g_{i}:\mathbb{F}_{q}\rightarrow\{0,1\}$ denote the indicator of $A_{i}$.

For an extremely brief and informal discussion, what we what we want is essentially $\langle f,\prod_{i\in[q]}g_{i}\rangle$, which, by Plancharel’s identity (see Fact A.5) is $\sum_{\alpha}\widehat{f}\cdot\widehat{\prod_{i}g_{i}}(\alpha)$. Since $\mathcal{C}$ is a vector space, we have that $\widehat{f}$ is supported on $\mathcal{C}^{\perp}$. Moreover, $\widehat{g_{i}}(\alpha_{i})$ is much larger when $\alpha_{i}=0$ than when $\alpha_{i}\neq 0$ if $A_{i}$ is random. This combined with the fact that most points in $\mathcal{C}^{\perp}$ have large weight, and a bit more Fourier analysis means that the inner product, $\langle f,\prod_{i}g_{i}\rangle$ is dominated by $\widehat{f}(0)\prod_{i\in[q]}\widehat{g_{i}}(0)$ which is the expected number of codewords in $A_{1}\times\cdots\times A_{q}$.

Now we give a slightly less informal overview. What we are trying to estimate is exactly

[TABLE]

Using Fourier analysis over $\mathbb{F}_{q}$, one can show that

[TABLE]

Using the fact that $\mathcal{C}$ is an $\operatorname{RS}[q,rq]$ code, one has (see Fact A.1) that $\mathcal{C}^{\perp}$ is an $\operatorname{RS}[q,q-rq-1]$ code. What will eventually help in the proof is that the weight distribution of Reed Solomon codes (and so in particular, $\mathcal{C}^{\perp}$) is well understood (see Theorem A.2).

Now clearly, it suffices to understand the term $\sum_{(\alpha_{1},\ldots,\alpha_{q})\in\mathcal{C}^{\perp}}\left(\prod_{i\in[q]}\widehat{g_{i}}(\alpha_{i})\right)=:R$. One way to control $|R|$ is to control $|R|^{2}=R\overline{R}$. Here, one can use the fact that the $A_{i}$’s are randomly and independently chosen to establish cancellation in many terms of ${\rm I\kern-3.00003ptE}[|R|^{2}]$. More formally, one can prove that

[TABLE]

It is a more or less standard fact that if $A_{i}$ is a uniformly random set of size $pq=:t$, then

[TABLE]

and

[TABLE]

for $\alpha_{i}\neq 0$. This difference, will be the reason why $|R|$ is typically much smaller than $\prod_{i\in[q]}\widehat{g_{i}}(0)$. To continue, let us believe the heuristic that most polynomials over $\mathbb{F}_{q}$ of degree $\Theta(q)$ have very few ($o(q)$) zeroes, we can use the rough estimate:

[TABLE]

And so, Markov’s Inequality gives that $|R|$ is unlikely to be much greater than $q^{\frac{rq}{2}}p^{\frac{q}{2}+o(q)}$. On the other hand, with high probability,

[TABLE]

Thus if $p\geq q^{-r+o(1)}$, we have that $(q^{q}/|\mathcal{C}|)\cdot X\geq\prod_{i\in[q]}\widehat{g_{i}}(0)-|R|>0$, and so in particular, $X>0$. The proof of Theorem 1.7 is essentially a much tighter, and more formal version of the above argument, and is postponed to Appendix B.

1.3.2 Proof sketch of Theorem 1.8

The starting point of Thoerem 1.8 is noticing that the only case not covered by Theorem 1.7 is $p\in(0,1)$ is some fixed constant, or equivalently $r=O\left(\frac{1}{\log q}\right)$. Here we have a somewhat crude weight distribution result for Reed Solomon codes (Proposition A.3) to compute the second moment. We first show that ${\rm I\kern-3.00003ptE}[X^{2}]=O\left(e^{\frac{1}{p}}{\rm I\kern-3.00003ptE}^{2}[X]\right)$. Using, for example the Paley-Zygmund Inequality (3), this means that $\mathbb{P}(X>0)\geq\Omega(e^{-\frac{1}{p}})$. Thus we have that $\{X>0\}$ with at least some (possibly small) constant probability. But what we need is that $\mathbb{P}(X>0)\geq 0.99$. For this, we now use the fact that $\text{AP}_{r}$ is monotone, and transitive-symmetric, which informally means that any two variables of $\text{AP}_{r}$ look the same (see Definition A.8 for a formal definition). Standard applications of hypercontractivity for the Boolean hypercube (see Theorem A.9) gives that for $p^{\prime}=p+O\left(\frac{1}{\log q}\right)$, we have that $\mathbb{P}(\text{AP}_{r}(p^{\prime})=1)\geq 0.99$.

The details of this proof are postponed to Appendix C.

One thing to note is that our definition of sharp threshold only makes sense when the critical probability $p_{r}$ is bounded away from $1$ (since otherwise trivially there is some function $\epsilon=\epsilon(q)=o(1)$ such that $p\cdot(1+\epsilon)=1$). So, we will restrict ourselves to the regime where $r=\Omega\left(\frac{1}{\log q}\right)$. Also, it is to be understood that all the statements above (and below) only make sense when $rq$ is an integer, and thus we shall restrict ourselves to this case.

Finally, we address the question of random list recovery with errors as another application of Theorem 1.7.

1.4 Random list recovery with errors

Given a random subset of points in $S\subseteq\mathbb{F}_{q}^{2}$, what is the largest fraction of any degree $d=\Theta(q)$ polynomial that is contained in this set? Using the Union Bound, it is easy to see that no polynomial of degree $d$ has more than $d\log_{\frac{1}{p}}q+o(q)$ points contained in $S$ (formal details are given in Section 3). We show that perhaps unsurprisingly, this is the truth. Formally,

Corollary 1.9**.**

Let $S$ be a randomly chosen subset of $\mathbb{F}_{q}^{2}$ where each point is picked independently with probability $p$. Then with probability $1-o(1)$,

[TABLE]

We restrict our attention to the case when $d=\Theta(q)$, where the above statement is nontrivial. This is the content of Section 3. However, we believe that the statement should hold for all rates, and error (in general) better than $O\left(\frac{q}{\log q}\right)$.

We make two final comments before proceeding to the proofs: (1) In Theorem 1.7, each $A_{i}$ is chosen by including each point independently. However, the same proof works if $A_{i}$ is a uniformly random set with a prescribed size. (2) Although we only state the lower bound for $\text{AC}^{0}[\oplus]$, one can check that all the tools (and, therefore, the lower bound) still work when we replace the $\oplus$ gates with any $\oplus_{p}$ ($\text{MOD}_{p}$) for any small prime $p$.

2 $\text{AC}^{0}[\oplus]$ lower bound for $\text{AP}_{r}$

We prove the lower bound by showing that $\text{AP}_{r}$ solves a biased version of the Coin Problem, and use the lower bounds known for such kinds of functions, obtained by [LSS*+*18], [CHLT19].

Definition 2.1** ($(p,\epsilon)$-coin problem).**

We say that a circuit $C=C^{n}$ on $n$ inputs solves the $(p,\epsilon)$-coin problem if

• •

For $X_{1},\ldots,X_{n}\sim\operatorname{Ber}(p(1-\epsilon))$,

[TABLE]

• •

For $X_{1},\ldots,X_{n}\sim\operatorname{Ber}(p(1+\epsilon))$,

[TABLE]

We shall abbreviate the $(p,\epsilon)$-coin problem on $n$ variables as $\text{CP}^{n}(p,\epsilon)$. We observe that a function $f:\{0,1\}^{n}\rightarrow\{0,1\}$ solves $\text{CP}^{n}\left(p,\epsilon\right)$ if it has a sharp threshold at $p$ with threshold interval at most $\epsilon$. The one obstacle we have to overcome in using Theorem 1.5 is that $\text{AP}_{r}$ has a sharp threshold at $p^{-c}\ll\frac{1}{2}$. However, we will show how to simulate biased Bernoulli r.v’s from almost unbiased ones. Let $\text{C}(s,d)$ to denote the class of functions on $n$ variables which have circuits of size $O(s)=O(s(n))$ and depth $d=d(n)$ using $\land$, $\lor$, $\lnot$, and $\oplus$ gates. Here, we make the following simple observation about the power of $\text{AC}^{0}[\oplus]$ circuits to solve biased and unbiased $\epsilon$-coin problem. First, we observe that it is possible to simulate a biased coin using an unbiased one.

Lemma 2.2**.**

Let $s$ be such that $\frac{1}{2^{s}}\leq p\in(0,1)$, and $\epsilon\leq\frac{1}{s^{K}}$ for a large constant $K$. Then, there is a CNF $F_{p}$ on $t\leq s^{2}$-variables such that for inputs $X_{1}\ldots,X_{t}\in\operatorname{Ber}\left(\frac{1}{2}+\epsilon\right)$,

[TABLE]

and for inputs $X_{1}\ldots,X_{t}\in\operatorname{Ber}\left(\frac{1}{2}-\epsilon\right)$,

[TABLE]

where $L=\lfloor\log_{2}(1/p)\rfloor$.

The idea is essentially that the AND of $k$ unbiased coin is a $2^{-k}$-biased coin. However, some extra work has to be done if we want other biases (say, $(0.15)\cdot 2^{k}$). The proof of this lemma is postponed to the Appendix D.2. This lemma now gives us the following:

Lemma 2.3**.**

Let $z\in(0,1)$ be a fixed constant. If $\text{CP}^{n}\left(\frac{1}{n^{z}},o(\epsilon\log n)\right)\in\text{C}^{n}(s,h)$, then there is a $t\leq\log^{2}n$ such that $\text{CP}^{nt}\left(\frac{1}{2},\epsilon\right)\in\text{C}^{nt}(zs\log n,h+2)$.

Proof.

Let $C$ be a circuit for $\text{CP}^{n}\left(\frac{1}{n^{z}},\delta\right)$-coin problem. Replace each input variable with the CNF $F_{\left(\frac{1}{n^{z}}\right)}$ from Lemma 2.2 on $t=O(\log^{4}n)$ independent variables. Call this circuit $\text{C}^{\prime}$, on $tn$ variables. If the bias of each of these input variables is $\frac{1}{2}+\epsilon$, then the guarantee of Lemma 2.2 is that output of the and gate is $1$ with probability at least $\frac{1}{n^{z}}(1+\Omega(\epsilon\log n))$. A similar computation gives that if the bias of the inputs are $\left(\frac{1}{2}-\epsilon\right)$, then the bias of the output is at most $\frac{1}{n^{z}}(1-\Omega(\epsilon\log n))$. Therefore, $\text{C}^{\prime}$ solves $\text{CP}^{nt}\left(\frac{1}{2},\epsilon\right)$, and has size at most $s\log n$, and depth $h+2$.

∎

Theorem 1.7 and Lemma 1.6, together, now give us the following corollary:

Corollary 2.4**.**

Let $q$ be a large enough prime power. Then $AP_{r}$ on $q^{2}$ inputs solves the $\left(q^{-r},\epsilon\right)$ coin problem, for $\epsilon=\omega\left(\max\left\{q^{-r},q^{\frac{r-1}{3}}\right\}\right)$

As a result, Theorem 1.5, and Lemma 2.2, and Lemma 2.3 together, give us the following bounded depth circuit lower bound for $\text{AP}_{r}$:

Theorem** (1.3, Restated).**

For any $r\in(0,1)$, and $h\in\mathbb{N}$, we have that

[TABLE]

3 Random list recovery with errors

In this section, we shall again consider Reed-Solomon codes $\operatorname{RS}[q,rq]$ where $r$ is some constant between [math] and $1$. Let us slightly abuse notation, as before, and think of a codeword $w\in\operatorname{RS}[q,rq]$ corresponding to a polynomial $p(X)$ as the set of all the zeroes of the polynomial $Y=p(X)$. That is, for a codeword $w=(w_{1},\ldots,w_{q})$ associated with polynomial $p$, we think of $w$ as a subset $\{(a_{i},p(a_{i}))\;|\;i\in[q]\}$ (recall that $\mathbb{F}=\{a_{1},\ldots,a_{q}\}$). For a set of points $S\subset\mathbb{F}_{q}^{2}$ and a codeword $w$ we say the agreement between $w$ and $S$ to denote the quantity $|w\cap S|$. For a code $\mathcal{C}$, we say that the agreement between $\mathcal{C}$ and S to denote $\max_{w\in\mathcal{C}}|m\cap S|$.

We are interested in the following question: For a set $S\subset\mathbb{F}_{q}^{2}$. What is the smallest $\ell$ such that there exists a $w\in\operatorname{RS}[q,rq]$ such that $|w\cap S|\geq q-\ell$? In other words, what is the largest agreement between $\operatorname{RS}[q,rq]$ and $S$? This is (very close to) the list recovery problem for codes with errors. Naturally, we seek to answer this question when $S$ is chosen randomly in an i.i.d. fashion with probability $p$. Theorem 1.7 gives asymptotically tight bounds in a relatively straightforward way for constant error rate.

One can observe that the only properties about Reed-Solomon codes that was used in Theorem 1.7 was the weight distribution in the dual space of codewords. However, (see Appendix A.1) these are also true for punctured Reed-Solomon codes codes. So, an analogus theorem also holds for punctured Reed Solomon codes. Formally,

Theorem 3.1**.**

Let $q,n,d$ be integers such that $q$ is a prime power and $n=\omega(\log q)$, and $q^{-\frac{d}{n}}\geq\frac{\log n}{q}$ and let $\epsilon=\omega\left(\max\left\{q^{-\frac{d}{n}}\sqrt{q^{1-\frac{d}{n}}\log\left(q^{1-\frac{d}{n}}\right)}\right\}\right)$. Let $\mathcal{C}$ be an $\operatorname{RS}[q,d]|_{n}$ code.

Let $A_{1},\ldots,A_{n}$ be independently chosen random subsets of $\mathbb{F}_{q}$ with each point picked independently with probability $q^{-\frac{d}{n}}(1+\epsilon)$. Then

[TABLE]

We do not repeat the proof but is it the exact same as that of Theorem 1.7. Let $\mathcal{E}_{a}$ denote the event that the agreement between $S$ and $\operatorname{RS}[q,rq]$ is $a$. Union bound gives us that

[TABLE]

So if $\ell$ is such that the RHS of 1 is $o(1)$. Then the agreement is almost surely less than $q-\ell$. For the other direction, we have the following corollary:

Corollary 3.2**.**

Let $\epsilon\geq\max\left\{10q^{-\frac{d}{q-\ell}},\sqrt{q^{1-\frac{d}{q-\ell}}\cdot\log q}\right\}$. Let $S$ be a randomly chosen subset of $\mathbb{F}_{q}^{2}$ with each point picked independently with probability at least $q^{-\frac{d}{q-\ell}}(1+\epsilon)$, then with probability at least $1-o(1)$, the agreement between $S$ and $\operatorname{RS}[q,d]$ is at least $q-\ell$.

Proof.

For $i\in[q-\ell]$, let us denote

[TABLE]

Let us use $S^{\prime}:=S_{1}\times\cdots\times S_{q-\ell}$. Let us denote $\mathcal{C}=\operatorname{RS}[q,d]|_{q-\ell}$. Formally, for a codeword $w\in\operatorname{RS}[q,d]$, denote $p_{w}$ to be the polynomial corresponding to $m$. We have

[TABLE]

We observe that the conditions in Theorem 1.7 hold, so

[TABLE]

as desired. ∎

Corollary** (1.9, restated).**

Given a random subset $S\subseteq\mathbb{F}_{q}^{2}$ where each point is picked with probability $p$, then with probability at least $1-o(1)$, the largest agreement $\operatorname{RS}[q,d]$ with $S$ is $d\log_{\frac{1}{p}}q-O\left(\frac{q}{\log\left(\frac{1}{p}\right)}\right)$.

Proof.

Let $a$ be an integer that denotes the maximum agreement between $S$ and $\operatorname{RS}(q,d)$. Suppose that $a\leq d\log_{\frac{1}{p}}q$, then setting $\ell=q-a$, and noting that the conditions for Corollary 3.2 are satisfied, we get that with probability at least $1-o(1)$, there is a polynomial that agrees with the set $S$ in the first $q-\ell$ coordinates. On the other hand, if $a\geq d\log_{\frac{1}{p}}q+4\frac{q}{\log\left(\frac{1}{p}\right)}$, again, setting $\ell=q-a$, Union Bound gives us:

[TABLE]

And so we have that with probability at least $1-o(1)$, the agreement of $\operatorname{RS}(q,d)$ with $S$ is $d\log_{\frac{1}{p}}q-O\left(\frac{q}{\log\left(\frac{1}{p}\right)}\right)$. ∎

4 Conclusion

We started off by attempting to prove a bounded depth circuit lower bound for Andreev’s Problem. This led us into (the decision version of the) random List Recovery of Reed-Solomon codes. Here we show a sharp threshold for a wide range of parameters, with nontrivial threshold intervals in some cases. However, one of the unsatisfactory aspects about Theorem 1.8 is that it is proved in a relatively ‘hands-off’ way possibly resulting in a suboptimal guarantee on $\epsilon$. The obvious open problem that is the following:

Open Problem:

Is Theorem 1.8 with a better bound on $\epsilon$?

If it is true with a much smaller $\epsilon$, it would extend in a straightforward way to the $\text{AC}^{0}[\oplus]$ lower bound as well. Another point we would like to make is that the only thing stopping us from proving Theorem 1.8 for general MDS codes is the lack of Proposition A.11

Acknowledgements

I am extremely grateful to Amey Bhangale, Suryateja Gavva, and Mary Wootters for the helpful discussions. I am especially grateful to Nutan Limaye and Avishay Tal for explaining [LSS*+*18] and [CHLT19] respectively to me. I am grateful to Partha Mukhopadhyay for suggesting the $\text{AC}^{0}[\oplus]$ lower bound problem for list recovery. I am also extremely grateful to Swastik Kopparty for the discussions that led to Corollary 1.9, and to Bhargav Narayanan for the discussions that led to Theorem 1.8.

Appendix A More preliminaries

A.1 Properties of Reed-Solomon codes

The first fact we will use is that the dual vector space of a Reed-Solomon code is also a Reed-Solomon code.

Fact A.1**.**

Let $\mathcal{C}:=\operatorname{RS}[q,d]$. Then $\mathcal{C}^{\perp}=\operatorname{RS}[q,q-d-1]$.

For $t\neq 0$, let $W_{t}$ be the number of codewords of weight $t$ in $\operatorname{RS}[q,d]$. This is a relatively well understood quantity.

Theorem A.2** ([EGS09]).**

We have:

[TABLE]

However, we just need the following slightly weaker bound that is easier to prove:

Proposition A.3**.**

We have $W_{q-i}\leq\frac{q^{d+1}}{i!}$.

Proof.

We have that $\mathcal{C}$ is a $d+1$-dimensional subspace of $\mathbb{F}^{q}$. Add $i$ extra constraints by choosing some set of $i$ coordinates and restricting them to [math]. As long as $i<d$, these new constraints strictly reduce the dimension of $\mathcal{C}$. There are exactly $\binom{q}{i}$ ways to choose the coordinates, and the resulting space has dimension $d+1-i$. Therefore, the number of codewords of weight at most $q-i$ is at most $q^{d+1-i}\cdot\binom{q}{i}\cdot\leq\frac{q^{d+1}}{i!}$. ∎

The above bound is asymptotically only a factor of $e$ away for small values of $i$.

A.1.1 Punctured Reed-Solomon codes

All of the statements above when instead of Reed-Solomon codes, one considers punctured Reed-Solomon codes. For a $w=(w_{1},\ldots,w_{n})\in\mathbb{F}_{q}^{n}$, and a set $S\subset[n^{\prime}]$, let us define

[TABLE]

For a subset $\mathcal{C}\subset\mathbb{F}_{q}^{n^{\prime}}$, let us define

[TABLE]

We call $\operatorname{RS}[q,d]|_{S}$ the $S$-punctured $\operatorname{RS}[q,d]$ code. Let $\mathcal{C}$ denote the $\operatorname{RS}[q,d]|_{n}$ code. Since the properties we will care about are independent of the specific set $S$, let is just parametrize this by $|S|=:n$. The following properties hold

$\mathcal{C}^{\perp}=\operatorname{RS}[q,q-d-1]_{n}$. 2. 2.

Let $W_{i}$ be the number of codewords in $\mathcal{C}$ code of weight $i$. Then we have

[TABLE]

Both facts can be easily checked.

A.2 Basic probability inequalities

We will use the standard (multiplicative) Chernoff bound for sums of i.i.d. Bernoulli random variables. Let $X_{1},\ldots,X_{n}$ be independent $\operatorname{Ber}(p)$ random variables. Let $X:=\sum_{i\in[n]}X_{i}$and denote $\mu={\rm I\kern-3.00003ptE}[X]=np$. Then for any $\epsilon\in(0,1)$, we have:

[TABLE]

We also have (a special case of) the Paley-Zygmund inequality, which states that for a nonnegative random variable $X$,

[TABLE]

A.3 Fourier analysis over $\mathbb{F}_{q}$

For functions $u,v:\mathbb{F}_{q}^{n}\rightarrow\mathbb{C}$, we have a normalized inner product $\langle u,v\rangle:=\frac{1}{q^{n}}\sum_{s\in\mathbb{F}_{q}^{n}}u(s)\overline{v(s)}$. Consider any symmetric, non-degenerate bi-linear map $\chi:\mathbb{F}_{q}^{n}\times\mathbb{F}_{q}^{n}\rightarrow\mathbb{R}/\mathbb{Z}$ (such a map exists). For an $\alpha\in\mathbb{F}_{q}^{n}$, the character function associated with $\alpha$, denoted by $\chi_{\alpha}:\mathbb{F}_{q}^{n}\rightarrow\mathbb{C}$ is given by $\chi_{\alpha}(x)=e^{-2\pi i\chi(\alpha,x)}$.

We have that for all distinct $\alpha,\beta\in\mathbb{F}_{q}$, we have that $\langle\chi_{\alpha},\chi_{\beta}\rangle=0$, and every function $f:\mathbb{F}_{q}\rightarrow\mathbb{C}$ can be written in a unique way as $f(x)=\sum_{\alpha\in\mathbb{F}_{q}}\widehat{f}(\alpha)\chi_{\alpha}(x)$. Here the $\widehat{f}(\alpha)$’s are called the Fourier coefficients, given by

[TABLE]

We will state some facts that we will use in the proof of Theorem 1.7. The interested reader is referred to the excellent book of Tao and Vu [TV06] (chapter 4) for further details.

Fact A.4**.**

For $\mathbb{F}_{q}^{n}\ni\alpha\neq 0$, we have:

[TABLE]

Fact A.5** (Plancherel’s Theorem).**

For functions $f,g:\mathbb{F}^{n}\rightarrow\mathbb{C}$, we have

[TABLE]

Fact A.6**.**

Suppose $g:\mathbb{F}_{q}^{n}\rightarrow\mathbb{C}$ can be written as a product $g(x)=\prod_{i\in[t]}g_{i}(x)$, then we have the Fourier coefficients of $g$ given by:

[TABLE]

Fact A.7**.**

If $g:\mathbb{F}_{q}^{n}\rightarrow\mathbb{C}$ is the indicator of a linear space $\mathcal{C}$, we have:

[TABLE]

A.4 Hypercontractivity and sharp thresholds

Here we state some tools from the analysis of Boolean function that we will use:

Definition A.8**.**

We say that a function $f:\{0,1\}^{n}\rightarrow\{0,1\}$ is transitive-symmetric if for every $i,j\in[n]$, there is a permutation $\sigma\in\mathfrak{S}_{n}$ such that:

$\sigma(i)=j$**

• 2.

$f(x_{\sigma(1)},\ldots,x_{\sigma_{n}})=f(x)$* for all $x\in\{0,1\}^{n}$.*

Let $f:\{0,1\}\rightarrow\{0,1\}$ be a monotone function. We will state an important theorem by Friedgut and Kalai, as stated in the excellent reference [O’D14], regarding sharp thresholds for balanced symmetric monotone Boolean functions. This will be another important tool that we will use.

Theorem A.9** ( [FK96]).**

Let $f:\{0,1\}^{n}\rightarrow\{0,1\}$ be a nonconstant, monotone, transitive-symmetric function and let $F:[0,1]\rightarrow[0,1]$ be the strictly increasing function defined by $F(p)=\mathbb{P}(f(p)=1)$. Let $p_{\text{crit}}$ be the critical probability such that $F(p_{\text{crit}})=1/2$ and assume without loss of generality that $p_{\text{crit}}\leq 1/2$. Fix $0<\epsilon<1/4$ and let

[TABLE]

where $B>0$ is a universal constant. Then assuming $\eta\leq 1/2$,

[TABLE]

We will use an immediate corollary of the above theorem.

Corollary A.10**.**

Let $f:\{0,1\}^{n}\rightarrow\{0,1\}$ be a nonconstant, monotone, transitive-symmetric function. Let $F:[0,1]\rightarrow[0,1]$ be the strictly increasing function defined by $F(p)=\Pr(f(p)=1)$. Let $p$ be such that $F(p)\geq\epsilon$, and let $\eta=B\log(1/\epsilon)\cdot\frac{\log(1/p_{c})}{\log n}$. Then $F(p(1+2\eta))\geq 1-\epsilon$.

In particular, in the above corollary, if for some $\epsilon\in(0,1)$ we have that $F^{-1}(\epsilon)\in(0,1)$, then the function $f$ has a sharp threshold.

One easy observation that will allow us to use Theorem A.9 is the following:

Proposition A.11**.**

The Boolean function $\text{AP}_{r}:\{0,1\}^{q\times n}\rightarrow\{0,1\}$ is transitive-symmetric.

Proof.

For a pair of coordinates indexed by $(i_{1},j_{1})$ and $(i_{2},j_{2})$, it is easy to see that the map $(x,y)\mapsto(x+i_{2}-i_{1},y+j_{2}-j_{1})$ gives us what we need since the set of polynomials is invariant under these operations. ∎

Appendix B Proof of Theorem 1.7

First, we restate the theorem that we will prove in order to make a few more remarks:

Theorem** (Theorem 1.7 restated).**

Let $q$ be a prime power, and $r=r(q)$ and $\epsilon=\epsilon(q)$ be such that $q^{-r}\geq\frac{\log q}{q}$, and let $\epsilon\geq\omega\left(\max\left\{q^{-r},\sqrt{q^{r-1}\log\left(q^{1-r}\right)}\right\}\right)$.

Let $A_{1},\ldots,A_{q}$ be independently chosen random subsets of $\mathbb{F}$ with each point picked independently with probability least $q^{-r}(1+\epsilon)$. Then

[TABLE]

Remarks

Before we proceed to the proof, we first make some simple observations that hopefully make the technical conditions on $q,c,\epsilon$ above seem more natural.

We need $q$ to be a prime power for the existence of $\mathbb{F}_{q}$. 2. 2.

If $r$ is too large, i.e., if $q^{-r}\leq\frac{\log q}{q}(1-\delta)$, for some $\delta>0$, then we will almost surely not contain any codeword. Indeed, we will almost surely have some $i\in[n]$ such that $A_{i}=\emptyset$. 3. 3.

The reason for $\epsilon=\sqrt{q^{r-1}\log\left(q^{1-r}\right)}$ is more or less the same reason as above in that this helps us prove that w.h.p., $|A|$ is not much smaller than expected, as in Claim B.1. This is probably not the best dependence possible, and we make no attempt to optimize. But as $q^{-r}$ gets closer to $\frac{\log q}{q}$, then this condition gets closer to the truth.

We now proceed to the proof.

Proof.

Let us abbreviate $\mathbb{F}=\mathbb{F}_{q}$, denote the subspace $\mathcal{C}\leq\mathbb{F}^{q}$ to be the $\operatorname{RS}[q,rq]$ code. Let $f:\mathbb{F}^{q}\rightarrow\mathbb{C}$ be the indicator of $\mathcal{C}$, i.e., $f(x)=1$ iff $x\in\mathcal{C}$ and [math] otherwise. Let $A_{i}\subset\mathbb{F}$ for $i\in[n]$ and let $g_{i}:\mathbb{F}\rightarrow\{0,1\}$ be the indicator for $A_{i}$. Let us slightly abuse notation and also think of $g_{i}:\mathbb{F}^{q}\rightarrow\mathbb{C}$ which depends only on the $i$’th variable.

We will estimate the quantity $|\mathcal{C}\cap\left(A_{1}\times\cdots\times A_{q}\right)|=q^{q}\langle f,g\rangle$. Setting $|\widehat{f}(0)|=:\rho$, standard steps yield:

[TABLE]

where the first equality is due to Plancherel’s identity, the third inequality is using Fact A.6, the and last equality is because of the fact that $\widehat{g_{i}}(\beta_{i})$ is nonzero only if $\operatorname{supp}(\beta_{i})\subseteq\{i\}$. Let us denote

[TABLE]

For $\alpha\in\mathbb{F}^{q}$, let us define $M(\alpha)=\prod_{i\in[n]}\widehat{g_{i}}(\alpha_{i})$, to be the ‘monomial’ corresponding to $\alpha$. So, we have $R=\sum_{\mathcal{C}^{\perp}\setminus\{\overline{0}\}}M(\alpha)$, and $|R|^{2}=\sum_{\alpha,\beta\in\mathcal{\mathbb{C}}^{\perp}\setminus\{\overline{0}\}}M(\alpha)\overline{M(\beta)}$. By linearity of expectation, we have

[TABLE]

For $\alpha\neq\beta$, let $t$ be a coordinate such that $\alpha_{t}\neq\beta_{t}$. We have:

[TABLE]

The last equality is because at least one of $\alpha_{i}$ or $\beta_{i}$ is nonzero, and $\alpha_{i}-\beta_{i}$ is nonzero. Therefore, for $\alpha\neq\beta$, we have:

[TABLE]

Where the second equality is because $A_{1},\ldots A_{q}$ are chosen independently. For $\alpha=\beta$, it is easy to see that ${\rm I\kern-3.00003ptE}[M(\alpha)\overline{M(\alpha)}]=\prod_{i\in[q]}|\widehat{g_{i}}(\alpha_{i})|^{2}$. So, we have the identity:

[TABLE]

The following two identities are easy to check:

[TABLE]

Equipped with 5, 6, 7, and Proposition A.3, we have:

[TABLE]

Markov’s inequality gives us:

[TABLE]

On the other hand, we have:

Claim B.1**.**

For $q,c$ as given above, let $\epsilon=\omega\left(\sqrt{q^{r-1}\log\left(q^{1-r}\right)}\right)$, and $p=q^{-r}(1+\epsilon)$. Then we have:

[TABLE]

The proof is postponed to the Appendix.

So, using 4, Claim B.1, and 8, and setting $p=q^{-r}(1+\epsilon)$ we have that with probability at least $1-o(1)$,

[TABLE]

Where the inequality follows from the fact that for $x\in[0,1]$,

[TABLE]

It remains to check that if $\epsilon=\omega\left(q^{-r}\right)$, then $\rho^{-1}\langle f,g\rangle>0$, which completes the proof. ∎

Appendix C Proof of Theorem 1.8

Here, we address the case when $r=O\left(\frac{1}{\log q}\right)$. In this case, we observe that Theorem 1.7 does not give us a sharp threshold for the random list recovery since in this case, $p\in(0,1)$. However, this case can be handled by the second moment method and Theorem A.9.

Proof of Theorem 1.8.

Let us use $X$ to denote the number of codewords of $\text{RS}[q,rq]$ contained in a randomly chosen set $S$. Linearity of expectation gives us

[TABLE]

Again, we have that if $p=q^{-r}(1-\epsilon)$, we have that ${\rm I\kern-3.00003ptE}[X]=(1-\epsilon)^{q}$. And so, by Union Bound, we have that with high probability $X=0$. On the other hand, if $p=q^{-r}(1+\epsilon)$, we will show that with high probability, $X>0$. We start off by computing the second moment.

Lemma C.1**.**

We have

[TABLE]

Proof.

Let us denote $\mathcal{C}=\operatorname{RS}[q,rq]$. For every $w\in\mathcal{C}$, let us denote $X_{w}$ for the indicator random variable for the event $\{w\subset S\}$. We have:

[TABLE]

∎

When ${\rm I\kern-3.00003ptE}[X]=\omega(1)$, one can bound second moment by $2e^{\frac{1}{p}}{\rm I\kern-3.00003ptE}^{2}[X]$. The Paley-Zygmund inequality 3 immediately gives:

[TABLE]

Now, Corollary A.10 gives us that if $p\geq q^{-r}\left(1+\omega_{p}\left(\frac{1}{\log q}\right)\right)$, then $\mathbb{P}(X>0)=1-o(1)$.

This, combined with Lemma 1.6 and Theorem 1.7 finishes the proof. ∎

Appendix D Technical lemmas

D.1 Proofs from Section B

Proof of Claim B.1.

For $i\in[n]$, let define the indicator random variables

[TABLE]

and

[TABLE]

Let $Y=\sum_{i}Y_{i}$, and $Z=\sum_{i}Z_{i}$. First off, Chernoff bound gives us that there is a constant $C<\frac{1}{200}$ such that $\mathbb{P}(Y_{i}=1)\leq e^{-C\epsilon^{2}pq}$, so we have ${\rm I\kern-3.00003ptE}[Y]\leq ne^{-C\epsilon^{2}pq}$. Since all the $Y_{i}$’s are independent, Chernoff bound again gives us that

[TABLE]

Moreover, since we have $q^{-r}\geq\frac{\log q}{q}$, we have

[TABLE]

Let us abbreviate $t=2qe^{-C\epsilon^{2}pq}$. Taking 10 and 11 into consideration, we have that with probability at least $1-o(1)$,

[TABLE]

It remains to check that if $\epsilon\gg\sqrt{\frac{\log\left(pq\right)}{pq}}$, then $t+\frac{2t\log q}{\epsilon}\leq 0.1q$. With this in mind, we compute

[TABLE]

which finishes the proof.

∎

D.2 Proofs from Section 2

Proof of Lemma 2.2.

Consider the sequence of integers $\{k_{i}\}_{i\in\mathbb{N}}$ such that for every $i$, $k_{i}$ is the largest such that

[TABLE]

We make a basic observation:

Observation D.1**.**

We have that $k_{1}=\lfloor\log_{2}(1/p)\rfloor\leq s$ and for all $j\geq 2$, we have that $k_{j}\leq 3$.

Let $\ell$ be the largest such that $k_{\ell}>0$ and $\sum_{i\in[\ell]}i\cdot k_{i}<s^{2}$. Let $t=\sum_{i\in[\ell]}i\cdot k_{i}$. Consider the $CNF$ given by

[TABLE]

where the clause $C_{i}^{j}$ is an $\lor$ of $j$ independent variables. We first estimate $p_{\frac{1}{2}}:=\mathbb{P}(C_{p}=1)$ when $X_{1},\ldots,X_{n}\sim\operatorname{Ber}\left(\frac{1}{2}\right)$.

Using the fact that $k_{1}\leq s$ and Observation D.1, we have $\ell=\Omega(\sqrt{t})$. Therefore

[TABLE]

And so for $X_{1},\ldots,X_{n}\sim\operatorname{Ber}\left(\frac{1}{2}+\epsilon\right)$ we bound

[TABLE]

Similarly, for $X_{1},\ldots,X_{n}\sim\operatorname{Ber}\left(\frac{1}{2}-\epsilon\right)$:

[TABLE]

∎