An Adaptive Sublinear-Time Block Sparse Fourier Transform

TL;DR

This paper introduces an adaptive, sublinear-time algorithm for efficiently computing the dominant Fourier coefficients of block sparse signals, reducing sample complexity below traditional bounds and demonstrating the necessity of adaptivity in such measurements.

Contribution

It presents the first sublinear-time sparse FFT algorithm for block sparse signals with improved sample complexity, highlighting the importance of adaptivity in measurement strategies.

Findings

Achieves $O^*(k_0k_1 + k_0 ext{log}(1+k_0) ext{log} n)$ sample complexity.

First sublinear-time algorithm for model-based compressed sensing.

Non-adaptive algorithms require $oldsymbol{ ext{Omega}(k_0k_1 ext{log}(n/k_0k_1)))}$ samples, showing adaptivity's necessity.

Abstract

The problem of approximately computing the $k$ dominant Fourier coefficients of a vector $X$ quickly, and using few samples in time domain, is known as the Sparse Fourier Transform (sparse FFT) problem. A long line of work on the sparse FFT has resulted in algorithms with $O(k\log n\log (n/k))$ runtime [Hassanieh et al., STOC'12] and $O(k\log n)$ sample complexity [Indyk et al., FOCS'14]. These results are proved using non-adaptive algorithms, and the latter $O(k\log n)$ sample complexity result is essentially the best possible under the sparsity assumption alone. This paper revisits the sparse FFT problem with the added twist that the sparse coefficients approximately obey a $(k_0,k_1)$-block sparse model. In this model, signal frequencies are clustered in $k_0$ intervals with width $k_1$ in Fourier space, where $k= k_0k_1$ is the total sparsity. Signals arising in applications are often well approximated by this model with $k_0\ll k$. Our main result is the first sparse FFT algorithm for $(k_0, k_1)$-block sparse signals with the sample complexity of $O^*(k_0k_1 + k_0\log(1+ k_0)\log n)$ at constant signal-to-noise ratios, and sublinear runtime. A similar sample complexity was previously achieved in the works on model-based compressive sensing using random Gaussian measurements, but used $\Omega(n)$ runtime. To the best of our knowledge, our result is the first sublinear-time algorithm for model based compressed sensing, and the first sparse FFT result that goes below the $O(k\log n)$ sample complexity bound. Our algorithm crucially uses {\em adaptivity} to achieve the improved sample complexity bound, and we prove that adaptivity is in fact necessary if Fourier measurements are used: Any non-adaptive algorithm must use $\Omega(k_0k_1\log \frac{n}{k_0k_1})$ samples for the $(k_0,k_1$)-block sparse model, ruling out improvements over the vanilla sparsity assumption.

Equations

$${\hat{X}}_f=\frac{1}{n}\sum _{i\in [n]}{X}_i{\omega }_n^{-ft},f\in [n],$$

$$\Vert \hat{X}-{\hat{X}}^{\prime }{\Vert }_2\le (1+ϵ)\underset{\hat{Y}\text{ is }(k_0,k_1)\text{-block sparse}}{\min }\Vert \hat{X}-\hat{Y}{\Vert }_2$$

$$\mathrm{Err}(\hat{X},k_0,k_1)$$

$$Z^{r}_{j}=\frac{1}{k_{1}}\sum_{i\in[k_{1}]}(G\cdot X^{r})_{j+\frac{n}{k_{1}}\cdot i},\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ j\in\Big{[}\frac{n}{k_{1}}\Big{]}$$

$$\widehat{Z}^{r}_{j}=(\widehat{X}^{r}\star\widehat{G})_{jk_{1}}=\sum_{f\in[n]}\widehat{G}_{f-k_{1}\cdot j}\widehat{X}_{f}\omega_{n}^{a_{r}\cdot f},\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ j\in\Big{[}\frac{n}{k_{1}}\Big{]}$$

$$\frac{\sum_{r\in[2k_{1}]}|\widehat{Z}^{r}_{j}|^{2}}{2k_{1}}\geq(1-\delta)\|\widehat{X}_{I_{j}}\|_{2}^{2}-3\delta\cdot\bigg{(}\|\widehat{X}_{I_{j}\cup I_{j-1}\cup I_{j+1}}\|_{2}^{2}+\delta\sum_{j^{\prime}\in[\frac{n}{k_{1}}]\backslash\{j\}}\frac{\|\widehat{X}_{I_{j^{\prime}}}\|_{2}^{2}}{|j^{\prime}-j|^{F-1}}\bigg{)}.$$

$${\text{Minimize}}_{\{s^r{\}}_{r\in [2k_1]}}\text{leavevmode}\sum _{r\in [2k_1]}{s}^r\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{ subject to }\text{leavevmode}\text{leavevmode}\sum _{\begin{array}{c}j\text{ is covered by }{s}_r\\ \text{ in }{\hat{Z}}^r\text{ for some }r\in [2k_1]\end{array}}\Vert {\hat{X}}_{I_j}{\Vert }_2^2\ge (1-\alpha )\cdot \Vert {\hat{X}}^{\ast }{\Vert }_2^2,$$

$$\begin{split}\mathbb{E}\Big{[}\sum_{r\in[2k_{1}]}s^{r}\Big{]}&\leq\frac{10}{\delta}k_{0}\cdot\log\frac{1}{p}\sum_{r\in[2k_{1}]}\sum_{q=1}^{\log_{2}\frac{10k_{0}}{\delta}}w^{r}_{q}\cdot 10\cdot 2^{q}\\ &=\frac{100k_{0}\log\frac{1}{p}}{\delta}\cdot\frac{1}{1-\delta/(10k_{0})}\sum_{q=1}^{\log_{2}\frac{10k_{0}}{\delta}}\sum_{r\in[2k_{1}]}\frac{\gamma^{r}}{\|\gamma\|_{1}}\\ &\leq 200\frac{k_{0}}{\delta}\log\frac{k_{0}}{\delta}\log\frac{1}{p},\end{split}$$

$$\tilde{S}=\Big{\{}j\in\Big{[}\frac{n}{k_{1}}\Big{]}\,:\,\sum_{r\in[2k_{1}]}\Big{(}|\widehat{Z}^{r}_{j}|^{2}\cdot\frac{\gamma^{r}}{\|\widehat{Z}^{r}\|_{2}^{2}}\Big{)}\geq\delta\cdot\frac{\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}}{k_{0}}\Big{\}}.$$

$$\displaystyle\sum_{r\in[2k_{1}]}\Big{|}\|\widehat{Z}^{r}_{S^{*}}\|_{2}^{2}-\gamma^{r}\Big{|}_{+}\leq 40\delta\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}.$$

$$\Vert \gamma {\Vert }_1\le 10\sum _{r\in [2k_1]}\Vert {\hat{Z}}^r{\Vert }_2^2.$$

$$q_{j}^{r}=\min\bigg{\{}q\in\mathbb{Z}_{+}\,:\,|\widehat{Z}_{j}^{r}|^{2}\geq\frac{\|\widehat{Z}^{r}\|_{2}^{2}}{10\cdot 2^{q}}\bigg{\}},$$

$${\mathbb{P}}_1[j\text{ covered}]$$

$$=\frac{1}{1-\delta /(10k_0)}\sum _{r\in [2k_1]}\sum _{q_j^r\le q\le {\log }_2\frac{10k_0}{\delta }}{2}^{-q}\frac{{\gamma }^r}{\Vert \gamma {\Vert }_1}$$

$$\ge \frac{1}{2}\sum _{r\in [2k_1]:q_j^r\le {\log }_2\frac{10k_0}{\delta }}{2}^{-q_j^r}\frac{{\gamma }^r}{\Vert \gamma {\Vert }_1}$$

$$=\frac{1}{2}\sum _{r\in [2k_1]}{2}^{-q_j^r}\frac{{\gamma }^r}{\Vert \gamma {\Vert }_1}-\frac{1}{2}\sum _{r\in [2k_1]:q_j^r>{\log }_2\frac{10k_0}{\delta }}{2}^{-q_j^r}\frac{{\gamma }^r}{\Vert \gamma {\Vert }_1},$$

$$\begin{array}{rl}\frac{1}{2}\sum _{r\in [2k_1]}{2}^{-q_j^r}\frac{{\gamma }^r}{\Vert \gamma {\Vert }_1}& \ge \frac{10}{4\Vert \gamma {\Vert }_1}\sum _{r\in [2k_1]}\frac{|{\hat{Z}}_j^r{|}^2{\gamma }^r}{\Vert {\hat{Z}}^r{\Vert }_2^2}\\ & \ge \frac{10\delta }{4\Vert \gamma {\Vert }_1}\cdot \frac{{\sum }_{r\in [2k_1]}\Vert {\hat{Z}}^r{\Vert }_2^2}{k_0}\ge \frac{\delta }{4k_0},\end{array}$$

$$\begin{array}{rl}\frac{1}{2}\sum _{r\in [2k_1]:q_j^r>{\log }_2\frac{10k_0}{\delta }}{2}^{-q_j^r}\frac{{\gamma }^r}{\Vert \gamma {\Vert }_1}& \le \frac{1}{2}\sum _{r\in [2k_1]}\frac{\delta }{10k_0}\frac{{\gamma }^r}{\Vert \gamma {\Vert }_1}=\frac{\delta }{20k_0}.\end{array}$$

$$\begin{split}\mathbb{P}[j\text{ covered}]&\geq 1-\bigg{(}1-\frac{\delta}{5k_{0}}\bigg{)}^{\frac{10}{\delta}k_{0}\cdot\log\frac{1}{p}}\geq 1-\exp\bigg{(}-\frac{10}{5}\cdot\log\frac{1}{p}\bigg{)}\geq 1-p,\end{split}$$

$$\mathbb{E}\Big{[}\sum_{r\in[2k_{1}]}\textbf{s}^{(t)}_{r}\Big{]}=O\Big{(}\frac{k_{0}}{\delta}\log\frac{k_{0}}{\delta}\log\frac{1}{\delta p}\Big{)},$$

$$\mathbb{E}\Big{[}\sum_{r\in[2k_{1}]}\textbf{s}_{r}\Big{]}=\mathbb{E}\Big{[}\sum_{r\in[2k_{1}]}\max_{t=1,\dotsc,10\log\frac{1}{p}}\textbf{s}_{r}^{t}\Big{]}\leq\sum_{t=1}^{10\log\frac{1}{p}}\mathbb{E}\Big{[}\sum_{r\in[2k_{1}]}\mathbf{s}_{r}^{(t)}\Big{]}=O\Big{(}k_{0}\log\frac{k_{0}}{\delta}\log\frac{1}{p}\log\frac{1}{\delta p}\Big{)}.$$

$$\mathbb{E}[|L|]=O\Big{(}\frac{k_{0}}{\delta}\log\frac{k_{0}}{\delta}\log\frac{1}{p}\log^{2}\frac{1}{\delta p}\Big{)},$$

$$\displaystyle\sum_{r\in[2k_{1}]}\Big{|}\|\widehat{Z}^{r}_{{S}^{*}}\|_{2}^{2}-\gamma^{r}\Big{|}_{+}\leq 40\delta\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}$$

$$\Vert \gamma {\Vert }_1\le 10\sum _{r\in [2k_1]}\Vert {\hat{Z}}^r{\Vert }_2^2$$

$$\sum _{j\in S^{\ast }\backslash L}\Vert {\hat{X}}_{I_j}^{\prime }{\Vert }_2^2=\sum _{j\in (S^{\ast }\cap \tilde{S})\backslash L}\Vert {\hat{X}}_{I_j}^{\prime }{\Vert }_2^2+\sum _{j\in S^{\ast }\backslash (\tilde{S}\cup L)}\Vert {\hat{X}}_{I_j}^{\prime }{\Vert }_2^2.$$

$$\sum _{j\in S^{\ast }\backslash (\tilde{S}\cup L)}\Vert {\hat{X}}_{I_j}^{\prime }{\Vert }_2^2\le \sum _{j\in S^{\ast }\backslash \tilde{S}}\Vert {\hat{X}}_{I_j}^{\prime }{\Vert }_2^2\le 100\sqrt{\delta }\Vert {\hat{X}}^{\prime }{\Vert }_2^2$$

$$\displaystyle\mathbb{E}\Big{[}\sum_{j\in(S^{*}\cap\tilde{S})\backslash L}\|\widehat{X}^{\prime}_{I_{j}}\|_{2}^{2}\Big{]}$$

$$\displaystyle\leq\sum_{j\in\tilde{S}}\|\widehat{X}^{\prime}_{I_{j}}\|_{2}^{2}\cdot\mathbb{P}\big{[}j\notin L\big{]}.$$

$$\mathbb{P}\big{[}j\notin L\big{]}\leq\delta p,\quad\forall j\in\tilde{S}.$$

$$\mathbb{E}\Big{[}\sum_{j\in(S^{*}\cap\tilde{S})\backslash L}\|\widehat{X}^{\prime}_{I_{j}}\|_{2}^{2}\Big{]}\leq\delta p\cdot\|\widehat{X}^{\prime}\|_{2}^{2},$$

$$\sum _{j\in (S^{\ast }\cap \tilde{S})\backslash L}\Vert {\hat{X}}_{I_j}^{\prime }{\Vert }_2^2\le \delta \cdot \Vert {\hat{X}}^{\prime }{\Vert }_2^2$$

$$U_b=\frac{B}{n}\sum _{i\in [\frac{n}{B}]}{X}_{\sigma (\Delta +j+B\cdot i)}{G}_{j+B\cdot i},j\in [B].$$

$${\hat{U}}_b^{\ast }=\sum _{f\in [n]}{\hat{X}}_f{\hat{G}}_{\sigma f-b\frac{n}{B}}{\omega }_n^{\sigma \Delta f},b\in [B].$$

Full text

This paper revisits the sparse FFT problem with the added twist that the sparse coefficients approximately obey a $(k_{0},k_{1})$-block sparse model. In this model, signal frequencies are clustered in $k_{0}$ intervals with width $k_{1}$ in Fourier space, and $k=k_{0}k_{1}$ is the total sparsity. Signals arising in applications are often well approximated by this model with $k_{0}\ll k$.

Our main result is the first sparse FFT algorithm for $(k_{0},k_{1})$-block sparse signals with a sample complexity of $O^{*}(k_{0}k_{1}+k_{0}\log(1+k_{0})\log n)$ at constant signal-to-noise ratios, and sublinear runtime. A similar sample complexity was previously achieved in the works on model-based compressive sensing using random Gaussian measurements, but used $\Omega(n)$ runtime. To the best of our knowledge, our result is the first sublinear-time algorithm for model based compressed sensing, and the first sparse FFT result that goes below the $O(k\log n)$ sample complexity bound.

Interestingly, the aforementioned model-based compressive sensing result that relies on Gaussian measurements is non-adaptive, whereas our algorithm crucially uses adaptivity to achieve the improved sample complexity bound. We prove that adaptivity is in fact necessary in the Fourier setting: Any non-adaptive algorithm must use $\Omega(k_{0}k_{1}\log\frac{n}{k_{0}k_{1}})$ samples for the $(k_{0},k_{1}$)-block sparse model, ruling out improvements over the vanilla sparsity assumption. Our main technical innovation for adaptivity is a new randomized energy-based importance sampling technique that may be of independent interest.

1 Introduction

The discrete Fourier transform (DFT) is one of the most important tools in modern signal processing, finding applications in audio and video compression, radar, geophysics, medical imaging, communications, and many more. The best known algorithm for computing the DFT of a general signal of length $n$ is the Fast Fourier Transform (FFT), taking $O(n\log n)$ time, which matches the trivial $\Omega(n)$ lower bound up to a logarithmic factor.

In recent years, significant attention has been paid to exploiting sparsity in the signal’s Fourier spectrum, which is naturally the case for numerous of the above applications. By sparse, we mean that the signal can be well-approximated by a small number of Fourier coefficients. Given this assumption, the computational lower bound of $\Omega(n)$ no longer applies. Indeed, the DFT can be computed in sublinear time, while using a sublinear number of samples in the time domain [GIIS14, GST08].

The problem of computing the DFT of signals that are approximately sparse in the Fourier domain has received significant attention in several communities. The seminal work of [CT06a, RV08] in compressive sensing first showed that only $k\log^{O(1)}n$ samples in time domain suffice to recover a length $n$ signal with at most $k$ nonzero Fourier coefficients. A different line of research on the Sparse Fourier Transform (sparse FFT), with origins in computational complexity and learning theory, has resulted in algorithms that use $k\log^{O(1)}n$ samples and $k\log^{O(1)}n$ runtime (i.e., the runtime is sublinear in the length of the input signal). Many such algorithms have been proposed in the literature [GL89, KM91, Man92, GGI*+*02, AGS03, GMS05, Iwe10, Aka10, HIKP12b, HIKP12a, LWC12, BCG*+*12, HAKI12, PR13, HKPV13, IKP14, IK14, Kap16, PS15]; we refer the reader to the recent surveys [GIIS14, GST08] for a more complete overview.

The best known runtime for computing the $k$-sparse FFT is due to Hassanieh et al. [HIKP12a], and is given by $O(k\log n\log(n/k))$, asymptotically improving upon the FFT for all $k=o(n)$. The recent works of [IKP14, Kap16] also show how to achieve a sample complexity of $O(k\log n)$ (which is essentially optimal) in linear time, or in time $k\log^{O(1)}n$ at the expense of $\mathrm{poly}(\log\log n)$ factors. Intriguingly, the aforementioned algorithms are all non-adaptive. That is, these algorithms do not exploit existing samples in guiding the selection of the new samples to improve approximation quality. In the same setting, it is also known that adaptivity cannot improve the sample complexity by more than an $O(\log\log n)$ factor [HIKP12a].

Despite the significant gains permitted by sparsity, designing an algorithm for handling arbitrary sparsity patterns may be overly generic; in practice, signals often exhibit more specific sparsity structures. A common example is block sparsity, where significant coefficients tend to cluster on known partitions, as opposed to being unrestricted in the signal spectrum. Other common examples include tree-based sparsity, group sparsity, and dispersive sparsity [BCDH10, BBC*+*16, EHC15, Bac10].

Such structured sparsity models can be captured via the model-based framework [BCDH10], where the number of sparsity patterns may be far lower than ${n\choose k}$. For the compressive sensing problem, this restriction has been shown to translate into a reduction in the sample complexity, even with non-adaptive algorithms. Specifically, one can achieve a sample complexity of $O(k+\log|\mathcal{M}|)$ with dense measurement matrices based on the Gaussian distribution, where $\mathcal{M}$ is the set of permitted sparsity patterns. Reductions in the sample complexity with other types of measurement matrices, e.g., sparse matrices based on expanders, are typically less [BBC14, IR13]. Other benefits of exploiting model-based sparsity include faster recovery and improved noise robustness [BCDH10, BBC14].

Surprisingly, in stark contrast to the extensive work on exploiting model-based sparsity with general linear measurements, there are no existing sparse FFT algorithms exploiting such structure. This paper presents the first such algorithm, focusing on the special case of block sparsity. Even for this relatively simple sparsity model, achieving the desiderata turns out to be quite challenging, needing a whole host of new techniques, and intriguingly, requiring adaptivity in the sampling.

To clarify our contributions, we describe our model and the problem statement in more detail.

Model and Basic Definitions

The Fourier transform of a signal $X\in\mathbb{C}^{n}$ is denoted by $\widehat{X}$, and defined as

[TABLE]

where $\omega_{n}$ is the $n$-th root of unity. With this definition, Parseval’s theorem takes the form $\|X\|^{2}=n\|\widehat{X}\|_{2}^{2}$.

We are interested in computing the Fourier transform of signals that, in frequency domain, are well-approximated by a block sparse signal with $k_{0}$ blocks of width $k_{1}$, formalized as follows.

Definition 1.1** (Block sparsity).**

Given a sequence $X\in\mathbb{C}^{n}$ and an even block width $k_{1}$, the $j$-th interval is defined as I_{j}=\big{(}(j-1/2)k_{1},(j+1/2)k_{1}\big{]}\cap\mathbb{Z} for j\in\big{[}\frac{n}{k_{1}}\big{]}, and we refer to $\widehat{X}_{I_{j}}$ as the $j$-th block. We say that a signal is $(k_{0},k_{1})$-block sparse if it contains non-zero values within at most $k_{0}$ of these intervals.

Block sparsity is of direct interest in several applications [BCDH10, BCW10]; we highlight two examples here: (i) In spectrum sensing, cognitive radios seek to improve the utilization efficiency in a sparsely used wideband spectrum. In this setting, the frequency bands being detected are non-overlapping and predefined. (ii) Audio signals often contain blocks corresponding to different sounds at different frequencies. Such blocks may be non-uniform, and can be modeled by the $(k,c)$* model* in which $k$ coefficients are arbitrarily spread across $c$ different clusters. It was argued in [CIHB09] that any signal from the $(k,c)$ model is also $(3c,k/c)$-block sparse in the uniform model.

Our goal is to output a list of frequencies and values estimating $\widehat{X}$, yielding an $\ell_{2}$-distance to $\widehat{X}$ not much larger than that of the best $(k_{0},k_{1})$-block sparse approximation. Formally, we say that an output signal $\widehat{X}^{\prime}$ satisfies the $\ell_{2}/\ell_{2}$ block-sparse recovery guarantee if

[TABLE]

for an input parameter $\epsilon>0$.

The sample complexity and runtime of our algorithm are parameterized by the signal-to-noise ratio (SNR) of the input signal, defined as follows.

Definition 1.2** (Tail noise and signal-to-noise ratio (SNR)).**

We define the tail noise level as

[TABLE]

and its normalized version as $\mu^{2}:=\frac{1}{k_{0}}\mathrm{Err}^{2}(\widehat{X},k_{0},k_{1})$, representing the average noise level per block. The signal-to-noise ratio is defined as $\mathrm{SNR}:=\frac{\|\widehat{X}\|^{2}}{\mathrm{Err}^{2}(\widehat{X},k_{0},k_{1})}$.

Throughout the paper, we assume that both $n$ and $k_{1}$ are powers of two. For $n$, this is a standard assumption in the sparse FFT literature. As for $k_{1}$, the assumption comes without too much loss of generality, since one can always round the block size up to the nearest power of two and then cover the original $k_{0}$ blocks with at most $2k_{0}$ larger blocks, thus yielding a near-identical recovery problem other than a possible increase in the SNR. We also assume that $\frac{n}{k_{1}}$ exceeds a large absolute constant; if this fails, our stated scaling laws can be obtained using the standard FFT.

We use $O^{*}(\cdot)$ notation to hide $\log\log\mathrm{SNR}$, $\log\log n$, and $\log\frac{1}{\epsilon}$ factors. Moreover, to simplify the notation in certain lemmas having free parameters that will be set in terms of $\epsilon$, we assume throughout the paper that \epsilon=\Omega\big{(}\frac{1}{\mathrm{poly}\log n}\big{)}, and hence $\log\frac{1}{\epsilon}=O(\log\log n)$. This is done purely for convenience, and since the dependence on $\epsilon$ is not our main focus; the precise expressions with $\log\frac{1}{\epsilon}$ factors are easily inferred from the proofs. Similarly, since the low-SNR regime is not our key focus, we assume that $\mathrm{SNR}\geq 2$, and thus $\log\mathrm{SNR}$ is positive.

Contributions.

We proceed by informally stating our main result; a formal statement is given in Section 5.2.

Theorem 1.1**.**

(Upper bound – informal version)* There exists an adaptive algorithm for approximating the Fourier transform with $(k_{0},k_{1})$-block sparsity that achieves the $\ell_{2}/\ell_{2}$ guarantee for any constant $\epsilon=\Theta(1)$, with a sample complexity of O^{*}\big{(}(k_{0}k_{1}+k_{0}\log(1+k_{0})\log n)\log\mathrm{SNR}), and a runtime of O^{*}\big{(}(k_{0}k_{1}\log^{3}n+k_{0}\log(1+k_{0})\log^{2}n)\log\mathrm{SNR}).*

Note that while we state the result for $\epsilon=\Theta(1)$ here, the dependence on this parameter is explicitly shown in the formal version.

The sample complexity of our algorithm strictly improves upon the sample complexity of $O(k_{0}k_{1}\log n)$ (essentially optimal under the standard sparsity assumption) when $\log(1+k_{0})\log\mathrm{SNR}\ll k_{1}$ and $\log\mathrm{SNR}\ll\log n$ (e.g., $\mathrm{SNR}=O(1)$).

Our algorithm that achieves the above upper bound crucially uses adaptivity. This is in stark contrast with the standard sparse FFT, where we know how to achieve the near-optimal $O(k\log n)$ bound using non-adaptive sampling [IKP14]. While relying on adaptivity can be viewed as a weakness, we provide a lower bound revealing that adaptivity is essential for obtaining the above sample complexity gains. We again state an informal version, which is formalized in Section 6.

Theorem 1.2**.**

(Lower bound – informal version)* Any non-adaptive sparse FFT algorithm that achieves the $\ell_{2}/\ell_{2}$ sparse recovery guarantee with $(k_{0},k_{1})$-block sparsity must use a number of samples behaving as \Omega\big{(}k_{0}k_{1}\log\frac{n}{k_{0}k_{1}}\big{)}.*

To the best of our knowledge, these two theorems provide the first results along several important directions, giving (a) the first sublinear-time algorithm for model-based compressive sensing; (b) the first model-based result with provable sample complexity guarantees in the Fourier setting; (c) the first proven gap between the power of adaptive and non-adaptive sparse FFT algorithms; and (d) the first proven gap between the power of structured (Fourier basis) and unstructured (random Gaussian entries) matrices for model-based compressive sensing.

To see that (d) is true, note that the sample complexity $O(k_{0}\log n+k_{0}k_{1})$ for block-sparse recovery can be achieved non-adaptively using Gaussian measurements [BCDH10], but we show that adaptivity is required in the Fourier setting.

Dependence of our results on SNR. The sample complexity and runtime of our upper bound depend logarithmically on the SNR of the input signal. This dependence is common for sparse FFT algorithms, and even for the case of standard sparsity, algorithms avoiding this dependence in the runtime typically achieve a suboptimal sample complexity [HIKP12b, HIKP12a]. Moreover, to our knowledge, all existing sparse FFT lower bounds consider the constant SNR regime (e.g., [DIPW10, PW11, HIKP12a]).

We also note that our main result, as stated above, assumes that upper bounds on the SNR and the tail noise are known that are tight to within a constant factor (in fact, such tightness is not required, but the resulting bound replaces the true values by the assumed values). These assumptions can be avoided at the expense of a somewhat worse dependence on $\log\mathrm{SNR}$, but we prefer to present the algorithm in the above form for clarity. The theoretical guarantees for noise-robust compressive sensing algorithms often require similar assumptions [FR13].

Our techniques:

At a high level, our techniques can be summarized as follows:

Upper bound. The high-level idea of our algorithm is to reduce the $(k_{0},k_{1})$-block sparse signal of length $n$ to a number of downsampled $O(k_{0})$-sparse signals of length $\frac{n}{k_{1}}$, and use standard sparse FFT techniques to locate their dominant values, thereby identifying the dominant blocks of the original signal. Once the blocks are located, their values can be estimated using hashing techniques. Despite the high-level simplicity, this is a difficult task requiring novel techniques, the most notable of which is an adaptive importance sampling scheme for allocating sparsity budgets to the downsampled signals. Further details are given in Section 2.

Lower bound. Our lower bound for non-adaptive algorithms follows the information-theoretic framework of [PW11], but uses a significantly different ensemble of structured approximately block-sparse signals occupying only a fraction O\big{(}\frac{1}{k_{0}k_{1}}\big{)} of the time domain. Hence, whereas the analysis of [PW11] is based on the difficulty of identifying one of (roughly) ${n\choose k}$ sparsity patterns, the difficulty in our setting is in non-adaptively finding where the signal is non-zero – one must take enough samples to cover the various possible time domain locations. The details are given in Section 6.

Interestingly, our upper bound uses adaptivity to circumvent the difficulty exploited in this lower bounding technique, by first determining where the energy lies, and then concentrating the rest of its samples on the “right” parts of the signal.

Notation:

For an even number $n$, we define [n]:=\big{(}-\frac{n}{2},\frac{n}{2}\big{]}\cap\mathbb{Z}, where $\mathbb{Z}$ denotes the integers. When we index signals having a given length $m$, all arithmetic should be interpreted as returning values in $[m]$ according to modulo-$m$ arithmetic. For $x,y\in\mathbb{C}$ and $\Delta\in\mathbb{R}$, we write $y=x\pm\Delta$ to mean $|y-x|\leq\Delta$. The support of a vector $X$ is denoted by $\mathrm{supp}(X)$. For a number $a\in\mathbb{R}$, we write $|a|_{+}:=\max\{0,a\}$ to denote the positive part of $a$.

Organization:

The paper is organized as follows. In Section 2, we provide an outline of our algorithm and the main challenges involved. We formalize our energy-based importance sampling scheme in Section 3, and provide the corresponding techniques for energy estimation in Section 4. The block-sparse FFT algorithm and its theoretical guarantees are given in Section 5, and the lower bound is presented and proved in Section 6. Several technical proofs are relegated to the appendices.

2 Overview of the Algorithm

One of our key technical contributions consists of a reduction from the $(k_{0},k_{1})$-block sparse recovery problem for signals of length $n$ to $O(k_{0})$-sparse recovery on a set of carefully-defined signals of reduced length $n/k_{1}$, in sublinear time. We outline this reduction below.

A basic candidate reduction to $O(k_{0})$-sparse recovery consists of first convolving $\widehat{X}$ with a filter $\widehat{G}$ whose support approximates the indicator function of the interval $[-k_{1}/2,k_{1}/2]$, and then considering a new signal whose Fourier transform consists of samples of $\widehat{X}\star\widehat{G}$ at multiples of $k_{1}$. The resulting signal $\widehat{Z}$ of length $n/k_{1}$ (a) naturally represents $\widehat{X}$, as every frequency of this sequence is a (weighted) sum of the frequencies in the corresponding block, and (b) can be accessed in time domain using a small number of accesses to $X$ (if $G$ is compactly supported; see below).

This is a natural approach, but its vanilla version does not work: Some blocks in $\widehat{X}$ may entirely cancel out, not contributing to $\widehat{Z}$ at all, and other blocks may add up constructively and contribute an overly large amount of energy to $\widehat{Z}$. To overcome this challenge, we consider not one, but rather $2k_{1}$ reductions: For each $r\in[2k_{1}]$, we apply the above reduction to the shift of $X$ by $r\cdot\frac{n}{2k_{1}}$ in time domain, and call the corresponding vector $Z^{r}$. We show that all shifts cumulatively capture the energy of $X$ well, and the major contribution of the paper is an algorithm for locating the dominant blocks in $\widehat{X}$ from a small number of accesses to the $Z^{r}$’s (via an importance sampling scheme).

Formal definitions: We formalize the above discussion in the following, starting with the notion of a flat filter that approximates a rectangle.

Definition 2.1** (Flat filter).**

A sequence $G\in\mathbb{R}^{n}$ with Fourier transform $\widehat{G}\in\mathbb{R}^{n}$ symmetric about zero is called an $(n,B,F)$-flat filter if (i) $\widehat{G}_{f}\in[0,1]$ for all $f\in[n]$; (ii) \widehat{G}_{f}\geq 1-\big{(}\frac{1}{4}\big{)}^{F-1} for all $f\in[n]$ such that $|f|\leq\frac{n}{2B}$; and (iii) \widehat{G}_{f}\leq\big{(}\frac{1}{4}\big{)}^{F-1}\big{(}\frac{n}{B|f|}\big{)}^{F-1} for all $f\in[n]$ such that $|f|\geq\frac{n}{B}$.

The following lemma, proved in Appendix A.1, shows that it is possible to construct such a filter having $O(FB)$ support in time domain.

Lemma 2.1**.**

(Compactly supported flat filter)* Fix the integers $(n,B,F)$ with $n$ a power of two, $B<n$, and $F\geq 2$ an even number. There exists an $(n,B,F)$-flat filter $\widehat{G}\in\mathbb{R}^{n}$, which (i) is supported on a length-$O(FB)$ window centered at zero in time domain, and (ii) has a total energy satisfying $\sum_{f\in[n]}|\widehat{G}_{f}|^{2}\leq\frac{3n}{B}$.*

Throughout the paper, we make use of the filter construction from Lemma 2.1, except where stated otherwise. To ease the analysis, we assume that $G$ and $\widehat{G}$ are pre-computed and can be accessed in $O(1)$ time. Without this pre-computation, evaluating $\widehat{G}$ is non-trivial, but possible using semi-equispaced Fourier transform techniques (cf., Section 4.2).

With the preceding definition, the set of $2k_{1}$ downsampled signals is given as follows.

Definition 2.2** (Downsampling).**

Given integers $(n,k_{1})$, a parameter \delta\in\big{(}0,\frac{1}{20}\big{)}, and a signal $X\in\mathbb{C}^{n}$, we say that the set of signals $\{Z^{r}\}_{r\in[2k_{1}]}$ with $Z^{r}\in\mathbb{C}^{\frac{n}{k_{1}}}$ is a $(k_{1},\delta)$-downsampling of $X$ if

[TABLE]

for an \big{(}n,\frac{n}{k_{1}},F\big{)}-flat filter with $F=10\log\frac{1}{\delta}$ and support O\big{(}F\frac{n}{k_{1}}\big{)}, where we define $X^{r}_{i}=X_{i+a_{r}}$ with $a_{r}=\frac{nr}{2k_{1}}$. Equivalently, in frequency domain, this can be written as

[TABLE]

by the convolution theorem and the duality of subsampling and aliasing (e.g., see Appendix C.1).

By the assumption of the bounded support of $G$, along with the choice of $F$, we immediately obtain the following lemma, showing that we do not significantly increase the sample complexity by working with $\{Z^{r}\}_{r\in[2k_{1}]}$ as opposed to $X$ itself.

Lemma 2.2**.**

(Sampling the downsampling signals)* Let $\{Z^{r}\}_{r\in[2k_{1}]}$ be a $(k_{1},\delta)$-downsampling of $X\in\mathbb{C}^{n}$ for some $(n,k_{1},\delta)$. Then any single entry $Z_{i}^{r}$ can be computed in O\big{(}\log\frac{1}{\delta}\big{)} time using O\big{(}\log\frac{1}{\delta}\big{)} samples of $X$.*

This idea of using $2k_{1}$ reductions fixes the above-mentioned problem of constructive and destructive cancellations: The $2k_{1}$ reduced signals $Z^{r}$ ($r\in[2k_{1}]$) cumulatively capture all the energy of $X$ well. That is, while the energy $|\widehat{Z}_{j}^{r}|_{2}^{2}$ can vary significantly as a function of $r$, we can tightly control the behavior of the sum $\sum_{r\in[2k_{1}]}|\widehat{Z}_{j}^{r}|_{2}^{2}$. This is formalized in the following.

Lemma 2.3**.**

(Downsampling properties)* Fix $(n,k_{1})$, a parameter \delta\in\big{(}0,\frac{1}{20}\big{)}, a signal $X\in\mathbb{C}^{n}$, and a $(k_{1},\delta)$-downsampling $\{Z^{r}\}_{r\in[2k_{1}]}$ of $X$. The following conditions hold:*

For all $j\in[\frac{n}{k_{1}}]$,

[TABLE] 2. 2.

The total energy satisfies $(1-12\delta)\|\widehat{X}\|_{2}^{2}\leq\frac{\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}}{2k_{1}}\leq 6\|\widehat{X}\|_{2}^{2}.$

The proof is given in Appendix A.2.

Location via sparse FFT: We expect each $Z^{r}$ ($r\in[2k_{1}]$) to be approximately $O(k_{0})$-sparse, as every block contributes primarily to one downsampled coefficient. At this point, a natural step is to run $O(k_{0})$-sparse recovery on the signals $Z^{r}$ to recover the dominant blocks. However, there are too many signals $Z^{r}$ to consider! Indeed, if we were to run $O(k_{0})$-sparse recovery on every $Z^{r}$, we would recover the locations of the blocks, but at the cost of $O(k_{0}k_{1}\log n)$ samples. This precludes any improvement on the vanilla sparse FFT.

It turns out, however, that it is possible to avoid running a $k_{0}$-sparse FFT on all $2k_{1}$ reduced signals, and to instead allocate budgets to them, some of which are far smaller than $k_{0}$, and some of which may be zero. This will be key in reducing the sample complexity.

Before formally defining budget allocation, we present the following definition and lemma, showing that we can use less samples to identify less of the dominant coefficients of a signal, or more samples to identify more dominant coefficients.

Definition 2.3**.**

(Covered frequency) Given an integer $m$, a frequency component $j$ of a signal $\widehat{Z}\in\mathbb{C}^{m}$ is called covered by budget $s$ in the signal $\widehat{Z}$ if $|\widehat{Z}_{j}|^{2}\geq\frac{\|\widehat{Z}\|_{2}^{2}}{s}$.

Lemma 2.4**.**

*(LocateReducedSignal guarantees – informal version) ** There exists an algorithm such that if a signal $X\in\mathbb{C}^{n}$, a set of budgets $\{s^{r}\}_{r\in[2k_{1}]}$, and a confidence parameter $p$ are given to it as input, then it outputs a list that, with probability at least $1-p$, contains any given $j\in[\frac{n}{k_{1}}]$ that is covered by $s^{r}$ in $\widehat{Z}^{r}$ for some $r\in[2k_{1}]$, where $\{\widehat{Z}^{r}\}_{r\in[2k_{1}]}$ denotes the $(k_{1},\delta)$-downsampling of $X$. Moreover, the list size is O\big{(}\sum_{r\in[2k_{1}]}s^{r}\big{)}, the number of samples that the algorithm takes is O\big{(}\sum_{r\in[2k_{1}]}s^{r}\log n\big{)}, and the runtime is O\big{(}\sum_{r\in[2k_{1}]}s^{r}\log^{2}n\big{)}.111 As stated in the formal version, additional terms in the runtime are needed when it comes to subtracting off a current estimate to form a residual signal. *

The formal statement and proof are given in Appendix F, and reveal that $s^{r}$ essentially dictates how many buckets we hash $\widehat{Z}^{r}$ into in order to locate the dominant frequencies (e.g., see [HIKP12a, IKP14]).

Hence, the goal of budget allocation is to approximately solve the following covering problem:

[TABLE]

for a suitable constant $\alpha\in(0,1)$, where $s^{r}$ is the budget allocated to $\widehat{Z}^{r}$, and $\widehat{X}^{*}$ is the best $(k_{0}.k_{1})$-block sparse approximation of $\widehat{X}$. That is, we want to minimize the total budget while accounting for a constant proportion of the signal energy.

Challenges in budget allocation: Allocating the budgets is a challenging task, as each block in the spectrum of the signal may have very different energy concentration properties in time domain, or equivalently, different variations in $|\widehat{Z}_{j}^{r}|^{2}$ as a function of $r$. To see this more concretely, in Figure 1, we show three hypothetical examples of such variations, in the case that $k_{0}=2k_{1}=6$ and all of the blocks have equal energy, leading to equal column sums in the matrices.

In the first example, each block contributes to a different $Z^{r}$, and thus the blocks could be located by running $1$-sparse recovery separately on the $2k_{1}$ signals. In stark contrast, in the second example, each block contributes equally to each $Z^{r}$, so we would be much better off running $k_{0}$-sparse recovery on a single (arbitrary) $Z^{r}$. Finally, in the third example, the best budget allocation scheme is completely unclear by inspection alone! We need to design an allocation scheme to handle all of these cases, and to do so without even knowing the structure of the matrix.

While the examples in Figure 1 may seem artificial, and are not necessarily feasible with the exact values given, we argue in Appendix E that situations exhibiting the same general behavior are entirely feasible.

Importance sampling: Our solution is to sample $r$ values with probability proportional to an estimate of $\|\widehat{Z}^{r}\|_{2}^{2}$, and sample sparsity budgets from a carefully defined distribution (see Section 3, Algorithm 1). We show that sufficiently accurate estimates of $\|\widehat{Z}^{r}\|_{2}^{2}$ for all $r\in[2k_{1}]$ can be obtained using $O(k_{0}k_{1})$ samples of $X$ via hashing techniques (cf., Section 4); hence, what we are essentially doing is using these samples to determine where most of the energy of the signal is located, and then favoring the parts of the signal that appear to have more energy. This is exactly the step that makes our algorithm adaptive, and we prove that it produces a total budget in (3) of the form $O(k_{0}\log(1+k_{0}))$, on average.

Ideally, one would hope to solve (3) using a total budget of $O(k_{0})$, since there are only $k_{0}$ blocks. However, the $\log(1+k_{0})$ factor is not an artifact of our analysis: We argue in Appendix E that very different techniques would be needed to remove it in general. Specifically, we design a signal $X$ for which the optimal solution to (3) indeed satisfies $\sum_{r\in[2k_{1}]}s^{r}=\Omega(k_{0}\log(1+k_{0}))$.

Iterative procedure and updating the residual: The techniques described above allow us to recover a list of blocks that contribute a constant fraction (e.g., $0.9$) of the signal energy. We use $O(\log\mathrm{SNR})$ iterations of our main procedure to reduce the SNR to a constant, and then achieve $(1+\epsilon)$-recovery with an extra “clean-up” step. Most of the techniques involved in this part are more standard, with a notable exception: Running a standard sparse FFT with budgets $s^{r}$ on the reduced space (i.e., on the vectors $Z^{r}$) is not easy to implement in $k_{0}k_{1}\mathrm{poly}(\log n)$ time when $Z^{r}$ are the residual signals. The natural approach is to subtract the current estimate $\widehat{\chi}$ of $\widehat{X}$ from our samples and essentially run on the residual, but subtraction in $k_{0}k_{1}\mathrm{poly}(\log n)$ time is not straightforward to achieve. Our solution crucially relies on a novel block semi-equispaced FFT (see Section 4.2), and the idea of letting the location primitives in the reduced space operate using common randomness (see Appendix F).

3 Location via Importance Sampling

As outlined above, our approach locates blocks by applying standard sparse FFT techniques to the downsampled signals arising from Definition 2.2. In this section, we present the techniques for assigning the corresponding sparsity budgets (cf., (3)).

We use a novel procedure called energy-based importance sampling, which approximately samples $r$ values with probability proportional to $\|\widehat{Z}^{r}\|^{2}$. Since these energies are not known exactly, we instead sample proportional to a general vector $\gamma=(\gamma^{1},\dotsc,\gamma^{2k_{1}})$, where we think of $\gamma^{r}$ as approximating $\|\widehat{Z}^{r}\|^{2}$. The techniques for obtaining these estimates are deferred to Section 4.

The details are shown in Algorithm 1, where we repeatedly sample from the distribution $w_{q}^{r}$, corresponding to independently sampling $r$ proportional to $\gamma^{r}$, and $q$ from a truncated geometric distribution. The resulting sparsity level to apply to $Z^{r}$ is selected to be $s^{r}=10\cdot 2^{q}$.

According to Definition 2.3, $s^{r}=10\cdot 2^{q}$ covers any given frequency $j$ for which $|\widehat{Z}^{r}_{j}|^{2}\geq\frac{\|\widehat{Z}^{r}\|_{2}^{2}}{10\cdot 2^{q}}$. The intuition behind sampling $q$ proportional to $2^{-q}$ is that this gives a high probability of producing small $q$ values to cover the heaviest signal components, while having a small probability of producing large $q$ values to cover the smaller signal components. We only want to do the latter rarely, since it costs significantly more samples.

We first bound the expected total sum of budgets returned by BudgetAllocation.

Lemma 3.1**.**

(BudgetAllocation budget guarantees)* For any integers $k_{0}$ and $k_{1}$, any positive vector $\gamma\in\mathbb{R}^{2k_{1}}$, and any parameters p\in\big{(}0,\frac{1}{2}\big{)} and $\delta\in(0,1)$, if the procedure BudgetAllocation in Algorithm 1 is run with inputs $(\gamma,k_{0},k_{1},\delta,p)$, then the expected value of the total sum of budgets returned, $\{s^{r}\}_{r\in[2k_{1}]}$, satisfies \mathbb{E}\big{[}\sum_{r\in[2k_{1}]}s^{r}\big{]}\leq 200\,\frac{k_{0}}{\delta}\log\frac{k_{0}}{\delta}\log\frac{1}{p}. The runtime of the procedure is O\big{(}\frac{k_{0}}{\delta}\log\frac{1}{p}+k_{1}\big{)}.*

Proof.

Each time a new $(r,q)$ pair is sampled, the sum of the $s^{r}$ values increases by at most $10\cdot 2^{q}$, and hence the overall expected sum is upper bounded by the number of trials $10k_{0}\log\frac{1}{p}$ times the expected value of $10\cdot 2^{q}$ for a single trial:

[TABLE]

where the second line follows from the definition of $w_{q}^{r}$, and the third line follows from $\frac{\delta}{10k_{0}}\leq\frac{1}{2}$ (since $\delta\leq 1$) and $\sum_{r\in[2k_{1}]}\frac{\gamma^{r}}{\|\gamma\|_{1}}=1$.

Runtime:

Note that sampling from $w_{q}^{r}$ amounts to sampling $q$ and $r$ values independently, and the corresponding alphabet sizes are O\big{(}\log\frac{k_{0}}{\delta}\big{)} and $O(k_{1})$ respectively. The stated runtime follows since we take O\big{(}\frac{k_{0}}{\delta}\log\frac{1}{p}\big{)} samples, and sampling from discrete distributions can be done in time linear in the alphabet size and number of samples [HMM93]. The second loop in Algorithm 1 need not be done explicitly, since the maximum $q$ value can be updated after taking each sample. ∎

As we discussed in Section 2, the $\log k_{0}$ term in the number of samples would ideally be avoided; however, in Appendix E.2, we argue that even the optimal solution to (3) can contain such a factor.

We now turn to formalizing the fact that the budgets returned by BudgetAllocation are such that most of the dominant blocks are found. To do this, we introduce the following notion.

Definition 3.1** (Active frequencies).**

Given $(n,k_{0},k_{1})$, a signal $X\in\mathbb{C}^{n}$, a parameter $\delta\in(0,1)$, and a $(k_{1},\delta)$-downsampling $\{Z^{r}\}_{r\in[2k_{1}]}$ of $X$, the set of active frequencies $\tilde{S}$ is defined as

[TABLE]

Observe that if $\gamma^{r}=\|\widehat{Z}^{r}\|_{2}^{2}$, this reduces to $\sum_{r\in[2k_{1}]}|\widehat{Z}^{r}_{j}|^{2}\geq\delta\cdot\frac{\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}}{k_{0}}$, thus essentially stating that the sum of the energies over $r\in[2k_{1}]$ for the given block index $j$ is an \Omega\big{(}\frac{\delta}{k_{0}}\big{)} fraction of the total energy. Combined with Lemma 2.3, this roughly amounts to $\|\widehat{X}_{I_{j}}\|_{2}^{2}$ exceeding an \Omega\big{(}\frac{\delta}{k_{0}}\big{)} fraction of $\|\widehat{X}\|_{2}^{2}$.

To formalize and generalize this intuition, the following lemma states that the frequencies within $\tilde{S}$ account for most of the energy in $X$, as long as each $\gamma^{r}$ approximates $\|\widehat{Z}^{r}\|_{2}^{2}$ sufficiently well.

Lemma 3.2**.**

(Properties of active frequencies)* Fix $(n,k_{0},k_{1})$, a parameter \delta\in\big{(}0,\frac{1}{20}\big{)}, a signal $X\in\mathbb{C}^{n}$, and a $(k_{1},\delta)$-downsampling $\{Z^{r}\}_{r\in[2k_{1}]}$ of $X$. Moreover, fix an arbitrary set S^{*}\subseteq\big{[}\frac{n}{k_{1}}\big{]} of cardinality at most $10k_{0}$, and a vector $\gamma\in\mathbb{R}^{2k_{1}}$ satisfying*

[TABLE]

Fix the set of active frequencies $\tilde{S}$ according to Definition 4, and define the signal $\widehat{X}_{\tilde{S}}$ to equal $\widehat{X}$ on all intervals $\{I_{j}\,;j\in\tilde{S}\}$ (see Definition 1.1), and zero elsewhere. Then $\|\widehat{X}_{S^{*}\backslash\tilde{S}}\|_{2}^{2}\leq 100\sqrt{\delta}\|\widehat{X}\|_{2}^{2}$.

The proof of Lemma 3.2 is given in Appendix B.1.

What remains is to show that if $j$ is active, then $j$ is covered by some $s^{r}$ in $\widehat{Z}^{r}$ with high constant probability upon running Algorithm 1. This is formulated in the following.

Lemma 3.3**.**

(BudgetAllocation covering guarantees)* Fix $(n,k_{0},k_{1})$, the parameters $\delta\in(0,1)$ and p\in\big{(}0,\frac{1}{2}\big{)}, a signal $X\in\mathbb{C}^{n}$, and a $(k_{1},\delta)$-downsampling $\{Z^{r}\}_{r\in[2k_{1}]}$ of $X$. Moreover, fix a vector $\gamma\in\mathbb{R}^{2k_{1}}$ satisfying*

[TABLE]

Suppose that BudgetAllocation in Algorithm 1 is run with inputs $(\gamma,k_{0},k_{1},\delta,p)$, and outputs the budgets $\{s^{r}\}_{r\in[2k_{1}]}$. Then for any active $j$ (i.e., $j\in\tilde{S}$ as per Definition 3.1), the probability that there exists some $r\in[2k_{1}]$ such that $j$ is covered by $s^{r}$ in $\widehat{Z}^{r}$ is at least $1-p$.

Proof.

Recall from Definition 2.3 that if a pair $(r,q)$ is sampled in the first loop of BudgetAllocation, then $j$ is covered provided that $|\widehat{Z}^{r}_{j}|^{2}\geq\frac{\|\widehat{Z}^{r}\|_{2}^{2}}{10\cdot 2^{q}}.$ We therefore define

[TABLE]

and note that the event described in the lemma statement is equivalent to some pair $(r,q)$ being sampled with $q^{r}_{j}\leq q$. Note that due to the range of $q$ from which we sample (cf., Algorithm 1), this can only occur if $q^{r}_{j}\leq\log_{2}\frac{10k_{0}}{\delta}$.

Taking a single sample: We first compute the probability of being covered for a single random sample of $(q,r)$, denoting the corresponding probability by $\mathbb{P}_{1}[\cdot]$. Recalling from line 4 of Algorithm 1 that we sample each $(q,r)$ with probability $w_{q}^{r}=\frac{2^{-q}}{1-\delta/(10k_{0})}\frac{\gamma^{r}}{\|\gamma\|_{1}}$, we obtain

[TABLE]

where the third line follows since $\delta/(10k_{0})\leq\frac{1}{2}$ due to the assumption that $\delta\leq 1$.

Bounding the first term in (7): Observe from (6) that $2^{-q_{j}^{r}}\geq\frac{1}{2}\frac{10|\widehat{Z}_{j}^{r}|^{2}}{\|\widehat{Z}^{r}\|_{2}^{2}}$, and recall the definition of being active in (4). Combining these, we obtain the following when $j$ is active:

[TABLE]

where the last inequality follows from the assumption on $\|\gamma\|_{1}$ in the lemma statement.

Bounding the second term in (7): We have

[TABLE]

Hence, we deduce from (7) that $\mathbb{P}_{1}[j\text{ covered}]\geq\frac{\delta}{5k_{0}}$.

Taking multiple independent samples: Since the sampling is done $\frac{10}{\delta}k_{0}\cdot\log\frac{1}{p}$ times independently, the overall probability of an active block $j$ being covered satisfies

[TABLE]

where we have applied the inequality $1-\zeta\leq e^{-\zeta}$ for $\zeta\geq 0$. ∎

3.1 The Complete Location Algorithm

In Algorithm 2, we give the details of MultiBlockLocate, which performs the above-described energy-based importance sampling procedure, runs the sparse FFT location algorithm (see Appendix F) with the resulting budgets, and returns a list $L$ containing the block indices that were identified.

MultiBlockLocate calls two primitives that are defined later in the paper, but their precise details are not needed in order to understand the location step:

• •

EstimateEnergies (see Section 4.4) provides us with a vector $\gamma$ providing a good approximation of each $\|\widehat{Z}^{r}\|_{2}^{2}$, in the sense of satisfying the preconditions of Lemmas 3.2 and 3.3;

• •

LocateReducedSignals (see Appendix F) accepts the sparsity budgets $\{s^{r}\}$ and runs a standard $s^{r}$-sparse fast FFT algorithm on each downsampled signal $Z^{r}$ in order to locate the dominant frequencies.

Note that in addition to $X$, these procedures accept a second signal $\widehat{\chi}$; this becomes relevant when we iteratively run the block sparse FFT (cf., Section 5), representing previously-estimated components that are subtracted off to produce a residual.

The required guarantees on LocateReducedSignals are given in Lemma 2.4 (and more formally in Appendix E.2), and in order to prove our main result on MultiBlockLocate, we also need the following lemma ensuring that we can compute energy estimates satisfying the preconditions of Lemmas 3.2 and 3.3; the procedure and proof are presented in Section 4.4.

Lemma 3.4**.**

(EstimateEnergies guarantees)* Given $(n,k_{0},k_{1})$, the signals $X\in\mathbb{C}^{n}$ and $\widehat{\chi}\in\mathbb{C}^{n}$ with $\|\widehat{X}-\widehat{\chi}\|_{2}^{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$, and the parameter \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)}, the procedure EstimateEnergies$(X,\widehat{\chi},n,k_{0},k_{1}$,$\delta)$ returns a vector $\gamma\in\mathbb{R}^{2k_{1}}$ such that, for any given set $S^{*}$ of cardinality at most $10k_{0}$, we have the following with probability at least $\frac{1}{2}$:*

\sum_{r\in[2k_{1}]}\Big{|}\|\widehat{Z}^{r}_{S^{*}}\|_{2}^{2}-\gamma^{r}\Big{|}_{+}\leq 40\delta\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}; 2. 2.

$\|\gamma\|_{1}\leq 10\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}$;

where $\{Z^{r}\}_{r\in[2k_{1}]}$ is the $(k_{1},\delta)$-downsampling of $X-\chi$ (see Definition 2.2).

Moreover, if $\widehat{\chi}$ is $(O(k_{0}),k_{1})$-block sparse, then the sample complexity is $O(\frac{k_{0}k_{1}}{\delta^{2}}\log\frac{1}{\delta}\log\frac{1}{\delta p})$, and the runtime is $O(\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}\frac{1}{\delta}\log^{2}n)$.

Remark 3.1**.**

The preceding lemma ensures that the $\gamma^{r}$ provide good approximations of $\|\widehat{Z}^{r}\|_{2}^{2}$ in a “restricted” and “one-sided” sense, while not over-estimating the total energy by more than a constant factor. Specifically, the first part concerns the energy of $\widehat{Z}^{r}$ restricted to a fixed set of size $O(k_{0})$, and characterizes the extent to which the energies are under-estimated. It appears to be infeasible to characterize over-estimation in the same way (e.g., replacing $|\cdot|_{+}$ by $|\cdot|$), since several of the samples could be overly large due to spiky noise.

Remark 3.2**.**

Here and subsequently, the $\mathrm{poly}(n)$ lower bounds regarding $(\widehat{X},\widehat{\chi})$ are purely technical, resulting from extremely small errors when subtracting off $\widehat{\chi}$. See Section 4.2 for further details.

We are now in a position to provide our guarantees on MultiBlockLocate, namely, on the behavior of the list size, and on the energy that the components in the list capture. Note that the output of MultiBlockLocate is random, since the same is true of EstimateEnergies, BudgetAllocation, and LocateReducedSignals.

Lemma 3.5**.**

(MultiBlockLocate guarantees)* Given $(n,k_{0},k_{1})$, the parameters \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)} and p\in\big{(}\frac{1}{n},\frac{1}{2}\big{)}, and the signals $X\in\mathbb{C}^{n}$ and $\chi\in\mathbb{C}^{n}$ with $\widehat{\chi}_{0}$ uniformly distributed over an arbitrarily length-$\frac{\|\widehat{\chi}\|^{2}}{\mathrm{poly}(n)}$ interval, the output $L$ of the function MultiBlockLocate$(X,\widehat{X},k_{1},k_{0},n,\delta,p)$ has the following properties for any set $S^{*}$ of cardinality at most $10k_{0}$:*

\mathbb{E}\big{[}|L|\big{]}=O\big{(}\frac{k_{0}}{\delta}\log\frac{k_{0}}{\delta}\log\frac{1}{p}\log^{2}\frac{1}{\delta p}\big{)}; 2. 2.

$\sum_{j\in S^{*}\backslash L}\|(\widehat{X}-\widehat{\chi})_{I_{j}}\|_{2}^{2}\leq 200\sqrt{\delta}\|\widehat{X}-\widehat{\chi}\|_{2}^{2}$* with probability at least $1-p$.*

Moreover, if $\widehat{\chi}$ is $(O(k_{0}),k_{1})$-block sparse, and we have \delta=\Omega\big{(}\frac{1}{\mathrm{poly}(\log n)}\big{)} and p=\Omega\big{(}\frac{1}{\mathrm{poly}(\log n)}\big{)}, then (i) the expected sample complexity is O^{*}\big{(}\frac{k_{0}}{\delta}\log(1+k_{0})\log n+\frac{k_{0}k_{1}}{\delta^{2}}\big{)}, and the expected runtime is O^{*}\big{(}\frac{k_{0}}{\delta}\log(1+k_{0})\log^{2}n+\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}n+\frac{k_{0}k_{1}}{\delta}\log^{3}n\big{)}; (ii) if the procedure returns $L$, then we are guaranteed that the algorithm used O^{*}\big{(}|L|\cdot\log n+\frac{k_{0}k_{1}}{\delta^{2}}\big{)} samples and O^{*}\big{(}|L|\cdot\log^{2}n+\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}n+\frac{k_{0}k_{1}}{\delta}\log^{3}n\big{)} runtime.

Remark 3.1**.**

The procedure MultiBlockLocate is oblivious to the choice of $S^{*}$ in this lemma statement.

Proof.

First claim: Note that in each iteration of the outer loop when we run BudgetAllocation$(\gamma,k_{0},k_{1},\delta,\frac{1}{2}\delta p)$, Lemma 3.1 implies that for any $t$, the following holds true:

[TABLE]

where $\textbf{s}^{(t)}_{r}$ is the $r$-th entry of the budget allocation vector $\mathbf{s}^{t}$ at iteration $t$. Therefore,

[TABLE]

We now apply Lemma 2.4, which is formalized in Appendix F; the assumption \max_{r\in[2k_{1}]}\textbf{s}_{r}=O\big{(}\frac{k_{0}}{\delta}\big{)} therein is satisfied due to the range of $q$ from which we sample in BudgetAllocation. We set the target success probability to $1-\frac{1}{2}\delta p$, which guarantees that the size of the list returned by the function LocateReducedSignals is O\big{(}\sum_{r\in[2k_{1}]}\textbf{s}_{r}\log\frac{1}{\delta p}\big{)}. Therefore, by (8), we have

[TABLE]

yielding the first statement of the lemma.

Second claim: Let $X^{\prime}=X-\chi$, and consider the set $S^{*}$ given in the lemma statement, and an arbitrary iteration $t$. By Lemma 3.4 in Section 4.4 (also stated above), the approximate energy vector $\gamma$ in any given iteration of the outer loop satisfies

[TABLE]

with probability at least $\frac{1}{2}$. When this is the case, the vector $\gamma$ meets the requirements of Lemmas 3.2 and 3.3. That means that the probability of having an energy estimate $\gamma$ that meets these requirements in at least one iteration is lower bounded by $1-(\frac{1}{2})^{10\log\frac{1}{p}}\geq 1-p$.

We now consider an arbitrary iteration in which the above conditions on $\gamma$ are satisfied. We write

[TABLE]

The second term is bounded by

[TABLE]

by Lemma 3.2, which uses the first condition on $\gamma$ in (9).

We continue by calculating the expected value of the first term in (10) with respect to the randomness of BudgetAllocation and LocateReducedSignals:

[TABLE]

We thus consider the probability $\mathbb{P}[j\notin L]$ for an arbitrary $j\in\tilde{S}$. If $j\in\tilde{S}$, then by Lemma 3.3 and the choice of the final parameter of $\frac{1}{2}\delta p$ passed to BudgetAllocation, there is at least one $r\in[2k_{1}]$ such that $j$ is covered, with probability at least $1-\frac{1}{2}\delta p$. We also know from Lemma 2.4 that the failure probability of LocateReducedSignals for some covered $j$ is at most $\frac{1}{2}\delta p$. A union bound on these two events gives

[TABLE]

Hence, we deduce from (12) that

[TABLE]

and Markov’s inequality gives

[TABLE]

with probability at least $1-p$. Combining this with (10)–(11), and using the assumption $\delta\leq\frac{1}{20}$ to write $\delta\leq 100\sqrt{\delta}$, we complete the proof.

Sample complexity and runtime: We first consider the sample complexity and runtime as a function of the output $L$.

There are two operations that cost us samples. The first is the call to EstimateEnergies, which costs $O(\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}\frac{1}{\delta})$ by Lemma 3.4. The second is the call to LocateReducedSignals; by Lemma 2.4 in Appendix F, with $\delta p$ in place of $p$, this costs O\big{(}\sum_{r\in[2k_{1}]}s^{r}\log\frac{1}{\delta p}\log\frac{1}{\delta}\log n\big{)} samples (recall that $\widehat{\chi}$ is $(O(k_{0}),k_{1})$-block sparse by assumption), which is O\big{(}|L|\log\frac{1}{\delta p}\log\frac{1}{\delta}\log n\big{)}. Adding these contributions gives the desired result; the $\log\frac{1}{p}$ and $\log\frac{1}{\delta}$ factors are hidden in the $O^{*}(\cdot)$ notation, since we have assumed that $\delta$ and $p$ behave as \Omega\big{(}\frac{1}{\mathrm{poly}\log n}\big{)}.

The time complexity follows by the a similar argument, with EstimateEnergies costing $O(\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}\frac{1}{\delta}\log^{2}n)$ by Lemma 3.4, and the call to LocateReducedSignals costing O\big{(}|L|\log\frac{1}{\delta p}\log\frac{1}{\delta}\log^{2}n+\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta p}\log^{3}n\big{)} by Lemma 2.4 in Appendix F. The complexity of EstimateEnergies dominates that of calling BudgetAllocation, which is O\big{(}k_{1}+\frac{k_{0}}{\delta}\log\frac{1}{p}\big{)} by Lemma 3.1.

The expected sample complexity and runtime follow directly from those depending on $L$, by simply substituting the expectation of $|L|$ given in the lemma statement. ∎

4 Energy Estimation

In this section, we provide the energy estimation procedure used in the MultiBlockLocate procedure in Algorithm 2, and prove its guarantees that were used in the proof of Lemma 3.5. To do this, we introduce a variety of tools needed, including hashing and the semi-equispaced FFT. While such techniques are well-established for the standard sparsity setting [IKP14], applying the existing semi-equispaced FFT algorithms separately for each $Z^{r}$ in our setting would lead to a runtime of $k_{0}k_{1}^{2}\mathrm{poly}(\log n)$. Our techniques allow us to compute the required FFT values for all $r$ in $k_{0}k_{1}\mathrm{poly}(\log n)$ time, as we detail in Section 4.2.

4.1 Hashing Techniques

The notion of hashing plays a central role in our estimation primitives, and in turn makes use of random permutations.

Definition 4.1** (Approximately pairwise-independent permutation).**

Fix $n$, and let $\pi\,:\,[n]\to[n]$ be a random permutation. We say that $\pi$ is approximately pairwise-independent if, for any $i,i^{\prime}\in[n]$ and any integer $t$, we have $\mathbb{P}[|\pi(i)-\pi(i^{\prime})|\leq t]\leq\frac{4t}{n}$.

It is well known that such permutations exist in the form of a simple modulo-$n$ multiplication; we will specifically use the following lemma from [IK14].

Lemma 4.1**.**

(Choice of permutation [IK14, Lemma 3.2])* Let $n$ be a power of two, and define $\pi(i)=\sigma\cdot i$, where $\sigma$ is chosen uniformly at random from the odd numbers in $[n]$. Then $\pi$ is an approximately pairwise-independent random permutation.*

We now turn to the notion of hashing a signal into buckets. We do this by applying the random permutation from Lemma 4.1 along with a random shift in time domain, and then applying a suitable filter according to Definition 2.1.

Definition 4.2** (Hashing).**

Given integers $(n,B)$, parameters $\sigma,\Delta\in[n]$, and the signals $X\in\mathbb{C}^{n}$ and $G\in\mathbb{C}^{n}$, we say that $U\in\mathbb{C}^{B}$ is an $(n,B,G,\sigma,\Delta)$-hashing of $X$ if

[TABLE]

Moreover, we define the following quantities:

• •

$\pi(j)=\sigma\cdot j$, representing the approximately pairwise random permutation;

• •

h(j)=\mathrm{round}\big{(}j\frac{B}{n}\big{)}, representing the bucket in $[B]$ into which a frequency $j$ hashes;

• •

$o_{j}(j^{\prime})=\pi(j^{\prime})-h(j)\frac{n}{B}$, representing the offset associated with two frequencies $(j,j^{\prime})$.

With these definitions, we have the following lemma, proved in Appendix C.1. Note that here we write the exact Fourier transform of $U$ as $\widehat{U}^{*}$, since later we will use $\widehat{U}$ for its near-exact counterpart to simplify notation.

Lemma 4.2**.**

(Fourier transform of hashed signal)* Fix $(n,B)$ and the signals $X\in\mathbb{C}^{n}$ and $G\in\mathbb{C}^{n}$ with the latter symmetric about zero. If $U$ is an $(n,B,G,\sigma,\Delta)$-hashing of $X$, then its exact Fourier transform $\widehat{U}^{*}$ is given by*

[TABLE]

We conclude this subsection by stating the following technical lemma regarding approximately pairwise independent permutations and flat filters.

Lemma 4.3**.**

(Additional filter property)* Fix $n$, and let $G$ be an $(n,B,F)$-flat filter. Let $\pi(\cdot)$ be an approximately pairwise-independent random permutation (cf., Definition 4.1), and for $f,f^{\prime}\in[n]$, define o_{f}(f^{\prime})=\pi(f^{\prime})-\frac{n}{B}\mathrm{round}\big{(}\pi(f)\frac{B}{n}\big{)}. Then for any $x\in\mathbb{C}^{n}$ and $f\in[n]$, we have*

[TABLE]

The proof is given in Appendix C.2.

4.2 Semi-Equispaced FFT

One of the steps of our algorithm is to take the inverse Fourier transform of our current estimate of the spectrum, so that it can be subtracted off and we can work with the residual. The semi-equispaced inverse FFT provides an efficient method for doing this, and is based on the application of the standard inverse FFT to a filtered and downsampled signal.

We start by describing an existing technique of this type for standard sparsity; the details are shown in the procedure SemiEquiInverseFFT in Algorithm 3, and the resulting guarantee from [IKP14, Sec. 12] is stated as follows.222Note that the roles of time and frequency are reversed here compared to [IKP14].

Lemma 4.4**.**

(SemiEquiInverseFFT guarantees [IKP14, Lemma 12.1, Cor. 12.2])* (i) Fix $n$ and a parameter $\zeta>0$. If $\widehat{X}\in\mathbb{C}^{n}$ is $k$-sparse for some $k$, then SemiEquiInverseFFT($\widehat{X},n,k,\zeta$) returns a set of values $\{{Y}_{j}\}_{|j|\leq k/2}$ in time $O(k\log\frac{n}{\zeta})$, satisfying*

[TABLE]

(ii) Given two additional parameters $\sigma,\Delta\in[n]$ with $\sigma$ being odd, it is possible to compute a set of values $\{{Y}_{j}\}$ for all $j$ equaling $\sigma j^{\prime}+\Delta$ for some $j^{\prime}$ with $|j^{\prime}|\leq k/2$, with the same runtime and approximation guarantee.

For the block-sparse setting, we need to adapt the techniques of [IKP14], making use of a two-level scheme that calls SemiEquiInverseFFT. The resulting procedure, SemiEquiInverseBlockFFT, is described in Algorithm 3. The main result of the procedure is the following analog of Lemma 4.4.

Lemma 4.5**.**

(SemiEquiInverseBlockFFT guarantees)* (i) Fix $(n,k_{0},k_{1})$, a $(k_{0},k_{1})$-block sparse signal $\widehat{X}\in\mathbb{C}^{n}$, and a constant $c\geq 1$. Define the shifted signals $\{{X}^{r}\}_{r\in[2k_{1}]}$ with ${X}^{r}_{i}={X}_{i+\frac{nr}{2k_{1}}}$. The procedure SemiEquiInverseBlockFFT($\widehat{X},n,k_{0},k_{1},c$) returns a set of values ${Y}^{r}_{j}$ for all $r\in[2k_{1}]$ and $|j|\leq\frac{k_{0}}{2}$ in time $O(c^{2}k_{0}k_{1}\log^{2}n)$, satisfying*

[TABLE]

(ii) Given two additional parameters \sigma,\Delta\in\big{[}\frac{n}{k_{1}}\big{]} with $\sigma$ odd, it is possible to compute a set of values ${Y}^{r}_{j}$ for all $r\in[2k_{1}]$ and $j$ equaling $\sigma j^{\prime}+\Delta$ (modulo $\frac{n}{k_{1}}$) for some $|j^{\prime}|\leq\frac{k_{0}}{2}$, with the same runtime and approximation guarantee.

The proof is given in Appendix C.3.

Remark 4.1**.**

When applying the preceding lemmas, the signal sparsity and the number of values we wish to estimate will not always be identical. However, this can immediately be resolved by letting the parameter $k$ or $k_{0}$ therein equal the maximum of the two.

4.3 Combining the Tools

In Algorithm 4, we describe two procedures combining the above tools. The first, HashToBins, accepts the signal $\widehat{X}$ and its current estimate $\widehat{\chi}$, uses SemiEquiInverseFFT to approximate the relevant entries of $\chi$, and computes a hashing of $X-\chi$ as per Definition 4.2. The second, HashToBinsReduced, is analogous, but instead accepts a $(k_{1},\delta)$-downsampling of $X$, and uses SemiEquiInverseBlockFFT. It will prove useful to allow the function to hash into a different number of buckets for differing $r$ values, and hence accept $\{G^{r}\}_{r\in[2k_{1}]}$ and $\{B^{r}\}_{r\in[2k_{1}]}$ as inputs. For simplicity, Algorithm 4 states the procedures without precisely giving the parameters passed to the semi-equispaced FFT, but the details are given in the proof of the following.

Lemma 4.6**.**

(HashToBins and HashToBinsReduced guarantees)* (i) Fix $(n,k,B,F)$, an $(n,B,F)$-flat filter $G$ supported on an interval of length $O(FB)$, a signal $X\in\mathbb{C}^{n}$, a $k$-sparse signal $\widehat{\chi}$ For any $(\sigma,\Delta)$, the procedure $\textsc{HashToBins}(X,\widehat{\chi},G,n,B,\sigma,\Delta)$ returns a sequence $\widehat{U}$ such that*

[TABLE]

where $\widehat{U}^{*}$ is the exact Fourier transform of the $(n,B,G,\sigma,\Delta)$-hashing of $X-\chi$ (see Definition 4.2), and $c=c^{\prime}+O(1)$ for $c^{\prime}$ in Algorithm 4. Moreover, the sample complexity is $O(FB)$, and the runtime is $O(cF(B+k)\log n)$.

(ii) Fix $(n,k_{0},k_{1})$ and the parameters $(\{B^{r}\}_{r\in[2k_{1}]},F,\delta)$. For each $r\in[2k_{1}]$, fix an \big{(}\frac{n}{k_{1}},B^{r},F\big{)}-flat filter $G^{r}$ supported on an interval of length $O(FB^{r})$. Moreover, fix a signal $X\in\mathbb{C}^{n}$ and its $(k_{1},\delta)$-downsampling $\{Z^{r}\}_{r\in[2k_{1}]}$ with \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)}, and a $(k_{0},k_{1})$-block sparse signal $\widehat{\chi}$. For any $(\sigma,\Delta)$, the procedure $\textsc{HashToBinsReduced}(\{Z^{r}\}_{r\in[2k_{1}]},\widehat{\chi},\{G^{r}\}_{r\in[2k_{1}},n,k_{1},\{B^{r}\}_{r\in[2k_{1}},\sigma,\Delta)$ returns a set of sequences $\{\widehat{U}^{r}\}_{r\in[2k_{1}]}$ such that

[TABLE]

where $\widehat{U}^{*r}$ is the exact Fourier transform of the \big{(}\frac{n}{k_{1}},B^{r},G^{r},\sigma,\Delta\big{)}-hashing for the $(k_{1},\delta)$-downsampling of $X-\chi$, and $c=c^{\prime}+O(1)$ for $c^{\prime}$ in Algorithm 4. Moreover, the sample complexity is O\big{(}F\sum_{r\in[2k_{1}]}B^{r}\log\frac{1}{\delta}\big{)}, and the runtime is O\big{(}c^{2}(B_{\mathrm{max}}F+k_{0})k_{1}\log^{2}n) with $B_{\mathrm{max}}=\max_{r\in[2k_{1}]}B^{r}$.

The proof is given in Appendix C.4.

Remark 4.2**.**

Throughout the paper, we consider $c$ in Lemma 4.6 to be a large absolute constant. Specifically, various results make assumptions such as $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$, and the results hold true when $c$ is sufficiently large compared to implied exponent in the $\mathrm{poly}(n)$ notation. Essentially, the $n^{-c}$ error term is so small that it can be thought of as zero, but we nevertheless handle it explicitly for completeness.

4.4 Estimating the Downsampled Signal Energies

We now come to the main task of this section, namely, approximating the energy of each $\widehat{Z}^{r}$. To do this, we hash into $B=\frac{4}{\delta^{2}}\cdot k_{0}$ buckets (cf., Definition 4.2), and form the estimate as the energy of the hashed signal. The procedure is shown in Algorithm 5.

Before stating the guarantees of Algorithm 5, we provide the following lemma characterizing the approximation quality for an exact hashing of a signal, as opposed to the approximation returned by HashToBinsReduced. Intuitively, the first part states that we can accurately estimate the top coefficients well without necessarily capturing the noise, and the second part states that, in expectation, we do not over-estimate the total signal energy by more than a small constant factor.

Lemma 4.7**.**

(Properties of exact hashing)* Fix the integers $(m,B)$, the parameters \delta\in\big{(}0,\frac{1}{20}\big{)} and $F^{\prime}\geq 10\log\frac{1}{\delta}$, and the signal $Y\in\mathbb{C}^{m}$ and $(m,B,F^{\prime})$-flat filter $H$ (cf., Definition 2.1). Let $U$ be an $(m,B,H,\sigma,\Delta)$-hashing of $Y$ for uniformly random $\sigma,\Delta\in[m]$ with $\sigma$ odd, and let $\pi(\cdot)$ be defined as in Definition 4.2. Then, letting $\widehat{U}^{*}$ denote the exact Fourier transform of $U$, we have the following:*

For any set $S\subset[m]$,

[TABLE]

where $\|\widehat{Y}_{S}\|_{2}^{2}$ denotes $\sum_{j\in S}|\widehat{Y}_{j}|^{2}$. 2. 2.

We have

[TABLE]

The proof is given in Appendix C.5.

We now present the following lemma, showing that the procedure EstimateEnergies provides us with an estimator satisfying the preconditions of Lemmas 3.2 and 3.3.

Lemma 3.4 (EstimateEnergies guarantees – re-stated from Section 3.1) Given $(n,k_{0},k_{1})$, the signals $X\in\mathbb{C}^{n}$ and $\widehat{\chi}\in\mathbb{C}^{n}$ with $\|\widehat{X}-\widehat{\chi}\|_{2}^{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$, and the parameter \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)}, the procedure EstimateEnergies$(X,\widehat{\chi},n,k_{0},k_{1}$,$\delta)$ returns a vector $\gamma\in\mathbb{R}^{2k_{1}}$ such that, for any given set $S^{*}$ of cardinality at most $10k_{0}$, we have the following with probability at least $\frac{1}{2}$:

\sum_{r\in[2k_{1}]}\Big{|}\|\widehat{Z}^{r}_{S^{*}}\|_{2}^{2}-\gamma^{r}\Big{|}_{+}\leq 40\delta\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}; 2. 2.

$\|\gamma\|_{1}\leq 10\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}$;

where $\{Z^{r}\}_{r\in[2k_{1}]}$ is the $(k_{1},\delta)$-downsampling of $X-\chi$ (see Definition 2.2).

*Moreover, if $\widehat{\chi}$ is $(O(k_{0}),k_{1})$-block sparse, then the sample complexity is $O(\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}\frac{1}{\delta})$, and the runtime is $O(\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}\frac{1}{\delta}\log^{2}n)$. *

Proof.

Analysis for the exact hashing sequence: We start by considering the case that the call to HashToBinsReduced is replaced by an evaluation of the exact hashing sequence $\widehat{U}^{*r}$, i.e., Definition 4.2 applied to $Z^{r}$ resulting from the $(k_{1},\delta)$-downsampling of $X-\chi$. In this case, by applying Lemma 4.7 with $\widehat{Y}=\widehat{Z}^{r}$, $B=\frac{4}{\delta^{2}}k_{0}$ and $S=S^{*}$ (and hence $|S|\leq 10k_{0}$), the right-hand side of the first claim therein becomes $(5\delta+(\frac{15}{4}+2)\delta^{2})\|\widehat{Z}^{r}\|_{2}^{2}\leq 6\delta\|\widehat{Z}^{r}\|_{2}^{2}$, since $\delta\leq\frac{1}{20}$. By applying the lemma separately for each $r\in[2k_{1}]$ with $\widehat{Y}=\widehat{Z}^{r}$, and summing the corresponding expectations in the two claims therein over $r$, we obtain \sum_{r\in[2k_{1}]}\mathbb{E}\big{[}\big{|}\|\widehat{Z}^{r}_{S^{*}}\|_{2}^{2}-\|\widehat{U}^{*}\|_{2}^{2}\big{|}_{+}\big{]}\leq 6\delta\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2} and \sum_{r\in[2k_{1}]}\mathbb{E}\big{[}\|\widehat{U}^{*}\|_{2}^{2}\big{]}\leq 3\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}. We apply Markov’s inequality with a factor of $6$ in the former and $3$ in the latter, to conclude that the quantities $\gamma^{*r}=\|\widehat{U}^{*r}\|_{2}^{2}$ satisfy

[TABLE]

with probability at least $1/2$.

Incorporating $\bf\frac{1}{n^{c}}$ error from use of semi-equispaced FFT in HashToBinsReduced: Since $\widehat{U}^{r}$ is computed using HashToBinsReduced, the energy vector $\gamma$ is different from the exact one $\gamma^{*}$, and we write

[TABLE]

By substituting $\gamma^{r}=\|\widehat{U}^{r}\|_{2}^{2}$ and $\gamma^{*r}=\|\widehat{U}^{*r}\|_{2}^{2}$, and using the identity \big{|}\|a\|_{2}^{2}-\|b\|_{2}^{2}\big{|}\leq 2\|a-b\|_{2}\cdot\|b\|_{2}+\|a-b\|_{2}^{2}, we can write

[TABLE]

Upper bounding the $\ell_{2}$ norm by the $\ell_{\infty}$ norm times the vector length, we have $\|\widehat{U}^{r}-\widehat{U}^{*r}\|_{2}\leq\sqrt{n}\|\widehat{U}^{r}-\widehat{U}^{*r}\|_{\infty}\leq n^{-c+1/2}\|\widehat{\chi}\|_{2}$, where the second inequality follows from Lemma 4.6. Moreover, from the definition of $\widehat{U}^{*r}$ in Definition 4.2 applied to $Z^{r}$, along with the filter property $\|\widehat{G}\|_{\infty}$ in Definition 2.1, it follows that $\|\widehat{U}^{*r}\|_{2}\leq\|\widehat{G}\|_{\infty}\|\widehat{Z}^{r}\|_{1}\leq\sqrt{n}\|\widehat{Z}^{r}\|_{2}$. Combining these into (19) gives

[TABLE]

where the second line is by Cauchy-Schwarz, and the third by $k_{1}\leq n$.

By the second part of Lemma 2.3 and the assumption $\delta\leq\frac{1}{20}$, we have $\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}\geq\frac{1}{4}\|\widehat{X}-\widehat{\chi}\|_{2}^{2}\geq\frac{1}{4n^{c^{\prime}}}\|\widehat{\chi}\|_{2}^{2}$, where the second equality holds for some $c^{\prime}>0$ by the assumption $\|\widehat{X}-\widehat{\chi}\|_{2}^{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$. Hence, (20) gives

[TABLE]

Since we have chosen $\delta>1/n$, the coefficient to the summation is upper bounded by $4\delta$ when $c$ is sufficiently large, thus yielding the first part of the lemma upon combining with (16).

To prove the second part, note that by the triangle inequality,

[TABLE]

where we have applied (17). Again applying (21) and noting that the coefficient to the summation is less than one for sufficiently large $c$, the second claim of the lemma follows.

Sample complexity and runtime: The only step that uses samples is the call to HashToBinsReduced. By Lemma 4.6 and the choices $B=\frac{4}{\delta^{2}}k_{0}$ and $F=10\log\frac{1}{\delta}$, this uses O\big{(}k_{1}FB\log\frac{1}{\delta}\big{)}=O(\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}\frac{1}{\delta}) samples per call. The time complexity follows by the same argument along the assumption that $\widehat{\chi}$ is $(O(k_{0}),k_{1})$-block sparse, with an additional $\log^{2}n$ factor following from Lemma 4.6. Note that the call to HashToBinsReduced dominates the computation of $\gamma^{r}$, which is $O(k_{1}B)$, ∎

5 The Block-Sparse Fourier Transform

In this section, we combine the tools from the previous sections to obtain the full sublinear-time block sparse FFT algorithm, and provide its guarantees.

5.1 Additional Estimation Procedures

Before stating the final algorithm, we note the main procedures that it relies on: MultiBlockLocate, PruneLocation, and EstimateValues. We presented the first of these in Section 3. The latter two are somewhat more standard, and hence we relegate them to the appendices. However, for the sake of readability, we provide some intuition behind them here, and state their guarantees.

We begin with PruneLocation. The procedure MultiBlockLocate gives us a list of block indices containing the dominant signal blocks with high probability, with a list size L=O^{*}\big{(}k_{0}\log k_{0}\big{)}. Estimating the values of all of these blocks in every iteration would not only cost $O^{*}(k_{0}k_{1}\log k_{0})$ samples, but would also destroy the sparsity of the input signal: Most of the blocks correspond to noise, and thus the estimation error may dominate the values being estimated. The PruneLocation primitive is designed to alleviate these issues, pruning $L$ to a list that contains mostly “signal” blocks, i.e., blocks that contain a large amount of energy. Some false positives and false negatives occur, but are controlled by Lemma 5.1 below. The procedure is given in Algorithm 8 in Appendix G.

The following lemma shows that with high probability, the pruning algorithm retains most of the energy in the head elements, while removing most tail elements.

Lemma 5.1**.**

(PruneLocation guarantees)* Given $(n,k_{0},k_{1})$, a list of block indices $L$, the parameters $\theta>0$, \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)} and $p\in(0,1)$, and the signals $X\in\mathbb{C}^{n}$ and $\widehat{\chi}\in\mathbb{C}^{n}$ with $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$, the output list $L^{\prime}$ of PruneLocation$(X,\widehat{\chi},L,n,k_{0},k_{1},\delta,p,\theta)$ has the following properties:*

1. a.

Let $S_{\mathrm{tail}}$ denote the tail elements in the signal $\widehat{X}-\widehat{\chi}$, defined as

[TABLE]

where $I_{j}$ is defined in Definition 1.1. Then, we have

[TABLE] 2. b.

Let $S_{\mathrm{head}}$ denote the head elements in the signal $\widehat{X}-\widehat{\chi}$, defined as

[TABLE]

Then, we have

[TABLE]

Moreover, provided that $\|\widehat{\chi}\|_{0}=O(k_{0}k_{1})$, the sample complexity is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta p}\log\frac{1}{\delta})$, and the runtime is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta p}\log\frac{1}{\delta}\log n+k_{1}\cdot|L|\log\frac{1}{\delta p})$.

The proof is given in Appendix G.

We are left with the procedure EstimateValues, which is a standard procedure for estimating the values at the frequencies within the blocks after they have been located. The details are given in Algorithm 9 in Appendix H.

Lemma 5.2**.**

(EstimateValues guarantees)* For any integers $(n,k_{0},k_{1})$, list of block indices $L$, parameters \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)} and $p\in(0,1/2)$, and signals $X\in\mathbb{C}^{n}$ and $\widehat{\chi}\in\mathbb{C}^{n}$ with $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$, the output $W$ of the function EstimateValues$(X,\widehat{\chi},L,n,k_{0},k_{1},\delta,p)$ has the following property:*

[TABLE]

with probability at least $1-p$, where $I_{j}$ is the $j$-th block. Moreover, provided that $\|\widehat{\chi}\|_{0}=O(k_{0}k_{1})$, the sample complexity is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{p}\log\frac{1}{\delta})$, and the runtime is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{p}\log\frac{1}{\delta}\log n+k_{1}\cdot|L|\log\frac{1}{p})$.

The proof is given in Appendix H.

5.2 Statement of the Algorithm and Main Result

Our overall block-sparse Fourier transform algorithm is given in Algorithm 6. It first calls ReduceSNR, which performs an iterative procedure that picks up high energy components of the signal, subtracts them from the original signal, and then recurses on the residual signal $X^{(i)}=X-\chi^{(i)}$. Once this is done, the procedure RecoverAtConstSNR performs a final “clean-up” step to obtain the $(1+O(\epsilon))$-approximation guarantee.

With these definitions in place, we can now state our final result, which formalizes Theorem 1.1.

Theorem 1.1 (Upper bound – formal version) Given $(n,k_{0},k_{1})$, the parameter \epsilon\in\big{(}\frac{1}{n},\frac{1}{20}\big{)}, and the signal $X\in\mathbb{C}^{n}$, if $X$, $\mathrm{SNR}^{\prime}$, $\mu^{2}$, and $\nu^{2}$ satisfy the following for $(\mu^{2},\mathrm{SNR})$ given in Definition 1.2:

$\mu^{2}\leq\nu^{2}$; 2. 2.

$\|\widehat{X}\|_{2}^{2}\leq(k_{0}\nu^{2})\cdot\mathrm{SNR}^{\prime}$; 3. 3.

$\mathrm{SNR}^{\prime}=O(\mathrm{poly}(n))$; 4. 4.

$\mu^{2}\geq\frac{\|\widehat{X}\|_{2}^{2}}{\mathrm{poly}(n)}$;

then with probability at least $0.8$, the procedure BlockSparseFT$(X,n,k_{0},k_{1},\mathrm{SNR}^{\prime},\nu^{2},\epsilon)$ satisfies the following:

(i) The output $\widehat{\chi}$ satisfies

[TABLE]

*(ii) The sample complexity is is $O^{*}(k_{0}\log(1+k_{0})\log\mathrm{SNR}^{\prime}\log n+k_{0}k_{1}\log\mathrm{SNR}^{\prime}+\frac{k_{0}}{\epsilon^{2}}\log(1+k_{0})\log n+\frac{k_{0}k_{1}}{\epsilon^{4}})$, and the runtime is $O^{*}(k_{0}\log(1+k_{0})\log\mathrm{SNR}^{\prime}\log^{2}n+k_{0}k_{1}\log\mathrm{SNR}^{\prime}\log^{3}n+\frac{k_{0}}{\epsilon^{2}}\log(1+k_{0})\log^{2}n+\frac{k_{0}k_{1}}{\epsilon^{4}}\log^{2}n+\frac{k_{0}k_{1}}{\epsilon^{2}}\log^{3}n)$. *

The assumptions of the theorem are essentially that we know upper bounds on the tail noise $\mu^{2}$ and SNR. Moreover, in order to get the $(1+O(\epsilon))$-approximation guarantee, the former upper bound should be tight to within a constant factor.

In the remainder of the section, we provide the proof of Theorem 1.1, deferring the technical details to the appendices.

Guarantees for ReduceSNR and RecoverAtConstSNR.

The following lemma proves the success of the function ReduceSNR. We again recall the definitions of $\mathrm{Err}^{2}$, $\mu^{2}$, and $\mathrm{SNR}$ in Definition 1.2.

Lemma 5.3**.**

(ReduceSNR guarantees)* Given $(n,k_{0},k_{1})$, parameters $(\nu,\mathrm{SNR}^{\prime})$, and a signal $X\in\mathbb{C}^{n}$, if $X$, $\mathrm{SNR}^{\prime}$, and $\nu^{2}$ satisfy the following for $(\mu^{2},\mathrm{SNR})$ given in Definition 1.2:*

$\mu^{2}\leq\nu^{2}$; 2. 2.

$\|\widehat{X}\|_{2}^{2}\leq(k_{0}\nu^{2})\cdot\mathrm{SNR}^{\prime}$; 3. 3.

$\mathrm{SNR}^{\prime}=O(\mathrm{poly}(n))$; 4. 4.

$\nu^{2}\geq\frac{\|\widehat{X}\|_{2}^{2}}{\mathrm{poly}(n)}$;

then the procedure ReduceSNR$(X,n,k_{0},k_{1},\mathrm{SNR}^{\prime},\nu^{2})$ satisfies the following guarantees with probability at least $0.9$ when the constant $\delta$ therein is sufficiently small:

(i) The output $\widehat{\chi}^{T}$ satisfies

[TABLE]

(ii) The number of samples used is $O^{*}(k_{0}\log(1+k_{0})\log\mathrm{SNR}^{\prime}\log n+k_{0}k_{1}\log\mathrm{SNR}^{\prime})$, and the runtime is $O^{*}(k_{0}\log(1+k_{0})\log\mathrm{SNR}^{\prime}\log^{2}n+k_{0}k_{1}\log\mathrm{SNR}^{\prime}\log^{3}n)$.

The proof is given in Appendix D.1

The following lemma proves the success of the function RecoverAtConstSNR.

Lemma 5.4**.**

(RecoverAtConstSNR guarantees)* Given $(n,k_{0},k_{1})$, parameters $\nu^{2}\geq\frac{\|\widehat{X}\|_{2}^{2}}{\mathrm{poly}(n)}$ and \epsilon\in\big{(}\frac{1}{n},\frac{1}{20}\big{)}, and the signals $X\in\mathbb{C}^{n}$ and $\widehat{\chi}\in\mathbb{C}^{n}$ satisfying*

$\mathrm{Err}^{2}(\widehat{X}-\widehat{\chi},10k_{0},k_{1})\leq k_{0}\nu^{2}$; 2. 2.

$\|\widehat{X}-\widehat{\chi}\|_{2}^{2}\leq 100k_{0}\nu^{2}$;

the procedure RecoverAtConstSNR$(X,\widehat{\chi},n,k_{0},k_{1},\epsilon)$ satisfies the following guarantees with probability at least $0.9$ when the constant $\eta$ therein is sufficiently small: (i) The output $\widehat{\chi}^{\prime}$ satisfies

[TABLE]

(ii) If $\widehat{\chi}$ is $(O(k_{0}),k_{1})$-block sparse, then the number of samples used is $O^{*}(\frac{k_{0}}{\epsilon^{2}}\log(1+k_{0})\log n+\frac{k_{0}k_{1}}{\epsilon^{4}})$ and the runtime is $O^{*}(\frac{k_{0}}{\epsilon^{2}}\log(1+k_{0})\log^{2}n+\frac{k_{0}k_{1}}{\epsilon^{4}}\log^{2}n+\frac{k_{0}k_{1}}{\epsilon^{2}}\log^{3}n)$.

The proof is given in Appendix D.2.

Proof of Theorem 1.1.

We are now in a position to prove Theorem 1.1 via a simple combination of Lemmas 5.3 and 5.4.

Success event associated with ReduceSNR:

Define a successful run of ReduceSNR($X,n,k_{0},k_{1},\mathrm{SNR},\nu^{2}$) to mean mean the following conditions on the output $\widehat{\chi}^{T}$:

[TABLE]

By Lemma 5.3, it follows that the probability of having a successful run of ReduceSNR is at least $0.9$. Note that the second condition is not explicitly stated in Lemma 5.3, but it follows by using $3k_{0}$ blocks to cover the parts where $\widehat{\chi}^{T}$ is non-zero, and $k_{0}$ blocks to cover the dominant blocks of $\widehat{X}$, in accordance with Definition 1.2.

Success event associated with RecoverAtConstSNR:

Define a successful run of RecoverAtConstSNR($X,\widehat{\chi},k_{0},k_{1},n,\epsilon$) to mean the following conditions on the output $\widehat{\chi}$:

[TABLE]

Conditioning on event of having a successful run of ReduceSNR, by Lemma 5.4, it follows that the probability of having a successful run to RecoverAtConstSNR is at least $0.9$.

By a union bound, the aforementioned events occur simultaneously with probability at least $0.8$, as desired. Moreover, the sample complexity and runtime are a direct consequence of summing the contributions from Lemmas 5.3 and 5.4.

6 Lower Bound

Our upper bound in Theorem 1.1, in several scaling regimes, provides a strict improvement over standard sparse FFT algorithms in terms of sample complexity. The corresponding algorithm is inherently adaptive, which raises the important question of whether adaptivity is necessary in order to achieve these improvements. In this section, we show that the answer is affirmative, by proving the following formalization of Theorem 1.2.

Theorem 1.2 (Lower bound – formal version) Fix $(n,k_{0},k_{1})$ and $C>0$, and suppose that there exists a non-adaptive algorithm that, when given a signal $Y$ with Fourier transform $\widehat{Y}$, outputs a signal $\widehat{Y}^{\prime}$ satisfying the following $\ell_{2}/\ell_{2}$-guarantee with probability at least $\frac{1}{2}$:

[TABLE]

*Then the number of samples taken by the algorithm must behave as \Omega\big{(}k_{0}k_{1}\log\frac{n}{k_{0}k_{1}}\big{)}. *

Hence, for instance, if $k_{0}=O(1)$ and $\mathrm{SNR}=O(1)$ then our adaptive algorithm uses $O(k_{1}+\log n)$ samples, whereas any non-adaptive algorithm must use \Omega\big{(}k_{1}\log\frac{n}{k_{1}}\big{)} samples.

The remainder of this section is devoted to the proof of Theorem 1.2. Throughout the section, we let $k=k_{0}k_{1}$ denote the total sparsity.

High-level overview: Our analysis follows the information-theoretic framework of [PW11]. However, whereas [PW11] considers a signal with $k$ arbitrary dominant frequency locations and uniform noise, we consider signals where the $k=k_{0}k_{1}$ dominant frequencies are (nearly) contiguous, and both the noise and signal are concentrated on an O\big{(}\frac{1}{k}\big{)} fraction of the time domain.

As a result, while the difficulty in [PW11] arises from the fact that the algorithm needs to recover roughly $\log\frac{n}{k}$ bits per frequency location for $k$ such locations, our source of difficulty is different. In our signal, there are only roughly $\log\frac{n}{k}$ bits to be learned about the location of all the blocks in frequency domain, but the signal is tightly concentrated on an O\big{(}\frac{1}{k}\big{)} fraction of the input space. As a consequence, any non-adaptive algorithm is bound to waste most of its samples on regions of the input space where the signal is zero, and only an O\big{(}\frac{1}{k}\big{)} fraction of its samples can be used to determine the single frequency that conveys the location of the blocks. In the presence of noise, this results in a lower bound on sample complexity of \Omega\big{(}k\log\frac{n}{k}\big{)}.

Information-theoretic preliminaries: We will make use of standard results from information theory, stated below. Here and subsequently, we use the notations $H(X)$, $H(Y|X)$, $I(X;Y)$ and $I(X;Y|U)$ for the (conditional) Shannon entropy and (conditional) mutual information (e.g., see [CT06b]). We first state Fano’s inequality, a commonly-used tool for proving lower bounds by relating a conditional entropy to an error probability.

Lemma 6.1**.**

(Fano’s Inequality [CT06b, Lemma 7.9.1])* Fix the random variables $(X,Y)$ with $X$ being discrete, let $X^{\prime}$ be an estimator of $X$ such that $X\to Y\to X^{\prime}$ forms a Markov chain (i.e., $X$ and $X^{\prime}$ are conditionally independent given $Y$), and define $P_{\mathrm{e}}:=\mathbb{P}[X^{\prime}\neq X]$. Then*

[TABLE]

where $\mathcal{X}=\mathrm{supp}(X)$. Consequently, if $X$ is uniformly distributed, then

[TABLE]

The next result gives the formula for the capacity of a complex-valued additive white Gaussian noise channel, often referred to as the Shannon-Hartley theorem. Here and subsequently, $\mathrm{CN}(\mu,\sigma^{2})$ denotes the complex normal distribution.

Lemma 6.2**.**

(Complex Gaussian Channel Capacity [CT06b, Thm. 2.8.1])* For $Z\sim\mathrm{CN}(0,\sigma_{z}^{2})$ and any complex random variable $X$ with $\mathbb{E}[|X|^{2}]=\sigma_{x}^{2}$, we have*

[TABLE]

with equality if $X\sim\mathrm{CN}(0,\sigma_{x}^{2})$.

The following lemma states the data processing inequality, which formalizes the statement that processing a channel output cannot increase the amount of information revealed about the input.

Lemma 6.3**.**

(Data Processing Inequality [CT06b, Thm. 2.8.1])* For any random variables $(X,Y,Z)$ such that $X\to Y\to Z$ forms a Markov chain, we have $I(X;Z)\leq I(X;Y)$.*

Finally, the following lemma bounds the mutual information between two vectors in terms of the individual mutual information terms between components of those vectors.

Lemma 6.4**.**

(Mutual Information for Vectors [CT06b, Lemma 7.9.2])* For any random vectors $\mathbf{X}=(X_{1},\dotsc,X_{n})$ and $\mathbf{Y}=(Y_{1},\dotsc,Y_{n})$, if the entries of $\mathbf{Y}$ are conditionally independent given $\mathbf{X}$, then $I(\mathbf{X};\mathbf{Y})\leq\sum_{i=1}^{n}I(X_{i};Y_{i})$.*

A communication game: We consider a communication game consisting of channel coding with a state known at both the encoder (Alice) and decoder (Bob), where block-sparse recovery is performed at the decoder. Recalling that we are in the non-adaptive setting, by Yao’s minimax principle, we can assume that the samples are deterministic and require probability-$\frac{1}{2}$ recovery over a random ensemble of signals, as opposed to randomizing the samples and requiring constant-probability recovery for any given signal in the ensemble. Hence, we denote the fixed sampling locations by $\mathcal{A}$.

We now describe our hard input distribution. Each signal in the ensemble is indexed by two parameters $(u,f^{*})$, and is given by

[TABLE]

where $C^{\prime}>0$ is a constant that will be chosen later, and where all indices are modulo-$n$. Hence, each signal is non-zero only in a window of length $\frac{C^{\prime}n}{k}$, and within that window, the signal oscillates at a rate dictated by $f^{*}$. Specifically, $u$ specifies where the signal is non-zero in time domain, and $f^{*}$ specifies where the energy is concentrated in frequency domain. We restrict the values of $u$ and $f^{*}$ to the following sets:

[TABLE]

The communication game is as follows:

Nature selects a state $U$ and a message $F$ uniformly from $\mathcal{U}$ and $\mathcal{F}$, respectively. 2. 2.

An encoder maps $(U,F)$ to the signal $X$ according to (25). 3. 3.

A state-dependent channel adds independent $\mathrm{CN}(0,\alpha)$ noise to $X_{t}$ for each $t\in\{u,\dotsc,u+\frac{C^{\prime}n}{k}-1\}$, while keeping the other entries noiseless. This is written as $Y_{t}=X_{t}+W_{t}$, where

[TABLE]

The channel output is given by $Y=\{Y_{t}\}_{t\in\mathcal{A}}$ for the sampling locations $\mathcal{A}$. 4. 4.

A decoder receives $U$ and $Y$, applies $(k_{0},k_{1})$-block sparse recovery to $Y$ to obtain a signal $\widehat{Y}^{\prime}$, and then selects $F^{\prime}$ to be the frequency $f^{\prime}\in\mathcal{F}$ such that the energy in $\widehat{Y}^{\prime}$ within the length-$k$ window centered at $f^{\prime}$ is maximized:

[TABLE]

where $I_{k}(f^{\prime})=\{f^{\prime}+\Delta\,:\,\Delta\in[k]\}$.

We observe that if adaptivity were allowed, then the knowledge of $U$ at the decoder would make the block-sparse recovery easy – one could let all of the samples lie within the window given in (25). The problem is that we are in the non-adaptive setting, and hence we must take enough samples to account for all of the possible choices of $U$.

We denote the subset of $\mathcal{A}$ falling into $\{u,\dotsc,u+\frac{C^{\prime}n}{k}-1\}$ by $\mathcal{A}_{u}$, its cardinality by $m_{u}$ and the total number of measurements by $m=|\mathcal{A}|=\sum_{u}m_{u}$. Moreover, we let $X_{\mathcal{A}_{u}}$ and $Y_{\mathcal{A}_{u}}$ denote the sub-vectors of $X$ and $Y$ indexed by $\mathcal{A}_{u}$.

Information-theoretic analysis: We first state the following lemma.

Lemma 6.5**.**

(Mutual information bound)* In the setting described above, the conditional mutual information $I(F;Y|U)$ satisfies I(F;Y|U)\leq\frac{C^{\prime}m}{k}\log\big{(}1+\frac{1}{\alpha}\big{)}, where $\alpha$ is variance of the additive Gaussian noise within $\mathcal{A}_{u}$.*

Proof.

We have

[TABLE]

where:

• •

Line 1 follows since $U$ is uniform on a set of cardinality $\frac{k}{C^{\prime}}$;

• •

Line 2 follows since given $U=u$, only the entries of $Y$ indexed by $\mathcal{A}_{u}$ are dependent on $F$ (cf., (25));

• •

Line 3 follows by noting that given $U=u$ we have the Markov chain $F\to X_{\mathcal{A}_{u}}\to Y_{\mathcal{A}_{u}}$, and applying the data processing inequality (Lemma 6.3);

• •

Line 4 follows from Lemma 6.4 and (26), where the conditional independence assumption holds because we have assumed the random variables $W_{t}$ are independent;

• •

Line 5 follows from the Shannon-Hartley Theorem (Lemma 6.2); in our case, the signal power is exactly one by (25), and the average noise energy is $\alpha$ by construction.

∎

Next, defining $\delta_{u}:=\mathbb{P}[F^{\prime}\neq F\,|\,U=u]$, Fano’s inequality (Lemma 6.1) gives

[TABLE]

and averaging both sides over $U$ gives

[TABLE]

where $\delta:=\mathbb{E}[\delta_{U}]=\mathbb{P}[F^{\prime}\neq F]$.

Hence, and by Lemma 6.5, if we can show that our $\ell_{2}/\ell_{2}$-error guarantee (24) gives $F^{\prime}=F$ constant probability, then we can conclude that

[TABLE]

We therefore conclude the proof of Theorem 1.2 by proving the following lemma.

Lemma 6.6**.**

(Probability of error characterization)* Fix $(n,k_{0},k_{1})$ and $C>0$. If the $(k_{0},k_{1})$-block sparse recovery algorithm used in the above communication game satisfies (24) with probability at least $\frac{1}{2}$, then there exist choices of $C^{\prime}$ and $\alpha$ such that the decoder’s estimate of $F^{\prime}$ according to (27) satisfies $F^{\prime}=F$ with probability at least $\frac{1}{4}$.*

Proof.

By the choice of estimator in (27), it suffices to show that $\widehat{Y}^{\prime}$, the output of the block-sparse Fourier transform algorithm, has more than half of its energy within the length-$k$ window $I_{k}(f^{*})$ centered of $f^{*}$. We show this in three steps.

**Characterizing the energy of $\widehat{X}$ within $I_{k}(f^{*})$: ** The Fourier transform of $X$ in (25) is a shifted sinc function of “width” $\frac{k}{C^{\prime}}$ centered at $f^{*}$ when the time window is centered at zero, and more generally, has the same magnitude as this sinc function. Hence, by letting $C^{\prime}$ be suitably large, we can ensure that an arbitrarily high fraction of the energy of $\widehat{X}$ falls within the length-$k$ window centered at $f^{*}\in\mathcal{F}$. Formally, we have

[TABLE]

for $\eta\in(0,1)$ that we can make arbitrarily small by choosing $C^{\prime}$ large.

Characterizing the energy of $\widehat{Y}$ within $I_{k}(f^{*})$: We now show that, when the noise level $\alpha$ in (26) it sufficiently small, the energy in $\widehat{Y}$ within $I_{k}(f^{*})$ is also large with high probability:

[TABLE]

To prove this, we first note that $|\widehat{Y}_{f}|^{2}=|\widehat{X}_{f}+\widehat{W}_{f}|^{2}$ for all $f\in[n]$, from which it follows that

[TABLE]

Upper bounding the summation over $|\widehat{W}_{f}|^{2}$ by the total noise energy, and upper bounding the summation over $|\widehat{X}_{f}|\cdot|\widehat{W}_{f}|$ using the Cauchy-Schwarz inequality, we obtain

[TABLE]

We therefore continue by bounding the total noise energy $\|\widehat{W}\|_{2}^{2}$; the precise distribution of the noise across different frequencies is not important for our purposes.

Recall that every non-zero entry of $X$ has magnitude one, and every non-zero time-domain entry of $W$ is independently distributed as $\mathrm{CN}(0,\alpha)$. Combining these observations gives $\mathbb{E}[\|W\|_{2}^{2}]=\alpha\|X\|_{2}^{2}$, or equivalently $\mathbb{E}[\|\widehat{W}\|_{2}^{2}]=\alpha\|\widehat{X}\|_{2}^{2}$ by Parseval. Therefore, by Markov’s inequality, we have $\|\widehat{W}\|_{2}^{2}\leq 4\alpha\|\widehat{X}\|_{2}^{2}$ with probability at least $\frac{3}{4}$. When this occurs, (31) gives

[TABLE]

If we choose $\alpha=\frac{\eta^{2}}{100}$, then we have $4(\alpha+\sqrt{\alpha})=\frac{\eta^{2}}{25}+\frac{4\eta}{10}\leq\eta$. In this case, by (29) and (32), the length-$k$ window $I_{k}(f^{*})$ centered at $f^{*}$ satisfies (30).

Characterizing the energy of $\widehat{Y}^{\prime}$ within $I_{k}(f^{*})$: The final step is to prove that (30) and (24) imply the following with constant probability for a suitable choice of $\eta$:

[TABLE]

where $\widehat{Y}^{\prime}$ is the output of the block-sparse recovery algorithm. This clearly implies that $F=F^{\prime}$, due to our choice of estimator in (27).

As a first step towards establishing (33), we rewrite (30) as

[TABLE]

We can interpret (34) as an error term $\|\widehat{Y}-\widehat{Y}^{*}\|_{2}^{2}$ for a signal $\widehat{Y}^{*}$ coinciding with $\widehat{Y}$ within $I_{k}(f^{*})$ and being zero elsewhere. Since $I_{k}(f^{*})$ contains $k$ contiguous elements, this signal is $(k_{0},k_{1})$-block sparse, and hence if the guarantee in (24) holds, then combining with (34) gives

[TABLE]

We henceforth condition on both (24) and the above-mentioned event $\|\widehat{W}\|_{2}^{2}\leq 4\alpha\|\widehat{X}\|_{2}^{2}$. Since the former occurs with probability at least $\frac{1}{2}$ by assumption, and the latter occurs with probability at least $\frac{3}{4}$, their intersection occurs with probability at least $\frac{1}{4}$.

Next, we write the conditions in (30) and (35) in terms of $\|\widehat{Y}\|_{2}^{2}$, rather than $\|\widehat{X}\|_{2}^{2}$. Since $\widehat{X}=\widehat{Y}-\widehat{W}$, we can use the triangle inequality to write $\|\widehat{X}\|_{2}\geq\|\widehat{Y}\|_{2}-\|\widehat{W}\|_{2}$, and combining this with $\|\widehat{W}\|_{2}^{2}\leq 4\alpha\|\widehat{X}\|_{2}^{2}$, we obtain $\|\widehat{X}\|_{2}\geq\frac{\|\widehat{Y}\|_{2}}{1+2\sqrt{\alpha}}$. Hence, we can weaken (30) and (35) to

[TABLE]

where the second step in each equation holds for sufficiently small $\eta$ due to the choice $\alpha=\frac{\eta^{2}}{100}$.

It only remains to use (36)–(37) to bound the left-hand side of (33). To do this, we first note that by interpreting both (36) and (37) as bounds on $\|\widehat{Y}\|_{2}^{2}$, and using $\|\widehat{Y}-\widehat{Y}^{\prime}\|_{2}^{2}\geq\|(\widehat{Y}-\widehat{Y}^{\prime})_{I_{k}(f^{*})}\|_{2}^{2}$ in the latter, we have

[TABLE]

since $\frac{0.01}{0.99}\leq 0.02$. Taking the square root and applying the triangle inequality to the $\ell_{2}$-norm on the left-hand side, we obtain

[TABLE]

Next, writing $\|\widehat{Y}^{\prime}\|_{2}=\|\widehat{Y}+(\widehat{Y}^{\prime}-\widehat{Y})\|_{2}$, and applying the triangle inequality followed by (37), we have $\|\widehat{Y}^{\prime}\|_{2}\leq 1.1\|\widehat{Y}\|_{2}$, and hence $\|\widehat{Y}\|_{2}\geq 0.9\|\widehat{Y}^{\prime}\|_{2}$. Squaring and substituting into (36), we obtain

[TABLE]

Finally, combining (38) and (39) yields (33), and we have thus shown that (33) holds (and hence $F=F^{\prime}$) with probability at least $\frac{1}{4}$. ∎

7 Acknowledgements

MK would like to thank Piotr Indyk for helpful discussions. VC and JS are supported by the European Commission (ERC Future Proof), SNF (200021-146750 and CRSII2-147633), and ‘EPFL Fellows’ program (Horizon2020 665667).

Appendix A Omitted Proofs from Section 2

A.1 Proof of Lemma 2.1

Our filter construction is similar to [IK14], but we prove and utilize different properties, and hence provide the details for completeness.

Definition A.1** (Rectangular pulse).**

For an even integer $B^{\prime}$, let $\mathrm{rect}^{B^{\prime}}$ denote the rectangular pulse of width $B^{\prime}-1$, i.e.

[TABLE]

For an integer $B^{\prime}>0$ a power of $2$, define the length-$n$ signal

[TABLE]

where the convolution is performed $F$ times. As noted in [IK14], we have $\mathrm{supp}(W^{F})\subseteq[-F\cdot B^{\prime},F\cdot B^{\prime}]$, and the Fourier transform is given by

[TABLE]

for $f\neq 0$, and $W_{0}=1$.

Lemma A.1**.**

(Properties of $W$)* For every even $F\geq 2$, the following hold for the signal $W$ defined in (40)–(41):*

1

$\widehat{W}_{f}\in[0,1]$* for all $f\in[n]$;*

2

There exists an absolute constant $C\geq 0$ such that for every $\lambda>1$,

[TABLE]

Proof.

First note that the maximum of $\widehat{W}_{f}$ is achieved at [math] and equals $1$. Since $F$ is even by assumption, we have from (41) that $\widehat{W}_{f}\geq 0$ for all $f$. These two facts establish the first claim.

To prove the second claim, note that for all $f\in[n]$, we have

[TABLE]

We claim that this can be weakened to

[TABLE]

For $f\in[-n/B^{\prime},n/B^{\prime}]$ the right-hand side is at least one, and hence this claim follows directly from the first claim above. On the other hand, if $|f|\geq n/B^{\prime}$, we have

[TABLE]

and hence (43) follows from (42).

Using (43), we have

[TABLE]

At the same time, for any $f\in[-\frac{n}{2B^{\prime}},\frac{n}{2B^{\prime}}]$, we have

[TABLE]

This means that

[TABLE]

Putting (44) together with (45), we get

[TABLE]

for an absolute constant $C^{\prime}>0$. The desired claim follows since $C^{\prime}(\pi/2)^{F}=((C^{\prime})^{1/(F-1)}\cdot(\pi/2)^{F/(F-1)}))^{F-1}\leq(C^{\prime}\cdot(\pi/2)^{2})^{F-1}$ (due to the assumption that $F\geq 2$). ∎

We now fix an integer $B$, and define $\widehat{G}$ by

[TABLE]

where $Z=\sum_{f\in[n]}\widehat{W}_{f}$. By interpreting this as a convolution with a rectangle, we obtain that the inverse Fourier transform $G_{t}$ is obtained via the multiplication of $W_{t}$ with a sinc pulse.

We proceed by showing that, upon identifying $B^{\prime}=8CB$ (where $B^{\prime}$ was used in defining $\widehat{W}$, and $C$ is the implied constant in Lemma A.1), this filter satisfies the claims of Lemma 2.1. We start with the three properties in Definition 2.1.

Proof of [Lemma 2.1 (filter property 1)]: For every $f$, we have

[TABLE]

Similarly, the non-negativity of $\widehat{G}$ follows directly from that of $\widehat{W}$. ∎

Proof of [Lemma 2.1 (filter property 2)]: For every $f\in[n]$ with $|f|\leq\frac{n}{2B}$, we have

[TABLE]

Since $B^{\prime}/B=8C$ by our choice of $B^{\prime}$ above, we get $\widehat{G}_{f}\geq 1-(1/4)^{F-1}$, as required. ∎

Proof of [Lemma 2.1 (filter property 3)]: For every $f\in[n]$ with $|f|\geq\frac{n}{B}$, we have

[TABLE]

Defining $\zeta\geq 1$ such that $|f|=(3+\zeta)\frac{n}{4B}$, this becomes

[TABLE]

Rearranging the definition of $\zeta$, we obtain $\zeta=\frac{4B|f|}{n}-3$, and hence $\zeta\geq\frac{B|f|}{n}$ due to the fact that $|f|\geq\frac{n}{B}$. Therefore, \widehat{G}_{f}\leq\big{(}\frac{n}{4B|f|}\big{)}^{F-1}. ∎

Proof of [Lemma 2.1 (additional property 1)]: We have already shown that ${W}$ is supported on a window of length $O(FB^{\prime})=O(FB)$ centered at zero. The same holds for ${G}$ since it is obtained via a pointwise multiplication of ${W}$ with a sinc pulse. ∎

Proof of [Lemma 2.1 (additional property 2)]: Since $\widehat{G}_{f}\in[0,1]$, the total energy across $|f|<\frac{n}{B}$ is at most $\frac{2n}{B}$. On the other hand, we have from the third property in Definition 2.1 that

[TABLE]

Combining this with the contribution from $|f|<\frac{n}{B}$ concludes the proof. ∎

A.2 Proof of Lemma 2.3

We are interested in the behavior of $\sum_{r\in[2k_{1}]}|\widehat{Z}^{r}_{j}|^{2}$ for each $j$ (first part), and summed over all $j$ (second part). We therefore begin with the following lemma, bounding this summation in terms of the signal $X$ and the filter $G$.

Lemma A.2**.**

(Initial downsampling bound)* For any integers $(n,k_{1})$, parameter \delta\in\big{(}0,\frac{1}{20}\big{)}, signal $X\in\mathbb{C}^{n}$ and its corresponding $(k_{1},\delta)$-downsampling $\{Z^{r}\}_{r\in[2k_{1}]}$, the following holds for all $j\in[\frac{n}{k_{1}}]$:*

[TABLE]

Proof.

Directly evaluating the sum: Using the definition of the signals $\widehat{Z}^{r}$ in (2), we write

[TABLE]

where $(\cdot)^{\dagger}$ denotes the complex conjugate. Since $a_{r}=\frac{nr}{2k_{1}}$, the term $\sum_{r\in[2k_{1}]}\omega_{n}^{a_{r}(f^{\prime}-f)}$ is equal to $2k_{1}$ if $f-f^{\prime}$ is a multiple of $2k_{1}$ (including $f=f^{\prime}$) and zero otherwise, yielding

[TABLE]

Without loss of generality, we consider $j=0$; otherwise, we can simply consider a version of $X$ shifted in frequency domain by $k_{1}j$. Setting $j=0$ in (46) and applying the triangle inequality, we obtain

[TABLE]

Bounding the right-hand side of (47): We write

[TABLE]

In Lemma A.3 below, we show that

[TABLE]

for all $f\in[n]$ and all $|j^{\prime}|\leq\frac{n}{2(2k_{1})}$ with $j^{\prime}\neq 0$. Definition 2.2 ensures that \big{(}\frac{1}{2}\big{)}^{F-1}\leq\delta, and substitution into (48) gives

[TABLE]

Next, we apply the Cauchy-Schwarz inequality to upper bound the inner summation over $f$ above for any fixed $j^{\prime}\in[\frac{n}{2k_{1}}]$, yielding

[TABLE]

where we used the fact that \big{\{}|\widehat{G}_{f-2k_{1}j^{\prime}}|\cdot|\widehat{X}_{f-2k_{1}j^{\prime}}|^{2}\big{\}}_{f=1}^{n} is a permutation of \big{\{}|\widehat{G}_{f}|\cdot|\widehat{X}^{*}_{f}|^{2}\big{\}}_{f=1}^{n}.

Wrapping up: Substituting (50) into (49) gives

[TABLE]

where the last inequality follows from the fact that $\sum_{\begin{subarray}{c}|j^{\prime}|\leq\frac{n}{2(2k_{1})}\\ j^{\prime}\neq 0\end{subarray}}\frac{1}{|j^{\prime}|^{(F-1)/2}}\leq 2\sum_{j^{\prime}=1}^{\infty}\frac{1}{|j^{\prime}|^{(F-1)/2}}$, which is upper bounded by $3$ for $F\geq 8$, a condition guaranteed by Definition 2.2. We therefore obtain the following bound from (47):

[TABLE]

The lemma follows by recalling that the choice $j=0$ was without loss of generality, with the general case amounting to replacing $\widehat{Z}_{0}$ by $\widehat{Z}_{j}$ and $\widehat{G}_{f}$ by $\widehat{G}_{f-k_{1}j}$. ∎

In the preceding proof, we made use of the following technical result bounding the product of the filter $G$ evaluated at two locations separated by some multiple of $2k_{1}$.

Lemma A.3**.**

(Additional filter property)* Given $(n,k_{1})$ and a parameter $F\geq 2$, if $G$ is an $(n,\frac{n}{k_{1}},F)$-flat filter, then the following holds for all $f\in[n]$ and all j\in\big{[}\frac{n}{k_{1}}\big{]} with $|j^{\prime}|\leq\frac{n}{2(2k_{1})}$ and $j^{\prime}\neq 0$:*

[TABLE]

Proof.

For clarity, let $f_{1}$ and $f_{2}$ denote the frequencies corresponding to $f$ and $f-2k_{1}j^{\prime}$ respectively, defined in the range $(-n/2,n/2]$ according to modulo-$n$ arithmetic. By definition, $f_{1}-f_{2}$ is equal to $2k_{1}j^{\prime}$ modulo-$n$, and since $|j^{\prime}|\leq\frac{n}{2(2k_{1})}$, we have $|2k_{1}j^{\prime}|\leq\frac{n}{2}$. This immediately implies that the distance $\Delta=|f_{1}-f_{2}|$ according to regular arithmetic is lower bounded by the distance according to modulo-$n$ arithmetic: $\Delta\geq 2k|j^{\prime}|$.

Since $f_{1}$ and $f_{2}$ are at a distance $\Delta$ according to regular arithmetic, it must be the case that either $|f_{1}|\geq\frac{\Delta}{2}$ or $|f_{2}|\geq\frac{\Delta}{2}$. Moreover, since $j^{\prime}\neq 0$, we have, from the above-established fact $\Delta\geq 2k_{1}|j^{\prime}|$, that $\frac{\Delta}{2}\geq k_{1}$, and hence we can apply the third filter property in Definition 2.1 to conclude that |G_{f_{\nu}}|\leq\big{(}\frac{1}{4}\big{)}^{F-1}\big{(}\frac{2k_{1}}{\Delta}\big{)}^{F-1} for either $\nu=1$ or $\nu=2$. Substituting $\Delta\geq 2k_{1}|j^{\prime}|$, upper bounding $G_{f_{\nu^{\prime}}}\leq 1$ (cf., Definition 2.1) for the index $\nu^{\prime}\in\{1,2\}$ differing from $\nu$, and taking the square root, we obtain the desired result. ∎

We are now in a position to prove the claims of Lemma 2.3

Proof of first part of Lemma 2.3: Recall from Lemma A.2 that

[TABLE]

We proceed by lower bounding $\sum_{f=1}^{n}|\widehat{G}_{f-k_{1}j}|^{2}\cdot|\widehat{X}_{f}|^{2}$ and upper bounding $\sum_{f=1}^{n}|\widehat{G}_{f-k_{1}j}|\cdot|\widehat{X}_{f}|^{2}$. Starting with the former, recalling that I_{j}=\big{(}(j-1/2)k_{1},(j+1/2)k_{1}\big{]}\cap\mathbb{Z}, we have

[TABLE]

where the second line is by the second filter property in Definition 2.1, and the third line is by the choice of $F$ in Definition 2.2.

Next, we upper bound $\sum_{f=1}^{n}|\widehat{G}_{f-k_{1}j}|\cdot|\widehat{X}_{f}|^{2}$ as follows:

[TABLE]

By the third property in Definition 2.1, the filter decays as $|\widehat{G}_{f}|\leq(\frac{1}{4})^{F-1}(\frac{k_{1}}{|f|})^{F-1}$ for $|f|\geq k_{1}$, and therefore the second term in (53) is bounded by

[TABLE]

where the second line follows from $|j^{\prime}-j|-1\geq\frac{|j^{\prime}-j|}{2}$, and the third line follows since the choice of $F$ in Definition 2.2 ensures that \big{(}\frac{1}{2}\big{)}^{F-1}\leq\delta. We bound the term $\sum_{f\in I_{j}\cup I_{j-1}\cup I_{j+1}}|\widehat{G}_{f-k_{1}j}|\cdot|\widehat{X}_{f}|^{2}$ in (53) using the first property in Definition 2.1, namely, $\widehat{G}_{f}\leq 1$:

[TABLE]

Hence, combining (53)–(55), we obtain

[TABLE]

The first claim of the lemma follows by combining (51), (52), and (56). ∎

Proof of second part of Lemma 2.3: By following the same steps as those used to handle (53), we obtain the following analog of (56) with $|\widehat{G}_{f}|^{2}$ in place of $|\widehat{G}_{f}|$:

[TABLE]

Combining (51), (56), and (57), we obtain

[TABLE]

and summing over $j\in[n]$ gives

[TABLE]

The double summation is upper bounded by $\sum_{j^{\prime}\in[n]}\|\widehat{X}_{I_{j^{\prime}}}\|_{2}^{2}\cdot 2\sum_{\Delta=1}^{\infty}\frac{1}{\Delta^{F-1}}=2\|\widehat{X}\|_{2}^{2}\cdot\sum_{\Delta=1}^{\infty}\frac{1}{\Delta^{F-1}}$, which in turn is upper bounded by $3\|\widehat{X}\|_{2}^{2}$ for $F\geq 4$, a condition guaranteed by Definition 2.2. We can therefore upper bound (58) by $\|\widehat{X}\|_{2}^{2}(3(1+3\delta)+3(3\delta^{2}+\delta))$, which is further upper bounded by $6\|\widehat{X}\|_{2}^{2}$ for $\delta\leq\frac{1}{20}$, as is assumed in Definition 2.2.

For the lower bound, we sum the first part of the lemma over all $j$, yielding

[TABLE]

We showed above that the double summation is upper bounded by $3\|\widehat{X}\|_{2}^{2}$, yielding an lower bound of $(1-\delta-9\delta-3\delta^{2})\|\widehat{X}\|_{2}^{2}$. This is lower bounded by $(1-12\delta)\|\widehat{X}\|_{2}^{2}$ for $\delta\leq\frac{1}{20}$. ∎

Appendix B Omitted Proofs from Section 3

B.1 Proof of Lemma 3.2

Note that for any j\in\big{[}\frac{n}{k_{1}}\big{]}, solving the first part of Lemma 2.3 for $\|\widehat{X}_{I_{j}}\|_{2}^{2}$ gives

[TABLE]

We will sum both sides over $j\in S^{*}\backslash\tilde{S}$; we proceed by analyzing the resulting terms.

Second term in (59) summed over $j\in S^{*}\backslash\tilde{S}$: We have

[TABLE]

where the last line follows by expanding the double summation to all j,j^{\prime}\in\big{[}\frac{n}{k_{1}}\big{]} with $j\neq j^{\prime}$, noting that $2\sum_{\Delta=1}^{\infty}\frac{1}{\Delta^{F-1}}\leq 2.5$ for $F\geq 4$ (a condition guaranteed by Definition 2.2), and then applying the assumption $\delta\leq\frac{1}{20}$.

First term in (59) summed over $j\in S^{*}\backslash\tilde{S}$: We first rewrite the sum of squares in terms of a weighted sum of fourth moments:

[TABLE]

by Cauchy-Schwarz applied to the length-$2k_{1}$ vectors containing entries $\|\widehat{Z}^{r}\|_{2}$ and $\frac{\|\widehat{Z}^{r}_{S^{*}\backslash\tilde{S}}\|_{2}^{2}}{\|\widehat{Z}^{r}\|_{2}}$.

The second summation inside the square root is upper bounded as

[TABLE]

where the first inequality follows since $\|\widehat{Z}^{r}_{S^{*}\backslash\tilde{S}}\|_{2}^{2}\leq\|\widehat{Z}^{r}_{S^{*}}\|_{2}^{2}$ and the second inequality uses \|\widehat{Z}^{r}_{S^{*}}\|_{2}^{2}\leq\gamma^{r}+\big{|}\|\widehat{Z}^{r}_{S^{*}}\|_{2}^{2}-\gamma^{r}\big{|}_{+}.

Now observe that by definition of $\tilde{S}$ (Definition 3.1), for every $j\notin\tilde{S}$, we have

[TABLE]

and summing both sides over all $j\in S^{*}\backslash\tilde{S}$ gives

[TABLE]

since $|S^{*}|\leq 10k_{0}$ by assumption. Applying this to the first term in (62), as well as $\frac{\|\widehat{Z}^{r}_{S^{*}\backslash\tilde{S}}\|_{2}^{2}}{\|\widehat{Z}^{r}\|_{2}^{2}}\leq 1$ for the second term, we obtain

[TABLE]

where we have applied the assumption (*) of the lemma.

Finally, substituting (63) into (61) yields

[TABLE]

where the last inequality uses the fact that $\frac{\sum_{r\in[2k_{1}]}\|\widehat{Z}^{r}\|_{2}^{2}}{2k_{1}}\leq 6\|X\|_{2}^{2}$ by the second part of Lemma 2.3. The proof is concluded by substituting (60) and (64) into (59), and using the assumption $\delta\leq\frac{1}{20}$ to deduce that \frac{1}{1-\delta}\big{(}43\sqrt{\delta}+10\delta\big{)}\leq 100\sqrt{\delta}.

Appendix C Omitted Proofs from Section 4

C.1 Proof of Lemma 4.2

The (exact) Fourier transform of $U$, denoted by $\widehat{U}^{*}$, is given by

[TABLE]

where we used the fact that $\omega_{B}^{(\cdot)}$ is periodic with period $B$, and then applied $\omega_{B}=\omega_{n}^{n/B}$. We see that (65) is the Fourier transform of the signal $\{X_{\sigma(\Delta+i)}G_{i}\}_{i\in[n]}$ evaluated at frequency $jn/B$, and hence, since multiplication and convolution are dual under the Fourier transform, we obtain

[TABLE]

where $Y_{i}=X_{\sigma(\Delta+i)}$. Now, by standard Fourier transform properties, we have $\widehat{Y}_{f}=\widehat{X}_{\sigma^{-1}f}\omega_{n}^{\Delta f}$, and substitution into (66) gives

[TABLE]

where we have used the assumed symmetry of $G$ about zero.

C.2 Proof of Lemma 4.3

For brevity, let \Psi=\sum_{f^{\prime}\neq f}|\widehat{X}_{f^{\prime}}|^{2}\mathbb{E}_{\pi}\big{[}|G_{o_{f}(f^{\prime})}|^{2}\big{]} denote the left-hand side of (14). Following the approach of [IK14, Lemma 3.3], we define the intervals \mathcal{F}_{t}=\big{(}\pi(f)-\frac{n}{B}2^{t},\pi(f)+\frac{n}{B}2^{t}\big{]} for $t=1,\dotsc,\log_{2}b$, and write

[TABLE]

where the second line follows by (i) upper bounding $\mathbb{P}[\pi(f^{\prime})\in\mathcal{F}_{t}\backslash\mathcal{F}_{t-1}]\leq\mathbb{P}[\pi(f^{\prime})\in\mathcal{F}_{t}]$ and applying the approximate pairwise independence property (cf., Definition 4.1); (ii) using the fact that there are at most $\frac{n}{B}\cdot 2^{t+1}$ integers within $\mathcal{F}_{t}$, and applying $|\widehat{G}_{f}|\leq 1$ for the case $t=1$.

To handle the term containing $|\widehat{G}_{o_{f}(f^{\prime\prime})}|^{2}$, we use the triangle inequality to write

[TABLE]

For any $f^{\prime\prime}$ with $\pi(f^{\prime\prime})\notin\mathcal{F}_{t-1}$, we have $|\pi(f)-\pi(f^{\prime\prime})|\geq\frac{n}{B}2^{t-1}$, and hence $|o_{f}(f^{\prime\prime})|\geq\frac{n}{B}(2^{t-1}-1)$. As a result, for $t\geq 2$, the third property in Definition 2.1 gives

[TABLE]

and hence

[TABLE]

This sum is less than $\frac{1}{2}$ for all $F\geq 2$, and hence substitution into (67) gives $\Psi\leq\frac{10}{B}\|\widehat{X}\|_{2}^{2}$, as desired.

C.3 Proof of Lemma 4.5

We use techniques resembling those used for a $(k_{1},\epsilon)$-downsampling in Section 2, but with the notable difference of using a more rapidly-decaying filter with bounded support in frequency domain.

Choice of filter: We let $G\in\mathbb{R}^{n}$ be the filter used in [IKP14] (as opposed to that used in Definition 2.1), satisfying the following:

• •

There exists an ideal filter ${G}^{\prime}$ satisfying $G^{\prime}_{f}\in[0,1]$ for all $f$, and

[TABLE]

such that $\|{G}-{G}^{\prime}\|_{2}\leq n^{-c}$;

• •

$\widehat{G}$ is supported on a window of length $O(ck_{1}\log n)$ centered at zero;

• •

Each entry of $\widehat{G}$ can be computed in time $O(1)$.

Intuition behind the proof. Before giving the details, we provide the intuition for the proof. Recall that our goal is to compute $X^{r}_{j}$ for $|j|\leq k_{0}/2$, for all $r\in[2k_{1}]$. To do this, we first note that $X^{0}_{j}$, for $|j|\leq k_{0}/2$ (i.e., for only one value of $r$, namely [math]), can be computed via a reduction to standard semi-equispaced FFT (Lemma 4.4) on an input signal of length $2n/k_{1}$. To achieve this, consider the signal $X\cdot G$ aliased to length $2n/k_{1}$, which is close to $X$ on all points $j$ such that $|j|\leq n/(2k_{1})$. In order to compute $X^{0}_{j}$ for $|j|\leq k_{0}/2$, it essentially suffices (modulo boundary issues; see below) to calculate $(X\cdot G)_{j}$ for $|j|\leq k_{0}/2$. We show below that this can be achieved using Lemma 4.4, because multiplication followed by aliasing are dual to convolution and subsampling: the input $(k_{0},k_{1})$-block sparse signal of length $n$ can be naturally mapped to an $O(k_{0}\log n)$-sparse signal in a reduced space with $\approx n/k_{1}$ points, in which standard techniques (Lemma 4.4) can be applied.

This intuition only shows how to compute the values of $X^{r}_{j}$ for $r=0$ and $|j|\leq k_{0}/2$, but we need the values for all $r\in[2k_{1}]$. As we show below, the regular structure of the set of shifts that we are interested in allows us use the standard FFT on a suitably defined set of length-$2k_{1}$ signals, without increasing the runtime by a $k_{1}$ factor. It is interesting to note that our runtime is $O(\log n)$ worse than the runtime of Lemma 4.4 due to the two-level nature of our scheme; this is for reasons similar to the $\log^{d}n$ scaling of runtime of high-dimensional semi-equispaced FFT (e.g. [GHI*+*13, Kap16]).

We now give the formal proof of the lemma.

Computing a convolved signal: Here we show that we can efficiently compute the values $\widehat{Y}_{j}^{r}=(\widehat{X}^{r}\star\widehat{G})_{\frac{k_{1}}{2}j}$ at all j\in\big{[}\frac{2n}{k_{1}}\big{]} where it is non-zero, for all values of $r\in[2k_{1}]$. We will later show that applying Lemma 4.4 to these signals (as a function of $j$) gives accurate estimates of the desired values of ${X}$.

Note that in the definition of $\widehat{Y}_{j}^{r}$, each non-zero block is convolved with a filter of support $O(ck_{1}\log n)$, and so contributes to at most $O(c\log n)$ values of $j$. Since there are $k_{0}$ non-zero blocks, there are $O(ck_{0}\log n)$ values of $j$ for which the result is non-zero.

The procedure is as follows:

For all $j$ such that $\widehat{Y}_{j}^{r}$ may be non-zero ($O(ck_{0}\log n)$ in total), compute $\widetilde{Y}_{j}^{b}=\frac{k_{1}}{2}\sum_{l=1}^{\frac{n}{2k_{1}}}\widehat{X}_{b+2k_{1}l}\widehat{G}_{\frac{k_{1}}{2}j-(b+2k_{1}l)}$ for $b\in[2k_{1}]$. That is, alias the signal $\{\widehat{X}_{f}\widehat{G}_{\frac{k_{1}}{2}j-f}\}$ down to length $2k_{1}$, and normalize by $\frac{2}{k_{1}}$ (for later convenience). Since $\widehat{G}$ is supported on an interval of length $O(ck_{1}\log n)$, this can be done in time $O(c\log n)$ per entry, for a total of $O(ck_{1}\log n)$ per $j$ value, or $O(c^{2}k_{0}k_{1}\log^{2}n)$ overall. 2. 2.

Compute the length-$2k_{1}$ inverse FFT of $\widetilde{Y}_{j}=(\widetilde{Y}_{j}^{1},\dotsc,\widetilde{Y}_{j}^{2k_{1}})$ to obtain $\widehat{Y}_{j}\in\mathbb{C}^{2k_{1}}$. This can be done in time $O(k_{1}\log(1+k_{1}))$ per $j$ value, or $O(k_{0}k_{1}\log(1+k_{1})\log n)$ overall.

We now show that $\widehat{Y}_{j}^{r}=\frac{k_{1}}{2}({X}^{r}\star{G})_{\frac{k_{1}}{2}j}$ for $r=1,\ldots,2k_{1}$. By the definition of the inverse Fourier transform, we have

[TABLE]

where the second line is by the definition of $\widetilde{Y}^{b}$, the third by the periodicity of $\omega_{2k_{1}}$, and the fifth since translation and phase shifting are dual under the Fourier transform. Hence, $\widehat{Y}_{j}^{r}=\frac{k_{1}}{2}(\widehat{X}^{r}\star\widehat{G})_{\frac{k_{1}}{2}j}$.

Applying the standard semi-equispaced FFT: For $r\in[2k_{1}]$, define $\widehat{Y}^{r}=(\widehat{Y}^{r}_{1},\dotsc,\widehat{Y}^{r}_{n/k_{1}})$. We have already established that the support of each $\widehat{Y}^{r}$ is a subset of a set having size at most $k^{\prime}=O(ck_{0}\log n)$. We can therefore apply Lemma 4.4 with $\zeta=n^{-(c+1)}$ to conclude that we can evaluate ${Y}^{r}_{j}$ for $|j|\leq\frac{k^{\prime}}{2}$ satisfying

[TABLE]

where ${Y}^{*r}$ is the exact inverse Fourier transform of $\widehat{Y}^{r}$. Moreover, this can be done in time O\big{(}k^{\prime}\log\frac{n/k_{1}}{n^{-(c+1)}}\big{)}=O(c^{2}k_{0}\log^{2}n) per $r$ value, or $O(c^{2}k_{0}k_{1}\log^{2}n)$ overall.

Proof of first part of lemma: It remains to show that the above procedure produces estimates of the desired $X$ values of the form (15).

Recall that $\widehat{Y}_{j}^{r}=\frac{k_{1}}{2}(\widehat{X}^{r}\star\widehat{G})_{\frac{k_{1}}{2}j}$. By the convolution theorem and the fact that subsampling and aliasing are dual (e.g., see Appendix C.1), the inverse Fourier transform of $\widehat{Y}^{r}$ satisfies the following when $|j|\leq\frac{n}{k_{1}}$:

[TABLE]

where the last line follows from the definition of $G^{\prime}$ in (68) and the assumption $\|G-G^{\prime}\|_{2}\leq n^{-c}$.

Combining (69) and (70) and using the triangle inequality, we obtain

[TABLE]

Since we have already shown that we can efficiently compute ${Y}^{r}_{j}$ for $|j|\leq\frac{k^{\prime}}{2}$ with $k^{\prime}=O(ck_{0}\log n)$, it only remains to show that $\|{Y}^{r}\|_{2}\leq n\|{X}\|^{2}$. To do this, we use the first line of (70) to write

[TABLE]

where the first line is the triangle inequality, the second line follows since the first filter assumption above ensures that $|G_{j}|\leq 2$ for all $j$, and the third line follows since the squared $\ell_{1}$-norm is upper bounded by the squared $\ell_{2}$-norm times the vector length.

Squaring both sides of (71) and summing over all $j$ gives $\|{Y}^{r}\|_{2}^{2}\leq 2k_{1}\|{X}\|^{2}\leq n^{2}\|{X}\|^{2}$ (under the trivial assumption $n\geq 2$), thus completing the proof.

Proof of second part of lemma: In the proof of the first part, we applied Lemma 4.4 to signals of length $\frac{2n}{k_{1}}$. It follows directly from the arguments in [IKP14, Cor. 12.2] that since we can approximate the entries of $X^{r}_{j}$ for all $|j|\leq\frac{k_{0}}{2}$, we can do the same for all $j$ equaling $\sigma j^{\prime}+b$ modulo-$\frac{2n}{k_{1}}$ for some $|j^{\prime}|\leq\frac{k_{0}}{2}$. Specifically, this follows since the multiplication by $\sigma$ and shift by $b$ simply amounts to a phase shift and a linear change of variables $f\to\sigma^{-1}f$ in frequency domain, both of which can be done in constant time.

However, the second part of the lemma regards indices modulo-$\frac{n}{k_{1}}$, as opposed to modulo-$\frac{2n}{k_{1}}$. To handle the former, we note that for any integer $a$, we either have $a\text{ mod }\frac{n}{k_{1}}=a\text{ mod }\frac{2n}{k_{1}}$ or a\text{ mod }\frac{n}{k_{1}}=\big{(}a+\frac{n}{k_{1}}\big{)}\text{ mod }\frac{2n}{k_{1}}. Hence, we obtain the desired result by simply performing two calls to the first part, one with a universal shift of $\frac{n}{k_{1}}$.

C.4 Proof of Lemma 4.6

C.4.1 First Part

Since $U_{X}$ is computed according to $X$ itself in Algorithm 4, we only need to compute the error in $U_{\chi}$.

In the definition of hashing in Definition 4.2, since $G$ has support $O(FB)$, we see that the values of $X$ used correspond to a permutation of an interval having length $k^{\prime}=O(FB)$. We can therefore apply the second part of Lemma 4.4 with sparsity $k^{\prime}$ and parameter $\zeta=n^{-c^{\prime}}$ for some $c^{\prime}>0$, ensuring an $\ell_{\infty}$-guarantee of $n^{-c^{\prime}}\|\chi\|_{2}$ for the signal $\chi$.

Since $\widehat{U}$ is computed from these values using (13) followed by the FFT, we readily obtain via the relation $\|v\|_{\infty}\leq\|v\|_{2}\leq\sqrt{m}\|v\|_{\infty}$ (for $v\in\mathbb{C}^{m}$) and Parseval’s theorem that $\widehat{U}$ has an $\ell_{\infty}$-guarantee of $n^{-(c^{\prime}-O(1))}\|\chi\|_{2}$, which can be made to equal $n^{-c}\|\widehat{\chi}\|_{2}$ by choosing $c^{\prime}=c+O(1)$.

Sample complexity and runtime:

The only operation that consumes samples from the signal $X$ is the hashing operation applied to $X$. From the definition of hashing in Definition 4.2, and the fact that the filter $G$ has support $O(FB)$, we find that the sample complexity is also $O(FB)$.

The runtime is dominated by the application of the semi-equispaced FFT, which is $O(cF(\|\widehat{\chi}\|_{0}+B)\log n)$ by Lemma 4.4. In particular, this dominates the $O(B\log B)$ time to perform the FFT in Algorithm 4, and the hashing operation, whose time complexity is the same as the sample complexity.

C.4.2 Second Part

Recall the definition of a $(k_{1},\delta)$-downsampling of a signal $X$ from (2):

[TABLE]

In order to compute the \big{(}\frac{n}{k_{1}},B^{r},G^{r},\sigma,\Delta\big{)}-hashing of $\widehat{Z}^{r}$ (cf., Definition 4.2), we use the samples of $Z^{r}_{j}$ at the locations $j=\sigma(j^{\prime}+\Delta)\mod\frac{n}{k_{1}}$ for $|j^{\prime}|\leq FB^{r}$; this is because $G^{r}$ is supported on $[-FB^{r},+FB^{r}]$. Note that $FB^{r}$ is further upper bounded by $O(FB_{\mathrm{max}})$.

We claim that in the second part of Lemma 4.5, it suffices to set the sparsity level to $O(FB_{\mathrm{max}}+k_{0})$. To see this, first note that $k_{0}$ is added in accordance with Remark 4.2 and the fact that $\widehat{\chi}$ is $(k_{0},k_{1})$-block sparse. Moreover, note that $\widehat{Z}^{r}$ has length $\frac{n}{k_{1}}$, and one sample of $Z^{r}_{j}$ can be computed from $X^{r}_{j+i\frac{n}{k_{1}}}=X^{r+2i}_{j}$ for $|i|\leq F$ as per Definition 2.2 and the fact that the filter $G$ is supported on $[-F\frac{n}{k_{1}},+F\frac{n}{k_{1}}]$. Therefore, all we need is $X^{r^{\prime}}_{j^{\prime}}$ for each $r^{\prime}\in[2k_{1}]$ and for all $j^{\prime}=\sigma(j+\Delta)\mod\frac{n}{k_{1}}$ with $|j|\leq FB_{\mathrm{max}}$.

Applying the second part of Lemma 4.5 with sparsity $O(FB_{\mathrm{max}}+k_{0})$ and parameter $\zeta=n^{-c^{\prime}}$ for some $c^{\prime}>0$, ensuring an $\ell_{\infty}$-guarantee of $2n^{-c^{\prime}}\|\chi\|_{2}$ on the computed values of $\chi$. By an analogous argument to the first case, this implies an $\ell_{\infty}$-guarantee of $n^{-c}\|\chi\|_{2}$ on the FFT $\widehat{U}^{r}$ of the hashing of $Z_{\chi}^{r}$, with $c=c^{\prime}+O(1)$.

Sample complexity and runtime:

We take $O(FB^{r})$ samples of the $r$-th downsampled signal each time we do the hashing, separately for each $r\in[2k_{1}]$. By Lemma 2.2, accessing a single sample of $Z^{r}_{X}$ costs us $O(\log\frac{1}{\delta})$ samples of $X$. Hence, the sample complexity is O\big{(}F\sum_{r\in[2k_{1}]}B^{r}\log\frac{1}{\delta}\big{)}.

We now turn to the runtime. By Lemma 4.5, the call to SemiEquiInverseBlockFFT with $O(FB_{\mathrm{max}}+k_{0})$ in place of $k_{0}$ takes time O\big{(}c^{2}(FB_{\mathrm{max}}+k_{0})k_{1}\log^{2}n). The hashing operation’s runtime matches its sample complexity, and since we have assumed $\delta\geq\frac{1}{n}$, its contribution is dominated by the preceding term.

C.5 Proof of Lemma 4.7

First part of lemma: We start with the following bound on the expression inside the expectation:

[TABLE]

where $h(S)=\{h(j):j\in S\}$ with h(j)=\mathrm{round}\big{(}\pi(j)\frac{B}{m}\big{)}, denoting the bucket into which element $j$ hashes. We define $S_{\mathrm{coll}}$ to be a subset of $S$ containing the elements that collide with each other, i.e., $S_{\mathrm{coll}}=\{j\in S\,|\,h(j)\cap h(S\backslash\{j\})\neq\emptyset\}$, yielding

[TABLE]

where the final line follows from the inequality $[a+b]_{+}\leq|a|+[b]_{+}$.

Bounding the first term in (72): We start by evaluating the expected value of the term corresponding to $S_{\mathrm{coll}}$ over the random permutation $\pi$:

[TABLE]

where the second line follows from the union bound, and the third line follows since $\pi$ is approximately pairwise independent as per Definition 4.1.

Bounding the second term in (72): We apply Lemma 4.2 to obtain $\widehat{U}^{*}_{h(j)}=\sum_{j^{\prime}\in[m]}\widehat{Y}_{j^{\prime}}\widehat{H}_{o_{j}(j^{\prime})}\omega_{m}^{\sigma\Delta j^{\prime}}$ with $o_{j}(j^{\prime})=\pi(j^{\prime})-h(j)\frac{m}{B}$. We write this as $\widehat{U}^{*}_{h(j)}=\widehat{Y}_{j}\widehat{H}_{o_{j}(j)}\omega_{m}^{\sigma\Delta j}+\mathrm{err}_{j}$ with $\mathrm{err}_{j}:=\sum_{j^{\prime}\in[m]\backslash\{j\}}\widehat{Y}_{j^{\prime}}\widehat{H}_{o_{j}(j^{\prime})}\omega_{m}^{\sigma\Delta j^{\prime}}$, yielding

[TABLE]

by $|\xi|_{+}\leq|\xi|$ and the triangle inequality. We have by definition that $|o_{j}(j)|\leq\frac{m}{2B}$, and hence item 2 in Definition 2.1 yields H_{o_{j}(j)}\geq 1-\big{(}\frac{1}{4}\big{)}^{F^{\prime}-1}, which in turn implies H_{o_{j}(j)}^{2}\geq 1-2\big{(}\frac{1}{4}\big{)}^{F^{\prime}-1}. Combining this with $H_{f}\leq 1$ from item 1 in Definition 2.1, we can weaken (74) to

[TABLE]

We proceed by bounding the expected value of $|\mathrm{err}_{j}|^{2}$. We first take the expectation over $\Delta$, using Parseval’s theorem to write

[TABLE]

Taking the expectation over $\pi$, we obtain

[TABLE]

where the final line follows from Lemma 4.3.

Substituting into (75), and using Jensen’s inequality to write $\mathbb{E}[|\mathrm{err}_{j}|]\leq\sqrt{\mathbb{E}[|\mathrm{err}_{j}|^{2}]}$, we obtain

[TABLE]

where the second line follows from the fact that $\|v\|_{1}\leq\sqrt{|S|}\|v\|_{2}$ for any $v\in\mathbb{C}^{|S|}$, as well as \Big{(}\frac{1}{4}\Big{)}^{F^{\prime}-1}\leq\delta by the choice of $F^{\prime}$. The claim follows by substituting (73) and (76) into (72).

Second part of lemma: By the definition of $\widehat{U}^{*}$ (cf., Definition 4.2), we have

[TABLE]

by Parseval. Taking the expectation with respect to $\pi$, we obtain

[TABLE]

where the final line follows by noting that $\pi(j)-b\frac{m}{B}$ is uniformly distributed over $[m]$, and applying the second part of Lemma 2.1.

Appendix D Omitted Proofs from Section 5

D.1 Proof of Lemma 5.3

Note on $\frac{1}{\mathrm{poly}(n)}$ assumptions in lemmas:

Throughout the proof, we apply Lemmas 3.5, 5.1, and 5.2. The first of these assumes that $\widehat{\chi}_{0}$ uniformly distributed over an arbitrarily length-\Omega\big{(}\frac{\|\widehat{\chi}\|^{2}}{\mathrm{poly}(n)}\big{)} interval, and the latter two use the assumption $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}^{2}$.

We argue that these assumptions are trivial and can be ignored. To see this, we apply a minor technical modification to the algorithm as follows. Suppose the implied exponent to the $\mathrm{poly}(n)$ notation is $c^{\prime}$. By adding a noise term to $\widehat{\chi}_{0}$ on each iteration uniform in $[-n^{-c^{\prime}+10}\|\widehat{\chi}\|^{2},n^{c^{\prime}+10}\|\widehat{\chi}\|^{2}]$, we immediately satisfy the first assumption above, and we also find that the probability of $\|\widehat{X}-\widehat{\chi}\|_{2}<\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}^{2}$ is at most $O(n^{-10})$, and the additional error in the estimate is $O(n^{c^{\prime}+10}\|\widehat{\chi}\|^{2})$. Since we only do $O(\log\mathrm{SNR}^{\prime})=O(n)$ iterations (by the assumption $\mathrm{SNR}^{\prime}=O(\mathrm{poly}(n))$), this does not affect the result because the accumulated noise added to $\widehat{\chi}_{0}$ which we denote by $\mathrm{err}(\widehat{\chi}_{0})$, does not exceed $\frac{\|\widehat{X}\|_{2}^{2}}{\mathrm{poly}(n)}$ which by the final assumption of the lemma implies that $\mathrm{err}(\widehat{\chi}_{0})\leq\nu^{2}$.

Overview of the proof:

We introduce the approximate support set of the input signal $\widehat{X}$, given by the union of the top $k_{0}$ blocks of the signal and the blocks whose energy is more than the tail noise level:

[TABLE]

From the definition of $\mu^{2}$ in Definition 1.2, we readily obtain $|S_{0}|\leq 2k_{0}$. For each $t=1,2,...,T$, define the set $S_{t}$ as

[TABLE]

where $L^{\prime}_{t}$ is the output of PruneLocation at iteration $t$ of ReduceSNR. $S_{t}$ contains the set of the head elements of $\widehat{X}$, plus every element that is modified by the algorithm so far.

We prove by induction on the iteration number $t=1,\dotsc,T$ that there exist events $\mathcal{E}_{0}\supseteq\mathcal{E}_{1}\supseteq...\supseteq\mathcal{E}_{T}$ such that conditioned on $\mathcal{E}_{t}$, the following conditions hold true:

1. a.

$|S_{t}|\leq 2k_{0}+\frac{tk_{0}}{T}$; 2. b.

$\|\widehat{\chi}^{(t)}_{I_{j}}\|_{2}^{2}=0$ for all j\in\big{[}\frac{n}{k_{1}}\big{]}\backslash S_{t}; 3. c.

$\|\widehat{X}-\widehat{\chi}^{(t)}\|_{2}^{2}\leq 99\cdot\mathrm{SNR}^{\prime}(k_{0}\nu^{2})/2^{t}$;

and for each $t\leq T$, we have $\mathbb{P}[\mathcal{E}_{t+1}|\mathcal{E}_{t}]\geq 1-\frac{1}{10T}$.

Base case of the induction:

We have already deduced that $|S_{0}|\leq 2k_{0}$ and defined $\widehat{\chi}^{(0)}=0$, and we have $\|\widehat{X}-\widehat{\chi}^{(0)}\|_{2}^{2}=\|\widehat{X}\|_{2}^{2}\leq\mathrm{SNR}^{\prime}\cdot(k_{0}\mu^{2})/2^{0}$ by the two assumptions of the lemma. Hence, we can let $\mathcal{E}_{0}$ be the trivial event satisfying $\mathbb{P}[\mathcal{E}_{0}]=1$.

Inductive step:

We seek to define an event $\mathcal{E}_{t+1}$ that occurs with probability at least $1-\frac{1}{10T}$ conditioned on $\mathcal{E}_{t}$, and such that the induction hypotheses a, b, and c are satisfied for $t+1$ conditioned on $\mathcal{E}_{t+1}$. To do this, we will introduce three events $\mathcal{E}_{\mathrm{loc},t}$, $\mathcal{E}_{\mathrm{prune},t}$, and $\mathcal{E}_{\mathrm{est},t}$, and set $\mathcal{E}_{t+1}=\mathcal{E}_{\mathrm{loc},t}\cap\mathcal{E}_{\mathrm{prune},t}\cap\mathcal{E}_{\mathrm{est},t}\cap\mathcal{E}_{t}$. Throughout the following, we let $\delta$, $\theta$, and $p$ be chosen as in Algorithm 6

Success event associated with MultiBlockLocate:

Let $\mathcal{E}_{\mathrm{loc},t}$ be the event of having a successful run of MultiBlockLocate($X,\widehat{\chi}^{(t)},n,k_{1},k_{0},\delta,p$) at iteration $t+1$ of the algorithm, meaning the following conditions on the output $L$:

[TABLE]

where $C$ is a constant to be specified shortly, for a small enough $\delta$. To show this, we invoke Lemma 3.5 with $S^{*}=S_{t}$. Note that inductive hypothesis (a) implies $|S_{t}|\leq(2+\frac{t}{\log\mathrm{SNR}^{\prime}})k_{0}\leq 3k_{0}$. By the first part of Lemma 3.5, we have \mathbb{E}\big{[}|L|\big{]}\leq C^{\prime}\frac{k_{0}}{\delta}\log\frac{k_{0}}{\delta}\log\frac{1}{p}\log^{2}\frac{1}{\delta p} for an absolute constant $C^{\prime}$, and hence (78) follows with $C=100C^{\prime}$ and probability at least $1-\frac{1}{100T}$, by Markov’s inequality.

By the second part of Lemma 3.5, (79) holds with probability at least $1-p$ provided that \delta\leq\big{(}\frac{0.1}{200}\big{)}^{2}, so by the union bound, the event $\mathcal{E}_{\mathrm{loc},t}$ occurs with probability at least $\mathbb{P}[\mathcal{E}_{\mathrm{loc},t}|\mathcal{E}_{t}]\geq 1-p-\frac{1}{100T}$.

Success event associated with PruneLocation:

Let $\mathcal{E}_{\mathrm{prune},t}$ be the event of having a successful run of PruneLocation($X,\widehat{\chi}^{(t)},L,k_{0},k_{1},\delta,p,n,\theta$) at iteration $t+1$ of the algorithm, meaning the following conditions on the output $L^{\prime}$:

[TABLE]

The probability of (80) holding:

In order to bound $|L^{\prime}\backslash S_{t}|$, first recall that the set $S_{\mathrm{tail}}$, defined in Lemma 5.1 part (a), has the following form:

[TABLE]

By substituting $\theta=10\cdot 2^{-(t+1)}\cdot\nu^{2}(\mathrm{SNR}^{\prime})$ and using $\|\widehat{X}-\widehat{\chi}^{(t)}\|_{2}^{2}\leq 99\cdot\mathrm{SNR}^{\prime}(k_{0}\nu^{2})/2^{t}$ from part c of the inductive hypothesis, we have

[TABLE]

where the last inequality holds when $\delta$ is sufficiently small. Hence,

[TABLE]

Now, to prove that (80) holds with high probability, we write

[TABLE]

To upper bound the first term, note that by the first part of Lemma 5.1, we have

[TABLE]

and hence by Markov’s inequality, the following holds with probability at least $1-\frac{1}{100T}$:

[TABLE]

where the third line follows from (78) (we condition on $\mathcal{E}_{\mathrm{loc},t}$), and the fifth line follows from $T=\log\mathrm{SNR}^{\prime}$ and the choice $p=\frac{\delta}{\log^{2}\frac{k_{0}}{\delta}\log^{4}\mathrm{SNR}^{\prime}}$ in Algorithm 6. Again using this choice of $p$, we claim that $\frac{100C\delta\log^{3}\frac{1}{\delta p}}{\log\frac{k_{0}}{\delta}\cdot\log\mathrm{SNR}^{\prime}}\leq 1$ for sufficiently small $\delta$ regardless of the values $(k_{0},\mathrm{SNR}^{\prime})$; this is because the dependence of $1/p$ on $k_{0}$ and $\mathrm{SNR}^{\prime}$ is logarithmic, so in the numerator contains $\log^{3}\log k_{0}$ and $\log^{3}\log\mathrm{SNR}^{\prime}$ while the denominator contains $\log k_{0}$ and $\log^{2}\mathrm{SNR}^{\prime}$. Hence,

[TABLE]

with probability at least $1-\frac{1}{100T}$.

We now show that the second term in (83) is zero, by showing that $S_{\mathrm{tail}}\cup S_{t}=[\frac{n}{k_{1}}]$. To see this, note that the term $\nu^{2}(\mathrm{SNR}^{\prime})/2^{t+1}$ in the bound on $S_{\mathrm{tail}}$ in (82) satisfies

[TABLE]

by applying $t\leq T=\log\mathrm{SNR}^{\prime}$, followed by the first assumption of the lemma. Hence,

[TABLE]

By part b of the inductive hypothesis, we have $\|(\widehat{X}-\widehat{\chi}^{(t)})_{I_{j}}\|_{2}^{2}=\|\widehat{X}_{I_{j}}\|_{2}^{2}$ for all $j\notin S_{t}$, and hence

[TABLE]

But from (77), we know that $S_{0}$ (and hence $S_{t}$) contains all $j$ with $\|\widehat{X}_{I_{j}}\|_{2}^{2}>4\mu^{2}$, so we obtain S_{\mathrm{tail}}\backslash S_{t}\supset\big{[}\frac{n}{k_{1}}\big{]}\backslash S_{t}, and hence S_{\mathrm{tail}}\cup S_{t}=(S_{\mathrm{tail}}\backslash S_{t})\cup S_{t}=\big{[}\frac{n}{k_{1}}\big{]}, as required.

Therefore, the probability of (80) holding conditioned on $\mathcal{E}_{t}$ and $\mathcal{E}_{\mathrm{loc},t}$ is at least $1-\frac{1}{100T}$.

The probability of (81) holding:

To show (81), we use the second part of Lemma 5.1. The set $S_{\mathrm{head}}$ therein is defined as

[TABLE]

By substituting $\theta=10\cdot 2^{-(t+1)}\cdot\nu^{2}(\mathrm{SNR}^{\prime})$ and using $\|\widehat{X}-\widehat{\chi}^{(t)}\|_{2}^{2}\leq 99\cdot\mathrm{SNR}^{\prime}(k_{0}\nu^{2})/2^{t}$ from part c of the inductive hypothesis, we have

[TABLE]

for sufficiently small $\delta$, and hence

[TABLE]

Next, we write

[TABLE]

and we proceed by upper bounding the four terms.

Bounding the first term in (86):

By the second part of Lemma 5.1 and the use of Markov, we have

[TABLE]

with probability at least $1-p$.

Bounding the second term in (86):

Conditioned on $\mathcal{E}_{\mathrm{loc},t}$, we have

[TABLE]

where we have applied (79).

Bounding the third term in (86):

We have

[TABLE]

by (85). Part a of the inductive hypothesis implies that $|S_{t}|\leq 3k_{0}$, and hence

[TABLE]

Bounding the fourth term in (86):

[TABLE]

where the equality follows from part b of the inductive hypothesis, and the final step holds since $S_{t}$ contains all elements with $\|\widehat{X}_{I_{j}}\|_{2}^{2}\geq\mu^{2}$ (cf., (77)).

Adding the above four contributions and applying the union bound, we find that conditioned on $\mathcal{E}_{t}$ and $\mathcal{E}_{\mathrm{loc},t}$, (81) holds with probability at least $\mathbb{P}[\mathcal{E}_{\mathrm{prune},t}|\mathcal{E}_{t}\cap\mathcal{E}_{\mathrm{loc},t}]\geq 1-p-\frac{1}{100T}$, provided that $\delta$ is a sufficiently small constant ($\delta\leq 0.1$).

Success event associated with EstimateValues:

Let $\mathcal{E}_{\mathrm{est},t}$ be the event of having a successful run of EstimateValues($X,\widehat{\chi}^{(t)},L^{\prime},k_{0},k_{1},\delta,p$) at iteration $t+1$ of the algorithm conditioned on $\mathcal{E}_{t}$, meaning the following conditions on the output signal $W$:

[TABLE]

where $\mathcal{F}$ contains the frequencies within the blocks indexed by $L^{\prime}$. By Lemma 5.2 and the fact that $|L^{\prime}|\leq 3k_{0}$ conditioned on $\mathcal{E}_{\mathrm{prune},t}$, $\mathcal{E}_{\mathrm{loc},t}$, and $\mathcal{E}_{t}$, it immediately follows that $\mathcal{E}_{\mathrm{est},t}$ occurs with probability at least $\mathbb{P}[\mathcal{E}_{\mathrm{est},t}|\mathcal{E}_{\mathrm{prune},t}\cap\mathcal{E}_{\mathrm{loc},t}\cap\mathcal{E}_{t}]\geq 1-p$.

Combining the events:

We can now wrap everything up as follows:

[TABLE]

Substituting the probability bounds into the above equation, we have

[TABLE]

by the choice of $p$ in Algorithm 6 along with $T=\log\mathrm{SNR}$.

Now we show that the event $\mathcal{E}_{t+1}=\mathcal{E}_{\mathrm{loc},t}\cap\mathcal{E}_{\mathrm{prune},t}\cap\mathcal{E}_{\mathrm{est},t}\cap\mathcal{E}_{t}$ implies the induction hypothesis. Conditioned on $\mathcal{E}_{\mathrm{prune},t}\cap\mathcal{E}_{t}$, we have (80), which immediately gives part a. Conditioned on $\mathcal{E}_{\mathrm{loc},t}\cap\mathcal{E}_{\mathrm{est},t}$, from the definition $S_{t+1}=S_{t}\cup L^{\prime}$, part b of the inductive hypothesis follows from the fact that only elements in $L^{\prime}$ are updated. Finally, conditioned on $\mathcal{E}_{t}\cap\mathcal{E}_{\mathrm{prune},t}\cap\mathcal{E}_{\mathrm{est},t}$, we have

[TABLE]

where the second line holds since $W$ is non-zero only for the blocks indexed by $L^{\prime}$, the third line follows from (81) and (87), and the last line holds for sufficiently small $\delta$ from part c of the induction hypothesis, and the upper bound $\mu^{2}\leq 2\nu^{2}(\mathrm{SNR}^{\prime})/2^{t+1}$ given in (84).

The first part of the lemma now follows from a union bound over the $T$ iterations and the fact that $\mathrm{err}(\widehat{\chi}_{0})\leq\nu^{2}$, and by noting that the three parts of the induction hypothesis immediately yield the two claims therein. We conclude by analyzing the sample complexity and runtime.

Sample complexity:

We have from Lemma 3.5 that the expected sample complexity of MultiBlockLocate in a given iteration is O^{*}\big{(}\frac{k_{0}}{\delta}\log(1+k_{0})\log n+\frac{k_{0}k_{1}}{\delta^{2}}\big{)}, and multiplying by the number $T=O(\log\mathrm{SNR}^{\prime})$ of iterations gives a total of O^{*}\big{(}\log\mathrm{SNR}^{\prime}\big{(}\frac{k_{0}}{\delta}\log(1+k_{0})\log n+\frac{k_{0}k_{1}}{\delta^{2}}\big{)}\big{)}. Hence, by Markov’s inequality, this is also the total sample complexity across all calls to MultiBlockLocate with probability at least $1-\frac{1}{100}$; this probability can be combined with the union bound that we applied above. Since $\delta=\Omega(1)$, the above sample complexity simplifies to $O^{*}(k_{0}\log(1+k_{0})\log\mathrm{SNR}^{\prime}\log n+k_{0}k_{1}\log\mathrm{SNR}^{\prime})$.

By Lemma 5.1, the sample complexity of PruneLocation is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta p}\log\frac{1}{\delta})$, and by Lemma 5.2, the sample complexity of EstimateValues is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{p}\log\frac{1}{\delta})$. Substituting the choices of $\delta$ and $p$, these behave as $O^{*}(k_{0}k_{1})$ per iteration, or $O^{*}(k_{0}k_{1}\log\mathrm{SNR}^{\prime})$ overall.

Runtime:

By Lemma 3.5, the expected runtime of MultiBlockLocate in a given iteration is O^{*}\big{(}\frac{k_{0}}{\delta}\log(1+k_{0})\log^{2}n+\frac{k_{0}k_{1}}{\delta^{2}}\log^{2}n+\frac{k_{0}k_{1}}{\delta}\log^{3}n\big{)}. Moreover, by Lemma 5.1, the runtime of PruneLocation as a function of $|L|$ is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta p}\log\frac{1}{\delta}\log n+k_{1}\cdot|L|\log\frac{1}{\delta p})$, and substituting \mathbb{E}\big{[}|L|\big{]}=O\big{(}\frac{k_{0}}{\delta}\log\frac{k_{0}}{\delta}\log\frac{1}{p}\log^{2}\frac{1}{\delta p}\big{)} from Lemma 3.5, this becomes O^{*}\big{(}\frac{k_{0}k_{1}}{\delta}\log n\big{)} in expectation, by absorbing the $\log\frac{1}{\delta}$ and $\log\frac{1}{p}$ factors into the $O^{*}(\cdot)$ notation.

Summing the preceding per-iteration expected runtimes, multiplying by the number of iterations $T$, and substituting the choices of $T$, $p$ and $\delta$, we find that their combined expectation is $O^{*}(k_{0}\log k_{0}\log\mathrm{SNR}^{\prime}\log^{2}n+k_{0}k_{1}\log\mathrm{SNR}^{\prime}\log^{3}n)$. Hence, by Markov’s inequality, this is also the total sample complexity across all calls to MultiBlockLocate and PruneLocation with probability at least $1-\frac{1}{100}$. Applying the union bound over this failure event, $\bar{\mathcal{E}}_{T}$, and the $1/\mathrm{poly}(n)$ probability failure event arising from random perturbations of $\widehat{\chi}_{0}$, we obtain the required bound of $0.9$ on the success probability.

By Lemma 5.2, the runtime of EstimateValues is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{p}\log\frac{1}{\delta}\log n+k_{1}\cdot|L^{\prime}|\log\frac{1}{p})$, which behaves as $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{p}\log\frac{1}{\delta}\log n)$ conditioned on $\mathcal{E}_{\mathrm{prune},t}\cap\mathcal{E}_{t}$ (see (80) and recall that $|S_{t}|\leq 3k_{0}$). By our choices of $p$ and $\delta$, this simplifies to $O^{*}(k_{0}k_{1}\log n)$ per iteration, or $O^{*}(k_{0}k_{1}\log\mathrm{SNR}^{\prime}\log n)$ overall.

D.2 Proof of Lemma 5.4

The proof resembles that of Lemma 5.3, but is generally simpler, and has some differing details. We provide the details for completeness.

Note on $\frac{1}{\mathrm{poly}(n)}$ assumptions in lemmas:

We use an analogous argument to the start of Section D.1 to handle the assumptions $|\widehat{X}_{0}-\widehat{\chi}_{0}|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}^{2}$ and $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}^{2}$ in Lemmas 3.5, 5.1, and 5.2. Specifically, we add a noise term to $\widehat{\chi}_{0}$ uniform in $[-n^{-c^{\prime}+10}\|\widehat{\chi}\|^{2},n^{c^{\prime}+10}\|\widehat{\chi}\|^{2}]$. This does not affect the result, since the noise added to $\widehat{\chi}_{0}$ which we denote by $\mathrm{err}(\widehat{\chi}_{0})$, does not exceed $\frac{\|\widehat{X}\|_{2}^{2}}{\mathrm{poly}(n)}$ which by the assumptions of the lemma implies that $\mathrm{err}(\widehat{\chi}_{0})\leq\epsilon\nu^{2}$.

Overview of the proof:

We first introduce the approximate support set of the input signal $\widehat{X}-\widehat{\chi}$, given by the top $10k_{0}$ blocks of the signal:

[TABLE]

We also introduce another set indexing blocks whose energy is sufficiently large:

[TABLE]

It readily follows from this definition and Definition 1.2 that $|S_{\epsilon}\backslash S_{0}|\leq k_{0}/\epsilon$.

The function calls three other primitives, and below, we show that each of them succeeds with high probability by introducing suitable success events. Throughout, we let $\theta$, $p$, and $\eta$ be as chosen in Algorithm 6

Success event of the location primitive:

Let $\mathcal{E}_{\mathrm{loc}}$ be the event of having a successful run of MultiBlockLocate($X,\widehat{\chi},k_{1},k_{0},n,\epsilon^{2},p$), meaning the following conditions on the output $L$:

[TABLE]

where $C$ is a constant to be specified shortly. To verify these conditions, we invoke Lemma 3.5 with $S^{*}=S_{0}$. By the first part of Lemma 3.5, we have \mathbb{E}\big{[}|L|\big{]}\leq C^{\prime}\frac{k_{0}}{\epsilon^{2}}\log\frac{k_{0}}{\epsilon}\log^{3}\frac{1}{\epsilon p} for an absolute constant $C^{\prime}$, and hence (90) follows with $C=100C^{\prime}$ and probability at least $1-\frac{1}{100}$, by Markov’s inequality.

By the second part of Lemma 3.5 with $\delta=\epsilon^{2}$, (91) holds with probability at least $1-p$, so by the union bound, the event $\mathcal{E}_{\mathrm{loc}}$ occurs with probability at least $1-p-\frac{1}{100}$.

Success event of the pruning primitive:

Let $\mathcal{E}_{\mathrm{prune}}$ be the event of having a successful run of PruneLocation($X,\widehat{\chi},L,n,k_{0},k_{1},\epsilon,p,\theta$), meaning the following conditions on the output $L^{\prime}$:

[TABLE]

Bounding the probability of (92):

In order to bound $|L^{\prime}\backslash S_{0}|$, first note that the set $S_{\mathrm{tail}}$, defined in Lemma 5.1 part (a), has the following form:

[TABLE]

By substituting $\theta=200\cdot\epsilon\nu^{2}$ (cf., Algorithm 6) and using the assumption $\|\widehat{X}-\widehat{\chi}\|_{2}^{2}\leq 100k_{0}\nu^{2}$ in the lemma, we have

[TABLE]

Hence,

[TABLE]

Now, to prove that (92) holds with high probability, we write

[TABLE]

We first upper bound the first term as follows:

[TABLE]

which follows directly from the definition of $S_{\epsilon}$. To upper bound the second term in (95), note that by Lemma 5.1 part (a) with $\delta=\epsilon$,

[TABLE]

and hence by Markov’s inequality, the following holds with probability at least $1-\frac{1}{100}$:

[TABLE]

where the third line follows from (90) (we condition on $\mathcal{E}_{\mathrm{loc}}$), and the fifth line follows from and the choice $p=\frac{\eta\epsilon}{\log^{2}\frac{k_{0}}{\epsilon}}$ in Algorithm 6. Again using this choice of $p$, we claim that $\frac{100C\eta\epsilon\log^{3}\frac{1}{\epsilon p}}{\log\frac{k_{0}}{\delta}}\leq 1$ for sufficiently small $\eta$ regardless of the value of $k_{0}$; this is because the dependence of $1/p$ on $k_{0}$ is logarithmic, so the numerator contains $\log^{3}\log k_{0}$, while the denominator contains $\log k_{0}$ which means that the ratio is upper bounded and can be made arbitrarily small by choosing a small enough constant $\eta$. Hence

[TABLE]

with probability at least $1-\frac{1}{100}$.

We now show that the third term in (95) is zero, by showing that $S_{\mathrm{tail}}\cup S_{\epsilon}=[\frac{n}{k_{1}}]$. To see this, note that the term $\nu^{2}$ in the definition of $S_{\mathrm{tail}}$ is more than $\frac{\mathrm{Err}^{2}(\widehat{X}-\widehat{\chi},10k_{0},k_{1})}{k_{0}}$ by the first assumption of the lemma, and hence

[TABLE]

However, the definition of $S_{\epsilon}$ in (89) reveals that the condition upper bounding $\|(\widehat{X}-\widehat{\chi})_{I_{j}}\|_{2}^{2}$ is redundant, and S_{\mathrm{tail}}\backslash S_{\epsilon}\supset\big{[}\frac{n}{k_{1}}\big{]}\backslash S_{\epsilon}, and hence S_{\mathrm{tail}}\cup S_{\epsilon}=(S_{\mathrm{tail}}\backslash S_{\epsilon})\cup S_{\epsilon}=\big{[}\frac{n}{k_{1}}\big{]}.

Bounding the probability of (93):

To show (93), we use the second part of Lemma 5.1. The set $S_{\mathrm{head}}$ therein is defined as

[TABLE]

By substituting $\theta=200\epsilon\cdot\nu^{2}$ (cf., Algorithm 6) and using the assumption $\|\widehat{X}-\widehat{\chi}\|_{2}^{2}\leq 100k_{0}\nu^{2}$ in the lemma, we have

[TABLE]

and hence

[TABLE]

Next, we write

[TABLE]

and we proceed by upper bounding the four terms.

Bounding the first term in (97):

By part b of Lemma 5.1, the choice $\delta=\epsilon$, and the use of Markov, we have

[TABLE]

with probability at least $1-p$.

Bounding the second term in (97):

Conditioned on $\mathcal{E}_{\mathrm{loc}}$, we have

[TABLE]

where we have applied (91).

Bounding the third term in (97):

We have

[TABLE]

by (96). We have by definition that $|S_{0}|=10k_{0}$ (cf., (88)), and hence

[TABLE]

Bounding the fourth term in (97):

We have

[TABLE]

which follows from the definition of $S_{0}$ in (88), along with Definition 1.2.

Hence by the union bound, it follows that $\mathcal{E}_{\mathrm{prune}}$ holds with probability at least $1-p-\frac{1}{1000}$ conditioned on $\mathcal{E}_{\mathrm{loc}}$.

Success event of estimation primitive:

Let $\mathcal{E}_{\mathrm{est}}$ be the event of having a successful run of EstimateValues($X,\widehat{\chi},L,n,3k_{0}/\epsilon,k_{1},\epsilon,p$), meaning the following conditions on the output, $W$:

[TABLE]

where $\mathcal{F}$ contains the frequencies within the blocks indexed by $L^{\prime}$. Since the assumption of the theorem implies that $\|\widehat{\chi}\|_{0}=O(k_{0}k_{1})$, by Lemma 5.2 (with $\delta=\epsilon$ and $3k_{0}/\epsilon$ in place of $k_{0}$) and the fact that conditioned on $\mathcal{E}_{\mathrm{prune}}$ we have $|L^{\prime}|\leq 3k_{0}/\epsilon$ (cf., (92)), it follows that $\mathcal{E}_{\mathrm{est}}$ occurs with probability at least $1-p$.

Combining the events:

We can now can wrap everything up.

Letting $\mathcal{E}$ denote the overall success event corresponding to the claim of the lemma, we have

[TABLE]

By the results that we have above, along with the union bound, it follows that

[TABLE]

for sufficiently small $\eta$ in Algorithm 6. Applying the union bound over $\bar{\mathcal{E}}$ and the $1/\mathrm{poly}(n)$ probability failure event arising from random perturbations of $\widehat{\chi}_{0}$, we obtain the required bound of $0.9$ on the success probability.

What remains is to first show that the statement of the lemma follows from $\mathcal{E}_{\mathrm{loc}}\cap\mathcal{E}_{\mathrm{prune}}\cap\mathcal{E}_{\mathrm{est}}$. To do this, we observe that, conditioned on these events,

[TABLE]

where the second line follows since $\widehat{\chi}^{\prime}=\widehat{\chi}+W$ and $W$ is non-zero only within the blocks indexed by $L^{\prime}$, the third line follows from (93) and (98), and the final line follows from the assumption $\|\widehat{X}-\widehat{\chi}\|_{2}^{2}\leq 100k_{0}\nu^{2}$ in the lemma.

Sample complexity:

We condition on $\mathcal{E}_{\mathrm{loc}}\cap\mathcal{E}_{\mathrm{prune}}\cap\mathcal{E}_{\mathrm{est}}$ and calculate the number of samples used.

By Lemma 3.5 with $\delta=\epsilon^{2}$, the sample complexity of MultiBlockLocate is $O^{*}(|L|\cdot\log n+\frac{k_{0}k_{1}}{\epsilon^{4}})$ which behaves $O^{*}(\frac{k_{0}}{\epsilon^{2}}\log(1+k_{0})\log n+\frac{k_{0}k_{1}}{\epsilon^{4}}\log\frac{1}{p})$ conditioned on $\mathcal{E}_{\mathrm{loc}}$ (see (91)). Moreover, by Lemma 5.1 with $\delta=\epsilon$, the sample complexity of PruneLocation is $O(\frac{k_{0}k_{1}}{\epsilon}\log\frac{1}{\epsilon p}\log\frac{1}{\epsilon})$. Moreover, by Lemma 5.2 with $\delta=\epsilon$, the sample complexity of EstimateValues is $O(\frac{k_{0}k_{1}}{\epsilon^{2}}\log\frac{1}{p}\log\frac{1}{\epsilon})$.

The claim follows by summing the three terms and noting from the choice of $p$ in Algorithm 6 that, up to $\log\log\frac{k_{0}}{\epsilon}$ factors, we can replace $p$ by $\epsilon$ in the above calculations.

Runtime:

As in sample complexity analysis, we condition on $\mathcal{E}_{\mathrm{loc}}\cap\mathcal{E}_{\mathrm{prune}}\cap\mathcal{E}_{\mathrm{est}}$.

First, by Lemma 3.5 with $\delta=\epsilon^{2}$, the runtime of MultiBlockLocate is O^{*}\big{(}|L|\cdot\log^{2}n+\frac{k_{0}k_{1}}{\epsilon^{4}}\log^{2}n+\frac{k_{0}k_{1}}{\epsilon^{2}}\log^{3}n\big{)}, which behaves as O^{*}\big{(}\frac{k_{0}}{\epsilon^{2}}\log\frac{k_{0}}{\epsilon}\cdot\log^{2}n+\frac{k_{0}k_{1}}{\epsilon^{4}}\log^{2}n+\frac{k_{0}k_{1}}{\epsilon^{2}}\log^{3}n\big{)} conditioned on $\mathcal{E}_{\mathrm{loc}}$ (see (90)). Second, by Lemma 5.1 with $\delta=\epsilon$, the runtime of PruneLocation is $O(\frac{k_{0}k_{1}}{\epsilon}\log\frac{1}{\epsilon p}\log\frac{1}{\epsilon}\log n+k_{1}\cdot|L|\log\frac{1}{\epsilon p})$, which behaves as $O(\frac{k_{0}k_{1}}{\epsilon}\log\frac{1}{\epsilon p}\log\frac{1}{\epsilon}\log n+k_{1}\cdot\frac{k_{0}}{\epsilon^{2}}\log\frac{k_{0}}{\epsilon}\log^{4}\frac{1}{\epsilon p})$ conditioned on $\mathcal{E}_{\mathrm{loc}}$ (see (91)). Finally, by Lemma 5.2 with $\delta=\epsilon$, the runtime of EstimateValues is $O(\frac{k_{0}k_{1}}{\epsilon}\log\frac{1}{p}\log\frac{1}{\epsilon}\log n+k_{1}\cdot|L^{\prime}|\log\frac{1}{p})$, which behaves as $O(\frac{k_{0}k_{1}}{\epsilon}\log\frac{1}{p}\log\frac{1}{\epsilon}\log n)$ conditioned on $\mathcal{E}_{\mathrm{prune}}$ (see (92) and recall that $|S_{0}|=10k_{0}$).

The claim follows by summing the above terms and replacing $p$ by $\epsilon$, with the $\log\log n$, $\log\log\mathrm{SNR}^{\prime}$ and $\log\frac{1}{\epsilon}$ terms absorbed into the $O^{*}(\cdot)$ notation.

Appendix E Discussion on Energy-Based Importance Sampling

Here we provide further discussing on the adaptive energy-based importance sampling scheme described in Sections 2–3. Recall from Definition 2.2 that given the signal $X$ and filter $G$, we are considering downsampled signals of the form $\widehat{Z}^{r}_{j}=(\widehat{X}^{r}\star\widehat{G})_{jk_{1}}$ with $X^{r}_{i}=X_{i+\frac{nr}{2k_{1}}}$ for $r\in[2k_{1}]$, and recall from (3) that the goal of energy-based importance sampling is to approximately solve the covering problem

[TABLE]

for suitable $\alpha\in(0,1)$, where $\widehat{X}^{*}$ is the best $(k_{0}.k_{1})$-block sparse approximation of $\widehat{X}$.

To ease the discussion, we assume throughout this appendix that the filter $G$ is a width-$\frac{n}{k_{1}}$ rectangle in time domain, corresponding to a sinc pulse of “width” $k_{1}$ in frequency domain. Such a filter is less tight than the one we use (see the proof of Lemma 2.1), but similar enough for the purposes of the discussion.

E.1 Examples – Flat vs. Spiky Energies

We begin by providing two examples for the $1$-block sparse case, demonstrating how the energies can vary with $r$. An illustration of the energy in each $\widehat{Z}^{r}$ is illustrated in Figure 2 in two different cases – one in which $X$ is a sinc pulse (i.e., rectangular in frequency domain), and one in which $X$ is constant (i.e., a delta function in frequency domain). Both of the signals are $(1,k_{1})$-block sparse with $k_{1}=16$, and the signal energy is the same in both cases. However, the sinc pulse gives significantly greater variations in $|\widehat{Z}^{r}_{j}|^{2}$ as a function of $r$. In fact, these examples demonstrate two extremes that can occur – in one case, the energy exhibits no variations, and in the other case, the energy is $O(k_{1})$ times its expected value for an O\big{(}\frac{1}{k_{1}}\big{)} fraction of the $r$ values.

The second example above is, of course, an extreme case of a $(1,k_{1})$-block sparse signal, because it is also $(1,1)$-block sparse. Nevertheless, one also observes a similar flatness in time domain for other signals; e.g., one could take the first example above and randomize the signs, as opposed to letting them all be positive.

E.2 The $\log(1+k_{0})$ factor

Here we provide an example demonstrating that, as long as we rely solely on frequencies being covered according to Definition 2.3, after performing the budget allocation, the extra $\log(1+k_{0})$ factor in our analysis is unavoidable. Specifically, we argue that for a certain signal $X$, the optimal solution to (3) satisfies $\sum_{r\in[2k_{1}]}s^{r}=\Omega(k_{0}\log(1+k_{0}))$. However, we do not claim that this $\log(1+k_{0})$ factor is unavoidable for arbitrary sparse FFT algorithms.

We consider a scenario where $k_{0}=\Theta(k_{1})=o(n)$, and for concreteness, we let both $k_{0}$ and $k_{1}$ behave as $\Theta(n^{0.1})$; hence, $\log(1+k_{0})=O(\log k_{0})$

Constructing a base signal: We first specify a base signal $W\in\mathbb{C}^{n}$ that will be used to construct the approximately $(k_{0},k_{1})$-block sparse signal. Specifically, we fix the integers $C$ and $L$, and set

[TABLE]

Hence, the signal contains $L$ regions of exponentially increasing width but exponentially decreasing magnitude. See Figure 3 for an illustration ($L=3$), and observe that we can express this function as a sum of rectangles having geometrically decreasing magnitudes. Hence, we can specify its Fourier transform as a sum of sinc functions.

The narrowest of the rectangles has width $\frac{Cn}{k_{1}}$, and hence the widest of the sinc pulses has width $\frac{k_{1}}{C}$. This means that by choosing $C$ to be sufficiently large, we can ensure that an arbitrarily high proportion of the energy lies in a window of length $k_{1}$ in frequency domain, meaning $W$ is approximately $1$-block sparse.

Constructing a block-spare signal: We construct a $k_{0}$-block sparse signal by adding multiple copies of $W$ together, each shifted by a different amount in time domain, and also modulated by a different frequency (i.e., shifted by a different amount in frequency domain). We choose $L$ such that the cases in (100) collectively occupy the whole time domain, yielding $L=\Theta(\log k_{1})=\Theta(\log k_{0})$.

Then, we set $k_{0}=\frac{k_{1}}{C}$ and let each copy of $W$ be shifted by a multiple of $\frac{Cn}{k_{1}}$, so that the copies are separated by a distance equal to the length of the thinnest segment of $W$, and collectively these thin segments cover the whole space $[n]$. As for the modulation, we choose these so that the resulting peaks in frequency domain are separated by $\Omega(k_{1}^{4})$, so that there the tail of the copy of $W$ corresponding to one block has a negligible effect on the other blocks. This is possible within $n$ coefficients, since we have chosen $k_{1}=O(n^{0.1})$.

Evaluating the values of $|Z_{j}^{r}|^{2}$ Recall that we are considering $G$ in (99) equaling a rectangle of width $\frac{n}{k_{1}}$. Because of the above-mentioned separation of the blocks in frequency domain, each copy of $W$ can essentially be treated separately. By construction, within a window of length $\frac{n}{k_{1}}$, we have one copy of $W$ at magnitude $\sqrt{2^{L-1}}$, two copies at magnitude $\sqrt{2^{L-2}}$, and so on. Upon subsampling by a factor of $k_{1}$, the relative magnitudes remain the same; there is no aliasing, since we let $G$ be rectangular. Hence, the dominant coefficients in the spectrum of the subsampled signal exhibit this same structure, having energies of a form such as $(8,4,4,2,2,2,2,1,1,1,1,1,1,1,1)$ when sorted and scaled (up to negligible leakage effects). Moreover, the matrix of $|\widehat{Z}_{j}^{r}|^{2}$ values (cf., Figure 1) essentially amounts to circular shifts of a vector of this form – the structure of any given $\widehat{Z}^{r}$ maintains this geometric structure, but possibly in a different order.

Lower bounding the sum of budgets allocated: We now turn to the allocation problem in (99). Allocating a sparsity budget $s$ to a signal $Z^{r}$ covers all coefficients $j$ for which $|\widehat{Z}^{r}_{j}|^{2}\geq\frac{\|\widehat{Z}\|^{2}}{s}$. For the signal we have constructed, the total energy $E$ is equally spread among the $L$ geometric levels: The $\ell$-th level consists of $2^{\ell-1}$ coefficients having energy $2^{1-\ell}\frac{E}{L}$, and hence covering that level requires that $s\geq L\cdot 2^{\ell}$.

Hence, setting $s=L\cdot 2^{\ell-1}$ covers the top $\ell$ levels, for a total of $2^{\ell}-1$ coefficients. That is, covering some number of coefficients requires letting $s$ be $\Omega(L)$ times that number, and hence covering a constant fraction of the $k_{0}$ coefficients requires the sum of sparsity budgets to be $\Omega(Lk_{0})$. Moreover, we have designed every block to have the same energy, so accounting for a constant fraction of the energy amounts to covering a constant fraction of the $k_{0}$ coefficients.

Since we selected $L=\Theta(\log k_{0})$, this means that the sum of sparsity budgets is $\Omega(k_{0}\log k_{0})$, so that the $\log k_{0}$ factor must be present in any solution to (99).

Appendix F Location of Reduced Signals

In Algorithm 7, we provide a location primitive that, given a sequence of budgets $s^{r}$, locates dominant frequencies in the sequence of reduced signals $Z^{r}$ using O\big{(}\sum_{r\in[2k_{1}]}s^{r}\log n\big{)} samples. The core of the primitive is a simple $k$-sparse recovery scheme, where $k$ frequencies are hashed into $B=Ck$ buckets for a large constant $C>1$, and then each bucket is decoded individually. Specifically, for each bucket that is approximately $1$-sparse (i.e., dominated by a single frequency that hashed into it) the algorithm accesses the signal at about a logarithmic number of locations and decodes the bit representation of the dominant frequency bit by bit. More precisely, to achieve the right sample complexity we decode the frequencies in blocks of $O(\log\log n)$ bits. Such schemes or versions thereof have been used in the literature (e.g., [GMS05, HIKP12c, Kap16]).

A novel aspect of our decoding scheme is that it receives access to the input signal $X$, but must run a basic sparse recovery scheme as above on each reduced signal $Z^{r}$. Specifically, for each $r$ it must hash $Z^{r}$ into $s^{r}$ buckets (the budget computed in MultiblockLocate and passed to LocateSignal as input). This would be trivial since $Z^{r}$ can be easily accessed given access to $X$ (cf., Lemma 2.2), but the fact that we need to operate on the residual signal $X-\chi$ (where $\widehat{\chi}$ is block sparse and given explicitly as input) introduces difficulties.

The difficulty is that we would like to compute $\chi$ on the samples that individual invocations of sparse recovery use, for each $r\in[2k_{1}]$, but computing this directly would be very costly. Our solution consists of ensuring that all invocations of sparse recovery use the same random permutation $\pi$, and therefore all need to access $X-\chi$ on a set of shifted intervals after a change of variables given by $\pi$ (crucially, this change of variables is shared across all $r$). The lengths of the intervals are different, and given by $s^{r}$, but it suffices to compute the values of $\chi$ on the shifts of the largest of these intervals, which is done in HashToBinsReduced (see Lemma 4.6). We present the details below in Algorithm 7.

For convenience, throughout this section, we use $m$ to denote the reduced signal length $n/k_{1}$.

Lemma 2.4 (LocateReducedSignal guarantees – formal version) Fix $(n,k_{0},k_{1})$, the signals $X,\widehat{\chi}\in\mathbb{C}^{n}$ with $\widehat{\chi}_{0}$ uniformly distributed over an arbitrarily length-$\frac{\|\widehat{\chi}\|^{2}}{\mathrm{poly}(n)}$ interval, the sparsity budgets $\{s^{r}\}_{r\in[2k_{1}]}$ with s^{r}=O\big{(}\frac{k_{0}}{\delta}\big{)} for all $r\in[2k_{1}]$, and the parameters \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)} and p\in\big{(}\frac{1}{n^{3}},\frac{1}{2}\big{)}, and let $\{Z^{r}\}_{r\in[2k_{1}]}$ be the $(k_{1},\delta)$-downsampling of $X-\chi$.

*Letting $L$ denote the output of $\textsc{LocateReducedSignals}(X,\widehat{\chi},n,k_{0},k_{1},\{s^{r}\}_{r\in[2k_{1}]},\delta,p)$, we have that for any j\in\big{[}\frac{n}{k_{1}}\big{]} such that $|Z_{j}^{r}|^{2}\geq\|Z^{r}\|_{2}^{2}/s^{r}$ for some $r\in[2k_{1}]$, one has $j\in L$ with probability at least $1-p$. The list size satisfies |L|=O\big{(}\sum_{r\in[2k_{1}]}s^{r}\log\frac{1}{\delta}\big{)}. Moreover, if $\widehat{\chi}$ is $(O(k_{0}),k_{1})$-block sparse, the sample complexity is O\big{(}\sum_{r\in[2k_{1}]}s^{r}\log\frac{1}{p}\log\frac{1}{\delta}\log n\big{)}, and the runtime is O\big{(}\sum_{r\in[2k_{1}]}s^{r}\log\frac{1}{p}\log\frac{1}{\delta}\log^{2}n+\frac{k_{0}k_{1}}{\delta}\log\frac{1}{p}\log^{3}n\big{)}. *

Proof.

We first note that the claim on the list size follows immediately from the fact that $B=O(s^{r})$ entries are added to the list for each $t$ and $r$, and the loop over $t$ is of length O\big{(}\log\frac{1}{p}\big{)}.

In order to prove the main claim of the lemma, it suffices to show that for any single value of $r$, if we replace the loop over $r$ by that single value, then $L$ contains any given j\in\big{[}\frac{n}{k_{1}}\big{]} such that $|Z_{j}^{r}|^{2}\geq\|Z^{r}\|_{2}^{2}/s^{r}$, with probability at least $1-p$. Since this essentially corresponds to a standard sparse recovery problem, we switch to simpler notation throughout the proof: We let $Y$ denote a generic signal $Z^{r}$, we write its length as $m=n/k_{1}$, we index its entries in frequency domain as $\widehat{Y}_{f}$, and we define $k=s^{r}$.

The proof now consists of two steps. First, we show correctness of the location algorithm assuming that the SemiEquiInverseBlockFFT computation in line 11 computes all the required values for the computation of $\widehat{U}$ in line 19. We then prove that SemiEquiInverseBlockFFT indeed computes all the required values of $\chi$, and conclude with sample complexity and runtime bounds.

Proving correctness of the location process We show that each element $f$ with $|\widehat{Y}_{f}|^{2}\geq\|\widehat{Y}\|_{2}^{2}/k$ is reported in a given iteration of the outer loop over $t=1,\ldots,C_{1}\log(2/p)$, with probability at least $9/10$. Since the loops use independent randomness, the probability of $f$ not being reported in any of the iterations is bounded by $(1/10)^{C_{1}\log(2/p)}\leq p/2$ if $C_{1}$ is sufficiently large.

Fix an iteration $t$. We first show that the random set ${\mathcal{A}}$ chosen in LocateReducedSignals has useful error-correcting properties with high probability. Specifically, we let $\mathcal{E}_{\mathrm{balanced}}$ denote the event that for every $\lambda\in[\Lambda],\lambda\neq 0$ at least a fraction $49/100$ of the numbers $\{\omega_{\Lambda}^{\lambda\cdot\beta}\}_{(\alpha,\beta)\in{\mathcal{A}}}$ have non-positive real part; in that case, we say that ${\mathcal{A}}$ is balanced. We have for fixed $\lambda\in[\Lambda],\lambda\neq 0$ that since the pair $(\alpha,\beta)$ was chosen uniformly at random from $[m]\times[m]$, the quantity $\omega_{\Lambda}^{\lambda\cdot\beta}$ is uniformly distributed on the set of roots of unity of order $2^{s}$ for some $s>0$ (since $\lambda\neq 0$). At least half of these roots have non-positive real part, so for every fixed $\lambda\in[\Lambda],\lambda\neq 0$ one has $\mathbb{P}_{\beta}[\text{Re}(\omega_{\Lambda}^{\lambda\cdot\beta})\leq 0]\geq 1/2$. It thus follows by standard concentration inequalities that for every fixed $\lambda$ at least $49/100$ of the numbers $\{\omega_{\Lambda}^{\lambda\cdot\beta}\}_{(\alpha,\beta)\in{\mathcal{A}}}$ have non-positive real part with probability at least $1-e^{-\Omega(|{\mathcal{A}}|)}=1-\exp(-\Omega(C_{3}\log\log m)))\geq 1-1/(100\log_{2}m)$ as long as $C_{3}$ is larger than an absolute constant. A union bound over $\Lambda\leq\log_{2}m$ values of $\lambda$ shows that $\mathbb{P}[\mathcal{E}_{\mathrm{balanced}}]\geq 1-(\log_{2}m)\cdot/(100\log_{2}m)=1-1/100$ for sufficiently large $m$ (recall from Section 1 that $\frac{n}{k_{1}}$ exceeds a large absolute constant by assumption). We henceforth condition on $\mathcal{E}_{\mathrm{balanced}}$.

Fix any $f$ such that $|\widehat{Y}_{f}|^{2}\geq||\widehat{Y}||_{2}^{2}/k$, and let $q=\sigma i$ for convenience. We show by induction on $g=1,\ldots,\log_{\Lambda}N$ that before the $g$-th iteration of lines 24–27 of Algorithm 7, we have that ${\bf f}$ coincides with ${\bf q}$ on the bottom $g\cdot\log_{2}\Lambda$ bits, i.e., ${\bf f}-{\bf q}=0\mod\Lambda^{g-1}$.

The base of the induction is trivial and is provided by $g=1$. We now show the inductive step. Assume by the inductive hypothesis that ${\bf f}-{\bf q}=0\mod\Lambda^{g-1}$, so that ${\bf q}={\bf f}+\Lambda^{g-1}(\lambda_{0}+\Lambda\lambda_{1}+\Lambda^{2}\lambda_{2}+\ldots)$. Thus, $(\lambda_{0},\lambda_{1},\ldots)$ is the expansion of $({\bf q}-{\bf f})/\Lambda^{g-1}$ base $\Lambda$, and $\lambda_{0}$ is the least significant digit. We now show that $\lambda_{0}$ is the unique value of $\lambda$ that satisfies the condition of line 25 of Algorithm 7, with high probability

In the following, we use the definitions of $\pi(f)$, $h(f)$, and $o_{f}(f^{\prime})$ from Definition 4.2 with $\Delta=0$. First, we have for each $a=(\alpha,\beta)\in{\mathcal{A}}$ and ${\bf w}\in{\bf W}$ that

[TABLE]

where $\mathrm{err}_{w}=\widehat{H}_{o_{f}(f)}^{-1}(\widehat{U}_{h(f)}-\widehat{U}^{*}_{h(f)})(\alpha+{\bf w}\cdot\beta)$.

And similarly

[TABLE]

where $\mathrm{err}=\widehat{H}_{o_{f}(f)}^{-1}(\widehat{U}_{h(f)}-\widehat{U}^{*}_{h(f)})(\alpha)$.

We will show that $f$ is recovered from bucket $h(f)$ with high (constant) probability. The bounds above imply that

[TABLE]

The rest of the proof consists of two parts. We first show that with high probability over the choice of $\pi$, the error terms $E^{\prime}({\bf w})$ and $E^{\prime\prime}$ are small in absolute value for most $a=(\alpha,\beta)\in{\mathcal{A}}$ with extremely high probability. We then use this assumption to argue that $f$ is recovered.

Bounding the error terms $E^{\prime}({\bf w})$ and $E^{\prime\prime}$ (part (i)).

We have by Parseval’s theorem that

[TABLE]

and

[TABLE]

where we used the fact that $\alpha+{\bf w}\beta$ is uniformly random in $[m]$ (due to $\alpha$ being uniformly random in $[m]$ and independent of $\beta$ by definition of ${\mathcal{A}}$ in line 6 of Algorithm 7).

Taking the expectation of the term $\widehat{H}_{o_{f}(f)}^{-2}\sum_{f^{\prime}\in[m]\setminus\{f\}}\widehat{H}_{o_{f}(f^{\prime})}^{2}|Y_{f^{\prime}}|^{2}$ with respect to $\pi$, we obtain

[TABLE]

by Lemma 4.3 (note that $F^{\prime}\geq 2$, so the lemma applies) and the choice $B=C_{2}\cdot k$ (line 20 of Algorithm 7). We thus have by Markov’s inequality together with the assumption that $|\widehat{Y}_{f}|^{2}\geq||\widehat{Y}||_{2}^{2}/k$ that

[TABLE]

and

[TABLE]

since $C_{2}$ is larger than an absolute constant by assumption.

Bounding $\mathrm{err}$ and $\mathrm{err}_{w}$ (numerical errors from semi-equispaced FFT computation): Recall that we have by assumption that $\widehat{\chi}_{0}$ uniformly distributed over an arbitrarily length-$\frac{\|\widehat{\chi}\|^{2}}{\mathrm{poly}(n)}$ interval, and that $\widehat{Y}=\widehat{Z}^{r}$ for some $\widehat{Z}^{r}$ in the $(k_{1},\delta)$-downsampling of $X-\chi$. By decomposing $\widehat{Z}^{r}_{j}=((\widehat{X}^{r}-\widehat{\chi}^{r})\star\widehat{G})_{jk_{1}}$ into a deterministic part and a random part (in terms of the above-mentioned uniform distribution), we readily obtain for some $c^{\prime}>0$ that

[TABLE]

with probability at least $1-\frac{1}{n^{4}}$. Since $p\geq\frac{1}{n^{3}}$ by assumption, we deduce that this also holds with probability at least $1-p/2$. By the the accuracy of the $\widehat{\chi}$ values stated in Lemma 4.5, along with the argument used in (70)–(71) its proof in Appendix C.3 to convert (103) to accuracy on hashed values, we know that $|\widehat{U}_{h(f)}-\widehat{U}^{*}_{h(f)}|\leq\|\widehat{U}-\widehat{U}^{*}\|_{\infty}\leq n^{-c+1}\|\widehat{\chi}\|_{2}$. Hence, by using $|\widehat{H}_{o_{f}(f)}|^{-2}\leq 2$ and $\alpha,\alpha+{\bf w}\cdot\beta\leq m$, we find that

[TABLE]

where the second inequality in each equation holds for some $c^{\prime}>0$ by (103).

Note also that $\widehat{H}_{o_{f}(f)}^{-1}\sum_{f^{\prime}\in[m]\setminus\{f\}}\widehat{H}_{o_{f}(f^{\prime})}|\widehat{Y}_{f^{\prime}}|\leq 2\|\widehat{Y}\|_{1}\leq 2\sqrt{m}\|\widehat{Y}\|_{2}$, since we have $\widehat{H}_{o_{f}(f)}^{-1}\leq 2$ and $\widehat{H}_{f^{\prime}}|\leq 1$ for all $f^{\prime}$. We can thus write

[TABLE]

which can thus be made to behave as $\frac{1}{\mathrm{poly}(n)}\|\widehat{Y}\|_{2}^{2}$ by a suitable choice of $c$.

Bounding the error terms $E^{\prime}({\bf w})$ and $E^{\prime\prime}$ (part (ii)).

By the union bound, we have $|E^{\prime}({\bf w})|^{2}\leq|\widehat{Y}_{f}|^{2}/1600$ and $|E^{\prime\prime}|^{2}\leq|\widehat{Y}_{f}|^{2}/1600$ simultaneously with probability at least $1-1/20$ – denote the success event by ${\mathcal{E}}^{t}_{f,\pi}({\bf w})$. Conditioned on ${\mathcal{E}}^{t}_{f,\pi}({\bf w})$, we thus have by (102) and (104), along with the fact that ${\mathcal{A}}$ is independent of $\pi$, that

[TABLE]

Another application of Markov’s inequality gives

[TABLE]

This means that conditioned on ${\mathcal{E}}^{t}_{f,\pi}({\bf w})$, with probability at least $1-e^{-\Omega(|{\mathcal{A}}|)}\geq 1-1/(100\log_{2}m)$ over the choice of ${\mathcal{A}}$, both events occur for all but $2/5$ fraction of $a\in{\mathcal{A}}$; denote this success event by ${\mathcal{E}}^{t}_{f,{\mathcal{A}}}({\bf w})$. We condition on this event in what follows. Let ${\mathcal{A}}^{*}({\bf w})\subseteq{\mathcal{A}}$ denote the set of values of $a\in{\mathcal{A}}$ that satisfy the bounds above.

In particular, we can rewrite  (101) as

[TABLE]

We thus have for $a\in{\mathcal{A}}^{*}({\bf w})$ that

[TABLE]

Showing that ${\mathcal{A}}^{*}({\bf w})\subseteq{\mathcal{A}}$ suffices for recovery.

By the above calculations, we get

[TABLE]

We proceed by analyzing the first term, and we will later show that the second term is small. Since ${\bf q}={\bf f}+\Lambda^{g-1}(\lambda_{0}+\Lambda\lambda_{1}+\Lambda^{2}\lambda_{2}+\ldots)$, by the inductive hypothesis, we have

[TABLE]

where we used the fact that $\omega_{N}^{N/\Lambda}=e^{2\pi f(N/\Lambda)/N}=e^{2\pi f/\Lambda}=\omega_{\Lambda}$. Thus, we have

[TABLE]

We now consider two cases. First suppose that $\lambda=\lambda_{0}$. Then $\omega_{\Lambda}^{(-\lambda+\lambda_{0})\cdot\beta}=1$, and it remains to note that by (105) we have $|\xi-1|\leq\frac{1+1/40}{1-1/40}-1<1/3$. Thus, every $a\in{\mathcal{A}}^{*}({\bf w})$ passes the test in line 25 of Algorithm 7. Since $|{\mathcal{A}}^{*}({\bf w})|\geq(3/5)|{\mathcal{A}}|$ by the argument above, we have that $\lambda_{0}$ passes the test in line 25. It remains to show that $\lambda_{0}$ is the unique element in $0,\ldots,\Lambda-1$ that passes this test.

Suppose that $\lambda\neq\lambda_{0}$. Then, by conditioning on $\mathcal{E}_{\mathrm{balanced}}$, at least a $49/100$ fraction of $\omega_{\Lambda}^{(-\lambda+\lambda_{0})\cdot\beta}$ have negative real part. This means that for at least $49/100$ of $a\in{\mathcal{A}}$, we have

[TABLE]

and hence the condition in line 16 of Algorithm 7 is not satisfied for any $\lambda\neq\lambda_{0}$.

We thus get that conditioned on $\mathcal{E}_{\mathrm{balanced}}$ and the intersection of ${\mathcal{E}}^{t}_{f,\pi}({\bf w})$ for all ${\bf w}\in{\bf W}$ and ${\mathcal{E}}^{t}_{f,{\mathcal{A}}}$, recovery succeeds for all values of $g=1,\ldots,\log_{\Lambda}N$. By a union bound over the failure events, we get that

[TABLE]

This shows that location is successful for $f$ in a single iteration $t$ with probability at least $98/100\geq 9/10$, as required.

Sample complexity and runtime. We first consider the calls to HashToBinsReduced. This is called for $\log\log m$ values of $(\alpha,\beta)$ and \log_{\Lambda}N=O\big{(}\frac{\log m}{\log\log m}\big{)} values of $g$ in each iteration, the product of which is $O(\log m)=O(\log n)$. Moreover, the number of iterations is O\big{(}\log\frac{1}{p}\big{)}. Hence, using Lemma 4.6, we find that the combination of all of these calls costs O\big{(}F\sum_{r\in[2k_{1}]}B^{r}\log\frac{1}{\delta}\log n\big{)} samples, with a runtime of O\big{(}(B_{\mathrm{max}}F+k_{0})k_{1}\log^{3}n), where $B_{\mathrm{max}}=O(\max_{r}s^{r})$, and $k_{0}$ is such that $\widehat{\chi}$ is $(O(k_{0}),k_{1})$-block sparse. By the assumption \max_{r}s^{r}=O\big{(}\frac{k_{0}}{\delta}\big{)}, the runtime simplifies to O\big{(}\frac{k_{0}k_{1}}{\delta}\log^{3}n)

∎

Appendix G Pruning the Location List

The pruning procedure is given in Algorithm 8. Its goal is essentially to reduce the size of the list returned by MultiBlockLocate (cf., Algorithm 1) from $O(k_{0}\log(1+k_{0}))$ to $O(k_{0})$. More formally, the following lemma shows that with high probability, the pruning algorithm retains most of the energy in the head elements, while removing most tail elements.

Lemma 5.1 (PruneLocation guarantees – re-stated from Section 5.1) Given $(n,k_{0},k_{1})$, the parameters $\theta>0$, \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)} and $p\in(0,1)$, and the signals $X\in\mathbb{C}^{n}$ and $\widehat{\chi}\in\mathbb{C}^{n}$ with $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$, the output $L^{\prime}$ of PruneLocation$(X,\widehat{\chi},L,k_{0},k_{1},\delta,p,n,\theta)$ has the following properties:

1. a.

Let $S_{\mathrm{tail}}$ denote the tail elements in the signal $\widehat{X}-\widehat{\chi}$, defined as

[TABLE]

where $I_{j}$ is defined in Definition 1.1. Then, we have

[TABLE] 2. b.

Let $S_{\mathrm{head}}$ denote the head elements in the signal $\widehat{X}-\widehat{\chi}$, defined as

[TABLE]

Then, we have

[TABLE]

*Moreover, provided that $\|\widehat{\chi}\|_{0}=O(k_{0}k_{1})$, the sample complexity is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta p}\log\frac{1}{\delta})$, and the runtime is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta p}\log\frac{1}{\delta}\log n+k_{1}\cdot|L|\log\frac{1}{\delta p})$. *

Proof.

We begin by analyzing the properties of the random variables $W_{j}$ used in the threshold test. We define $X^{\prime}=X-\chi$, let $\widehat{U}$ be the output of HashToBins, and let $\widehat{U}^{*}$ be its exact counterpart as defined in Lemma 4.6. It follows that we can write the random variable $W_{j}^{(t)}$ (cf., Algorithm 8) as

[TABLE]

where (i) $\mathrm{err}^{(t)}_{f}=\widehat{G}^{-1}_{o_{f}(f)}\sum_{f^{\prime}\in[n]\backslash\{f\}}\widehat{X}^{\prime}_{f^{\prime}}\widehat{G}_{o_{f}(f^{\prime})}\omega_{n}^{\sigma\Delta(f^{\prime}-f)}$, with $(\sigma,\Delta)$ implicitly depending on $t$; this follows directly from Lemma 4.2, along with the definitions $\pi(f)=\sigma f$ and $o_{f}(f^{\prime})=\pi(f^{\prime})-\frac{n}{B}h(f)$. (ii) $\widetilde{\mathrm{err}}^{(t)}_{f}=\widehat{G}^{-1}_{o_{f}(f)}(\widehat{U}_{h(f)}-\widehat{U}^{*}_{h(f)})\omega^{-\sigma\Delta f}$, a polynomially small error term (cf., Lemma 4.6).

Bounding $\mathrm{err}^{(t)}_{f}$ and $\widetilde{\mathrm{err}}^{(t)}_{f}$: In Lemma G.1 below, we show that

[TABLE]

where $c$ is used in HashToBins, and $c^{\prime}$ is value such that $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{n^{c^{\prime}}}\|\widehat{\chi}\|_{2}$. For (108), we upper bound the $\ell_{2}$ norm by the square root of the vector length times the $\ell_{\infty}$ norm, yielding

[TABLE]

We now calculate the probability of a given block $j$ passing the threshold test, considering two separate cases.

• •

If $j$ is in the tail: The probability for $j$ to pass the threshold is closely related to \mathbb{P}[W_{j}^{(t)}\geq\theta]=\mathbb{P}\big{[}\sqrt{W_{j}^{(t)}}\geq\sqrt{\theta}\big{]}. From (106), $\sqrt{W_{j}^{(t)}}$ is the $\ell_{2}$-norm of a sum of three signals, and hence we can apply the triangle inequality to obtain

[TABLE]

where we have applied (109).

By definition, for any $j\in S_{\mathrm{tail}}$, we have $\sqrt{\theta}-\|\widehat{X}^{\prime}_{I_{j}}\|_{2}\geq\sqrt{\frac{\delta}{k_{0}}}\|\widehat{X}^{\prime}\|_{2}$. Hence, and recalling that $\delta\geq\frac{1}{n}$, if $c$ if sufficiently large so that $\sqrt{\frac{\delta}{k_{0}}}-2n^{-c+c^{\prime}+1/2}\geq\sqrt{\frac{0.9\delta}{k_{0}}}$, then Markov’s inequality yields

[TABLE]

where the third line follows form (107) and $|I_{j}|=k_{1}$, and the final line follows from the choice $B=160\frac{k_{0}k_{1}}{\delta}$. Since $W_{j}$ is the median of $T$ independent such random variables, it can only exceed $\theta$ if there exists a subset of $t$ values of size $\frac{T}{2}$ with $W_{j}^{(t)}\geq\theta$. Hence,

[TABLE]

where we applied ${{T}\choose{T/2}}\leq 2^{T}$, followed by $T=10\log\frac{1}{\delta p}$ (cf., Algorithm 8).

• •

If j is in the head: We proceed similarly to the tail case, but instead use the triangle inequality in the form of a lower bound (i.e., $\|a+b\|_{2}\geq\|a\|_{2}-\|b\|_{2}$), yielding

[TABLE]

By definition, for any $j\in S_{\mathrm{head}}$, we have $\|\widehat{X}^{\prime}_{I_{j}}\|_{2}-\sqrt{\theta}\geq\sqrt{\frac{\delta}{k_{0}}}\|\widehat{X}^{\prime}\|_{2}$. Hence, if $c$ if sufficiently large so that $\sqrt{\frac{\delta}{k_{0}}}-2n^{-c+c^{\prime}+1/2}\geq\sqrt{\frac{0.9\delta}{k_{0}}}$, then analogously to the tail case above, we have

[TABLE]

and consequently $\mathbb{P}[W_{j}\leq\theta]\leq\delta p$.

First claim of lemma: Since $L^{\prime}\subset L$, we have

[TABLE]

Since we established that $\mathbb{P}[W_{j}\geq\theta]$ is at most $\delta p$, we obtain

[TABLE]

Second claim of lemma: In order to upper bound $\sum_{j\in(L\cap S_{\mathrm{head}})\backslash L^{\prime}}\|\widehat{X}^{\prime}_{I_{j}}\|_{2}^{2}$, we first calculate its expected value as follows:

[TABLE]

The probability \mathbb{P}\big{[}W_{j}\leq\theta\big{]} for $j\in L\cap S_{\mathrm{head}}$ is at most $\delta p$, and hence

[TABLE]

Sample complexity and runtime

For the sample complexity, note that the algorithm only uses samples via its call to HashToBins. By part (i) of Lemma 4.6 and the choices $B=160\frac{k_{0}k_{1}}{\delta}$ and $F=10\log\frac{1}{\delta}$, the sample complexity is $O(FB)=O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta})$ per hashing operation. Since we run the hashing in a loop $10\log\frac{1}{\delta p}$ times, the sample complexity is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta}\log\frac{1}{\delta p})$.

The runtime depends on three operations. The first is calling HashToBins, for which an analogous argument as that for the sample complexity holds, with the extra $\log n$ factor arising from Lemma 4.6. The second operation is the computation of $W_{j}^{(t)}$, which takes $|I_{j}|=O(k_{1})$ time for each $j\in L$. Hence, the total contribution from the loop is $O(k_{1}\cdot|L|\log\frac{1}{\delta p})$. Finally, since the median can be computed in linear time, computing the medians for every $j\in L$ costs $O(|L|\log\frac{1}{\delta p})$ time, which is dominated by the computation of $W_{j}^{(t)}$.

∎

In the preceding proof, we made use of the following.

Lemma G.1**.**

(EstimateValues guarantees – re-stated from Section 5.1)* Fix $(n,k_{0},k_{1},B)$, the signals $X\in\mathbb{C}^{n}$ and $\widehat{\chi}\in\mathbb{C}^{n}$ with $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{n^{c^{\prime}}}\|\widehat{\chi}\|_{2}$, and the uniformly random parameters $\sigma,\Delta\in[n]$ with $\sigma$ odd, and let $\widehat{U}$ be the output of $\textsc{HashToBins}(X,\widehat{\chi},G,n,B,\sigma,\Delta)$ and $\widehat{U}^{*}$ its exact counterpart. Then defining $\mathrm{err}_{f}=\widehat{G}^{-1}_{o_{f}(f)}\sum_{f^{\prime}\in[n]\backslash\{f\}}\widehat{X}^{\prime}_{f^{\prime}}\widehat{G}_{o_{f}(f^{\prime})}\omega_{n}^{\sigma\Delta(f^{\prime}-f)}$ and $\widetilde{\mathrm{err}}_{f}=\widehat{G}^{-1}_{o_{f}(f)}(\widehat{U}_{h(f)}-\widehat{U}^{*}_{h(f)})\omega^{-\sigma\Delta f}$ (for $h$ and $\sigma_{f}$ in Definition 4.2), we have*

[TABLE]

for $c$ used in HashToBins.

Proof.

We take the expectation of $|\mathrm{err}_{f}|^{2}$, first over $\Delta$:

[TABLE]

by Parseval. By Definition 2.1 and the definition of $o_{f}(\cdot)$, we can upper bound $|\widehat{G}_{o_{f}(f)}|^{-2}\leq 2$. Continuing, we take the expectation with respect to the random permutation $\pi$:

[TABLE]

by Lemma 4.3.

We now turn to $\widetilde{\mathrm{err}}_{f}$. We know from Lemma 4.6 that $|\widehat{U}_{h(f)}-\widehat{U}^{*}_{h(f)}|\leq\|\widehat{U}-\widehat{U}^{*}\|_{\infty}\leq n^{-c}\|\widehat{\chi}\|_{2}$. Hence, and again using $|\widehat{G}_{o_{f}(f)}|^{-2}\leq 2$, we find that

[TABLE]

where the second inequality follows since $\|\widehat{\chi}\|_{2}\leq n^{c^{\prime}}\|\widehat{X}^{\prime}\|_{2}$ for some $c^{\prime}>0$ by assumption. ∎

Appendix H Estimating Individual Frequency Values

Once we have located the blocks, we need to estimate the frequency values with them. The function EstimateValues in Algorithm 9 performs this task for us via basic hashing techniques. The following lemma characterizes the guarantee on the output.

Lemma 5.2 (Re-stated from Section 5.1) For any integers $(n,k_{0},k_{1})$, list of block indices $L$, parameters \delta\in\big{(}\frac{1}{n},\frac{1}{20}\big{)} and $p\in(0,1/2)$, and signals $X\in\mathbb{C}^{n}$ and $\widehat{\chi}\in\mathbb{C}^{n}$ with $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$, the output $W$ of the function EstimateValues$(X,\widehat{\chi},L,n,k_{0},k_{1},\delta,p)$ has the following property:

[TABLE]

*with probability at least $1-p$, where $I_{j}$ is the $j$-th block. Moreover, the sample complexity is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{p}\log\frac{1}{\delta})$, and if $\|\widehat{\chi}\|_{0}=O(k_{0}k_{1})$, then the runtime is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{p}\log\frac{1}{\delta}\log n+k_{1}\cdot|L|\log\frac{1}{p})$. *

Proof.

Let $X^{\prime}=X-\chi$, and let $\widehat{U}$ be the output of HashToBins and $\widehat{U}^{*}$ its exact counterpart. We start by calculating $W_{f}^{(t)}$ for an arbitrary $f\in\mathcal{F}$:

[TABLE]

where $\mathrm{err}^{(t)}_{f}=\widehat{G}^{-1}_{o_{f}(f)}\sum_{f^{\prime}\in[n]\backslash\{f\}}\widehat{X}^{\prime}_{f^{\prime}}\widehat{G}_{o_{f}(f^{\prime})}\omega^{\sigma\Delta(f^{\prime}-f)}$, and $\widetilde{\mathrm{err}}^{(t)}_{f}=\widehat{G}^{-1}_{o_{f}(f)}(\widehat{U}_{h(f)}-\widehat{U}^{*}_{h(f)})\omega^{-\sigma\Delta f}$, for $(\sigma,\Delta)$ implicitly depending on $t$.

Bounding $\mathrm{err}^{(t)}_{f}$ and $\widetilde{\mathrm{err}}^{(t)}_{f}$: Using Lemma G.1 in Appendix G, we have

[TABLE]

where $c$ is used in HashToBins, and $c^{\prime}$ is the exponent in the $\mathrm{poly}(n)$ notation of the assumption $\|\widehat{X}-\widehat{\chi}\|_{2}\geq\frac{1}{\mathrm{poly}(n)}\|\widehat{\chi}\|_{2}$.

In order to characterize $|W_{f}^{(t)}-\widehat{X}^{\prime}_{f}|^{2}$, we use the following:

[TABLE]

which follows directly from (116).

Characterizing $|W_{f}^{(t)}-\widehat{X}^{\prime}_{f}|^{2}$: We have from (114), (115), and (117) that

[TABLE]

where the last line follows by writing \mathbb{E}\big{[}|\mathrm{err}^{(t)}_{f}|\big{]}\leq\sqrt{\mathbb{E}\big{[}|\mathrm{err}^{(t)}_{f}|^{2}\big{]}} via Jensen’s inequality, and then applying (115).

Since $B=\frac{1200k_{0}k_{1}}{\delta}$ and we have assumed $\delta\geq\frac{1}{n}$, we have $B\leq 1200n^{3}$, and hence we have for sufficiently large $c$ that (118) simplifies to $\mathbb{E}[|W_{f}^{(t)}-\widehat{X}^{\prime}_{f}|^{2}]\leq\frac{25}{B}\|\widehat{X}^{\prime}\|_{2}^{2}$. This means that

[TABLE]

by Markov’s inequality.

Taking the median: Recall that $W_{f}$ is the median of $T$ independent random variables, with the median taken separately for the real and imaginary parts. Since $|W|^{2}=|\mathrm{Re}(W)|^{2}+|\mathrm{Im}(W)|^{2}$, we find that (119) holds true when $W_{f}^{(t)}-\widehat{X}^{\prime}_{f}$ is replaced by its real or imaginary part. Hence, with probability at least 1-{T\choose T/2}\big{(}\frac{1}{6t}\big{)}^{T/2}, we have $|\mathrm{Re}(W_{f}^{(t)}-\widehat{X}^{\prime}_{f})|^{2}<\frac{160t}{B}\|\widehat{X}^{\prime}\|_{2}^{2}$, and analogously for the imaginary part. Combining these and applying the union bound, we obtain

[TABLE]

where we first applied ${T\choose T/2}\leq 2^{T}$, and then the choice $T=10\log\frac{2}{p}$ from Algorithm 9 and the choice of $p\leq 1/2$.

We now bound the error as follows:

[TABLE]

We write the expected value of the second term as

[TABLE]

where we applied the change of variable $v=1+\frac{u}{\frac{320}{B}\|\widehat{X}^{\prime}\|_{2}^{2}}$. By incorporating (120) into this integral, we obtain

[TABLE]

where the second line is by explicitly evaluating the integral (with $T>2$), and the third by $T/2-1\geq 4$ (cf., Algorithm 9).

Summing (121) over $\mathcal{F}=\cup_{j\in L}I_{j}$, we find that the total error is upper bounded as follows:

[TABLE]

From (122), the expected value of the second term is at most $p\cdot\frac{80|\mathcal{F}|}{B}\|\widehat{X}^{\prime}\|_{2}^{2}$, and hence

[TABLE]

with probability at least $1-p$, by Markov’s inequality. The lemma now follows by the choice $B=\frac{1200}{\delta}k_{0}k_{1}$ in Algorithm 9.

Sample complexity and runtime:

To calculate the sample complexity, note that the only operation in the algorithm that takes samples is the call to HashToBins. By Lemma 4.6, and the choices $B=1200\frac{k_{0}k_{1}}{\delta}$ and $F=10\log\frac{1}{\delta}$, the sample complexity is $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta})$ per hashing performed. Since we run the hashing in a loop $10\log\frac{2}{p}$ times, this amounts to a total of $O(\frac{k_{0}k_{1}}{\delta}\log\frac{1}{\delta}\log\frac{1}{p})$.

The runtime depends on two operations. The first one is calling HashToBins, whose analysis follows similarly to the aforementioned sample complexity analysis using the assumption $\|\widehat{\chi}\|_{0}=O(k_{0}k_{1})$, but with an extra $\log n$ factor compared to the sample complexity, as per Lemma 4.6.

The other operation is computation of $W_{f}^{(t)}$, which takes unit time for each $f\in\mathcal{F}$. Since the size of $|\mathcal{F}|=k_{1}\cdot|L|$, running it in a loop costs $O(k_{1}\cdot|L|\log\frac{1}{p})$. Computing the median is done in linear time which consequently results in $|\mathcal{F}|T=O(k_{1}\cdot|L|\log\frac{1}{p})$. ∎