Serial Quantization for Sparse Time Sequences

TL;DR

This paper introduces SQuaTS, a novel serial quantization method for sparse time sequences inspired by group testing, which improves accuracy and efficiency over previous compressed sensing approaches by processing signals sequentially without full observation.

Contribution

The paper presents SQuaTS, a new serial quantization scheme for sparse signals that updates representations sequentially and extends to distributed scenarios, enhancing accuracy and computational efficiency.

Findings

SQuaTS achieves higher accuracy than previous CS-based schemes.

It operates without needing complete signal observation before quantization.

The method can be extended to distributed sparse signal acquisition.

Abstract

Sparse signals are encountered in a broad range of applications. In order to process these signals using digital hardware, they must be first sampled and quantized using an analog-to-digital convertor (ADC), which typically operates in a serial scalar manner. In this work, we propose a method for serial quantization of sparse time sequences (SQuaTS) inspired by group testing theory, which is designed to reliably and accurately quantize sparse signals acquired in a sequential manner using serial scalar ADCs. Unlike previously proposed approaches which combine quantization and compressed sensing (CS), our SQuaTS scheme updates its representation on each incoming analog sample and does not require the complete signal to be observed and stored in analog prior to quantization. We characterize the asymptotic tradeoff between accuracy and quantization rate of SQuaTS as well as its computational burden. We also propose a variation of SQuaTS, which trades rate for computational efficiency. Next, we show how SQuaTS can be naturally extended to distributed quantization scenarios, where a set of jointly sparse time sequences are acquired individually and processed jointly. Our numerical results demonstrate that SQuaTS is capable of achieving substantially improved representation accuracy over previous CS-based schemes without requiring the complete set of analog signal samples to be observed prior to its quantization, making it an attractive approach for acquiring sparse time sequences.

Figures

Equations

$$Q_M^{n,\mathrm{opt}}\left(\cdot \right)=\underset{Q_M^n\left(\cdot \right)}{\min }\mathbb{E}\left\{d_n\left(\mathbf{s},Q_M^n\left(\mathbf{s}\right)\right)\right\}.$$

$$d_n\left(\mathbf{s},\hat{\mathbf{s}}\right)\triangleq \frac{1}{n}||\mathbf{s}-\hat{\mathbf{s}}|{|}^2.$$

$${\mathbf{y}}_i={\mathbf{y}}_{i-1}\bigvee {\tilde{\mathbf{c}}}_i.$$

$${\mathbf{y}}_T=\underset{i=1}{\overset{T}{\bigvee }}{\tilde{\mathbf{c}}}_i.$$

$$\Pr\left({{\boldsymbol{y}}}_{T}\big{|}\hat{{\boldsymbol{C}}}_{\mathcal{X}_{w}}\right)\geq\Pr\left({{\boldsymbol{y}}}_{T}\big{|}\hat{{\boldsymbol{C}}}_{\mathcal{X}_{\tilde{w}}}\right),\quad\forall\tilde{w}\neq w.$$

$$D_T(l)\triangleq \frac{1}{T}\sum _{i=1}^T\mathbb{E}\left[|s[i]-Q_{l+1}(s[i]){|}^2\right].$$

$$R\ge R_{\epsilon }(l)\triangleq \underset{1\le i\le k}{\max }\frac{(1+\epsilon )k}{i\cdot T}\log \left(\left(\genfrac{}{}{0ex}{}{T-k}{i}\right)\cdot l^i\right).$$

$$D_{T}(l)\approx\frac{1}{12}2^{-2\log(l+1)}\Big{(}\int\limits_{\alpha\in\mathcal{R}}f_{s}^{1/3}(\alpha)d\alpha\Big{)}^{3}.$$

$$R_{\epsilon }(l)\le (1+\epsilon )\frac{k}{T}\log \left(T\cdot l\right).$$

$$b=\mathcal{O}\left(k\log T+k\log l\right).$$

$$nTl\le {\left(\frac{b}{|c(1)|}\right)}^{|c(1)|},$$

$$\frac{1}{\ln 2{\log }_2(\frac{k}{\ln 2})}-1\le {\epsilon }^{\prime },$$

$$R\ge {\bar{R}}_{\epsilon }(l)\triangleq \frac{(1+\epsilon )e}{T}k\log \left(T\cdot l\right),$$

$${\mathbf{y}}_T=\underset{m=1}{\overset{P}{\bigvee }}{\mathbf{z}}_m.$$

$$R\ge \underset{u\in \mathcal{I}(k)}{\max }\frac{(1+\epsilon )k}{u\cdot nT}\log \left(\vartheta \cdot l^u\right).$$

$$b\ge \underset{1\le i\le k}{\max }\frac{(1+\epsilon )k}{i}\log \left(\genfrac{}{}{0ex}{}{T-k}{i}\right)l^i,$$

$$R\ge \underset{1\le i\le k}{\max }\frac{(1+\epsilon )k}{i\cdot T}\log \left(\left(\genfrac{}{}{0ex}{}{T-k}{i}\right)\cdot l^i\right),$$

Full text

I Introduction

Quantization allows continuous-amplitude physical signals to be represented using discrete values and processed in digital hardware. Such continuous-to-discrete conversions play an important role in digital signal processing systems [1]. In theory, jointly mapping a set of samples via vector quantization yields the most accurate digital representation [2, Ch. 10]. However, as such joint mappings are difficult to implement, quantization is most commonly carried out using ADCs, which operate in a serial and scalar manner, namely, the analog signal is sampled and each incoming sample is sequentially mapped into a discrete representation using the same mapping [3]. Since ADCs operating at high frequencies are costly in terms of memory and power usage, it is often desirable to utilize low quantization rates, i.e., assign a limited number of bits per each input sample, inducing additional quantization error which degrades the digital representation accuracy [4, Ch. 23].

The quantization error encountered under bit budget constraints can be mitigated by accounting for underlying structure or the system task. Such quantization systems are the focus of several recent works. For example, scalar quantization mappings designed to maximize the mutual information and Fisher information with respect to a statistically related quantity were studied in [5] and [6], respectively. The work [7] showed that a quantization system using uniform ADCs can approach the performance achievable using vector quantizers when the system task is not to recover the analog signal, but to estimate some lower-dimensional information embedded into it. This approach was extended to massive multiple-input multiple-output (MIMO) channel estimation with quantized outputs in [8] as well as to the recovery of quadratic functions in [9], while a data-driven implementation was proposed in [10]. The systems proposed in [7, 8, 9, 10] used hybrid architectures, namely, allowed some constrained processing to be carried out in analog prior to quantization, in order to mitigate the error induced by bit-limited serial scalar ADCs.

Conventional quantization theory considers the acquisition of a discrete-time analog source into a digital form [1]. In some practical applications, such as sensor networks, multiple signals are acquired in distinct physical locations, while their digital representation is utilized in some central processing device, resulting in a distributed quantization setup. The recovery of a single parameter from the acquired signals was considered in [11, 12] and its extension to the recovery of a common source, known as the CEO problem, was studied in [13, 14], see also [15, Ch. 12]. Joint recovery of sources acquired in a distributed manner was studied in [16], which focused on sampling, while [17, 18] proposed non-uniform quantization mappings for the representation of multiple sources. Multivariate (vector) quantizers for arbitrary networks were considered in [19].

A common structure exhibited by physical signals is sparsity. Sparse signals are frequently encountered in various applications, ranging from biomedical and optical imaging [20, 21] to radar [22] and communications [23, 24]. An important property of sparse signals is the fact that they can be perfectly reconstructed from a lower-dimensional projection without knowledge of the sparsity pattern. This property is studied within the framework of CS [25, 26], which considers the recovery of sparse signals from their lower dimensional projections.

Recovery of sparse signals from quantized measurements is the focus of a large body of work [27, 28, 29, 30, 31, 32, 33]. The most common approach studied in the literature is to first project the signal in the analog domain and then quantize the compressed measurements, via one bit representation [27, 28], uniform quantization [29], sigma-delta quantization [30, 33], or vector source coding [31]. A detailed survey and analysis of methods combining quantization and CS can be found in [32]. In the context of distributed systems, CS for multiple signals acquired separately was studied in [34, 35, 36, 37, 38], while [39, 40] proposed vector quantization schemes for bit-constrained distributed CS. Despite the similarity, there is a fundamental difference between distributed quantization of sparse signals and distributed CS with quantized observations: In the quantization framework, the measurements are the sparse signals, while in CS the observations are a linear projection of the signals. Consequently, to utilize CS methods, one must first have access to the complete signal in order to project it and then quantize, imposing a major drawback when acquiring time sequences. On the other hand, a sequential approach allows to quantize without requiring that the entirety of the signal be accessible, which is particularly relevant not only in the distributed scenarios, but also for signals sparse in time, since storing the entire time-signal in analog form is expensive. Furthermore, while CS algorithms have been proven to achieve asymptotic recovery guarantees, their performance may be degraded in finite signal sizes. These drawbacks give rise to the need for a reliable and sequential method for quantizing and recovering sparse signals, which is the focus of this work.

Here, we propose SQuaTS, which is a method for quantizing and recovering discrete-time sparse time sequences utilizing standard serial scalar ADC quantizers 111While ADC traditionally refers to hardware which samples and quantizes an analog signal, here we denote by ADC only the quantization end of this hardware, i.e., we assume throughout the paper a discrete time input signal.. Our scheme is inspired by recent developments in group testing theory, and leverages coding principles initially designed for secure group testing [41] to facilitate recovery of the time sequence. In particular, SQuaTS first quantizes each sample using a scalar ADC, and uses the ADC output to update a single binary value, which in turn is used as a codeword from which the sequence can be accurately recovered with high probability. The resulting coding scheme, which quantizes the sparse signal directly and operates over the binary field, allows improved reconstruction compared to CS-based methods, which process a quantized linear projection of the real-valued observations, while also avoiding the need to store samples in analog by sequentially updating a single register.

We characterize the achievable accuracy of SQuaTS in the asymptotically large signal size regime, showing that any fixed desirable distortion level can be achieved with an overall number of bits which grows logarithmically in the signal dimensionality and linearly with the support size. We then characterize the computational complexity of SQuaTS, and propose a reduced complexity scheme for implementing SQuaTS at the cost of degraded representation accuracy.

Next, we discuss how SQuaTS can be naturally applied for distributed quantization of a set of temporally jointly sparse time sequences. We begin with the case where each acquired signal is conveyed to the central unit via a direct link, representing, e.g., single-hop networks. Then, we show how the technique can be extended to multi-hop networks, in which the quantized data must travel over multiple intermediate links to reach the central server, and formulate simplified network policies, dictating the behavior of each intermediate node. We characterize the achievable distortion of SQuaTS when applied in a distributed setup, and derive conditions on the system parameters under which it can achieve the same distortion as when applied in a non-distributed case, assuming that there exists at least a single path to the central unit.

Our numerical results demonstrate that both SQuaTS and its reduced complexity variation substantially outperform the conventional approach combining CS and quantization when applied to the digital representation of a single sparse time-sequence, as well as in distributed acquisition scenarios with jointly sparse signals. This demonstrates the potential of SQuaTS for feasible and reliable quantization of sequentially acquired sparse time sequences.

The rest of this paper is organized as follows: In Section II we review some preliminaries in quantization theory and present the system model. Section III proposes SQuaTS along with a discussion and an asymptotic performance analysis. Section IV presents a reduced complexity variation of SQuaTS. In Section V we apply SQuaTS for distributed quantization. Section VI details the simulation study, and Section VII provides concluding remarks. Proofs of the results stated in the paper are detailed in the appendix.

Throughout the paper, we use boldface lower-case letters for vectors, e.g., ${{\boldsymbol{x}}}$. Matrices are denoted with boldface upper-case letters, e.g., ${\boldsymbol{M}}$. Sets are expressed with calligraphic letters, e.g., $\mathcal{X}$, and $\mathcal{X}^{n}$ is the $n$th order Cartesian power of $\mathcal{X}$. The stochastic expectation is denoted by $\mathbb{E}\{\cdot\}$, $\bigvee$ is the Boolean OR operation, and $\mathcal{R}$ is the set of real numbers. All logarithms are taken to base-2.

II Preliminaries and System Model

II-A Preliminaries in Quantization Theory

To formulate the quantization of sparse signals setup, we first briefly review standard quantization notions. We begin with the definition of a quantizer:

Definition 1** (Quantizer).**

A quantizer $Q_{{M}}^{{n,m}}\left(\cdot\right)$ with $\log M$ bits, input size $n$, input alphabet $\mathcal{S}$, output size $m$, and output alphabet $\hat{\mathcal{S}}$, consists of: 1) An encoding function $g_{n}^{\rm e}:\mathcal{S}^{n}\mapsto\{0,1,\ldots,M-1\}\triangleq\mathcal{M}$ which maps the input from $\mathcal{S}^{n}$ into a discrete index $j\in\mathcal{M}$. 2) A decoding function $g_{m}^{\rm d}:\mathcal{M}\mapsto\hat{\mathcal{S}}^{m}$ which maps each index $j\in\mathcal{M}$ into a codeword ${\boldsymbol{q}}_{j}\in\hat{\mathcal{S}}^{m}$.

The quantizer output for an input ${{\boldsymbol{s}}}\in\mathcal{S}^{n}$ is $\hat{{{\boldsymbol{s}}}}=g_{m}^{\rm d}\left(g_{n}^{\rm e}\left({{\boldsymbol{s}}}\right)\right)\triangleq Q_{{M}}^{{n,m}}\left({{\boldsymbol{s}}}\right)$. An illustration is depicted in Fig. 1.

Scalar quantizers have scalar input and output, i.e., $n=m=1$ and $\mathcal{S}$ is a scalar space, while vector quantizers operate a multivariate input. When the input size and output size are equal, namely, $n=m$, we write $Q_{{M}}^{{n}}\left(\cdot\right)\triangleq Q_{{M}}^{{n,n}}\left(\cdot\right)$, while for scalar quantizers we use the notation $Q_{{M}}\left(\cdot\right)\triangleq Q_{{M}}^{{1}}\left(\cdot\right)$.

In the conventional quantization problem, a $Q_{{M}}^{{n}}\left(\cdot\right)$ quantizer is designed to minimize some distortion measure $d_{n}:\mathcal{S}^{n}\times\hat{\mathcal{S}}^{n}\mapsto\mathcal{R}^{+}$ between its input and its output. The performance of a quantizer is therefore characterized using two measures: The quantization rate, defined as $R\triangleq\frac{1}{n}\log M$, and the expected distortion $\mathbb{E}\{d_{n}\left({{\boldsymbol{s}}},\hat{{{\boldsymbol{s}}}}\right)\}$. For a fixed input size $n$ and codebook size $M$, the optimal quantizer is given by

[TABLE]

In the following, the distortion between a source realization ${{\boldsymbol{s}}}$ and a reconstruction sequence $\hat{{{\boldsymbol{s}}}}$ is defined as the MSE of their difference, which is given by

[TABLE]

Characterizing the optimal quantizer via (1) and the distortion via (2), as well as the optimal tradeoff between distortion and quantization rate, is in general a difficult task. Consequently, optimal quantizers are typically studied assuming either high quantization rate, i.e., $R\rightarrow\infty$, see, e.g., [42], or asymptotically large input size, namely, $n\rightarrow\infty$, typically with stationary inputs, via rate-distortion theory [2, Ch. 10].

Comparing high rate analysis for scalar quantizers and rate-distortion theory for vector quantizers demonstrates the sub-optimality of serial scalar quantization. For example, for quantizing a large-scale real-valued Gaussian random vector with i.i.d. entries and sufficiently large quantization rate $R$, where intuitively there is little benefit in quantizing the entries jointly over quantizing each entry independently, vector quantization notably outperforms serial scalar quantization [4, Ch. 23.2]. Nonetheless, vector quantizers are significantly more complex compared to scalar quantizers. One of the main sources for this increased complexity stems from the fact that vector quantizers operate on a set of analog samples. As a result, a digital signal processor (DSP) utilizing vector quantizers to acquire an analog time sequence must store $n$ samples in the analog domain before it produces a digital representation, which may be difficult to implement, especially for large $n$. Scalar quantizers, commonly used in ADC devices [43], do not store data in analog as each incoming sample is immediately converted into a digital representation, and are the focus here.

II-B System Model

We consider the acquisition of a sampled time sequence $s[i]$ observed over the period $i\in\{1,\ldots,T\}\triangleq\mathcal{T}$ into a digital representation $\hat{s}[i]$ using up to $b$ bits, i.e., $M=2^{b}$ codewords. The signal $s[i]$ is temporally sparse with support size $k<T$, where $k$ is a-priori known222While we carry out our derivations and analysis assuming $k$ is known, we only require an upper bound on it. In fact, SQuaTS method support operation with erroneous knowledge of $k$, as discussed in Section III-E.. We propose a quantization system which is specifically designed to exploit this sparsity to improve the recovery accuracy. In particular, we propose an encoder-decoder pair which utilizes tools from group testing theory to exploit the underlying sparsity of the continuous amplitude signal.

In order to avoid the need to store samples in analog, the system operates at each sample $s[i]$ independently. In particular, on each time instance $i\in\mathcal{T}$, the encoder updates a register of $b$ bits, whose value upon the encoding of $s[i]$ is denoted by ${{\boldsymbol{y}}}_{i}$. Once the complete time sequence is observed, i.e., $i=T$, the decoder uses the digital codeword ${{\boldsymbol{y}}}_{T}$ to produce an estimate of the sequence denoted $\{\hat{s}[i]\}_{i\in\mathcal{T}}$. An illustration of the system is depicted in Fig. 2. Since the decoder processes discrete codeword ${{\boldsymbol{y}}}_{T}$, while each ${{\boldsymbol{y}}}_{i}$, ${i<T}$ is stored only during the $i$th acquisition step, the system uses $b$ bits for digital representation.

III The SQuaTS System

We next detail the proposed SQuaTS system. The main rationale of SQuaTS is to facilitate quantization of sparse signals using conventional low-complexity serial scalar quantizers by utilizing group testing theory tools. Broadly speaking SQuaTS quantizes each incoming sample using a scalar ADC. However, instead of storing this quantized value, it is used to update a $b$ bits codeword, which is decoded into a digital representation of the sparse signal. This approach allows to quantize each incoming sample with relatively high resolution, while using a single $b$ bits register from which the digital representation of the complete signal is obtained.

To properly formulate SQuaTS, we first present the codebook generation in Subsection III-A. Then, we elaborate on the SQuaTS encoder and decoder structures in Subsections III-B and III-C, respectively. In Subsection III-D we characterize the achievable distortion of SQuaTS in the large signal size regime. Finally, in Subsection III-E we discuss the pros and cons of SQuaTS compared to previously proposed approaches for quantizing sparse signals.

III-A Codebook Generation

The SQuaTS system maintains a codebook used by its encoder and decoder. In particular, for a time sequence of $T$ samples, SQuaTS uses a codebook of $l\cdot T+1$ codewords, each consisting of $b$ bits, where $l$ is a fixed integer. We discuss the effect of $l$ on the MSE and the complexity of SQuaTS in Subsection III-D, and propose guidelines for setting its value to optimize the tradeoff between these performance measures.

Our codebook design is based on the wireless sensor coding scheme of [44], which is inspired by recent advances in group testing theory [45], and particularly the code proposed in [41] for secure group testing. The objective in group testing is to identify a subset of defective items in a larger set using as few measurements as possible. This objective can be recast as a codebook generation problem, such that for each outcome vector, i.e., a set of measurements, it should be possible to identify the non-zero inputs [45]. While this setup bears much similarity to our quantization of sparse sources problem, in group testing the inputs are represented over a binary field, while in our setting the inputs can be any real value. Consequently, the codebook here needs to be able not only to detect the indexes of the non-zero inputs, as in conventional group testing, but also to recover their value. To facilitate our design, we henceforth assume that the inputs are discretized to a set of $l+1$ different values, and show how this is incorporated into the overall scheme in the following subsections. We refer to section VI for a discussion on how the parameter $l$ relates to the overall quantization rate.

In particular, to formulate the codebook, we generate $T\cdot l\cdot b$ independent realizations from a Bernoulli distribution with mean value $\frac{\ln(2)}{k}$. These realizations form $l\cdot T$ mutually independent codewords. The codewords are then divided into $T$ bins, denoted $\mathcal{B}_{i}\triangleq\{{\boldsymbol{c}}_{j,i}\}_{j=1}^{l}$, $i\in\mathcal{T}$, and we add to each bin the all-zero codeword denoted ${\boldsymbol{c}}_{0}$. Since ${\boldsymbol{c}}_{0}$ is common to all the bins, the total number of codewords is $l\cdot T+1$. The benefits of this codebook design are discussed in Subsection III-E.

III-B Encoder Structure

Having generated $T$ bins of $l$ codewords, $\{\mathcal{B}_{i}\}_{i=1}^{T}$, we now discuss the encoding process. To that aim, we fix some scalar quantization mapping $Q_{{l+1}}(\cdot)$ over $\mathcal{R}$ with resolution $l+1$, denoted henceforth as $Q(\cdot)$ for simplicity, and let $\{q_{j}\}_{j=0}^{l}$ be the set of its possible outputs. The specific selection of the quantization mapping represents the acquisition hardware. For example, when using the common flash ADC architecture, $Q(\cdot)$ represents a uniform quantization mapping with $l+1$ uniformly spaced decision regions [3]. Without loss of generality, we assume that the scalar quantizer maps the input value [math] into the discrete value $q_{0}$, namely, $Q(0)=q_{0}$.

The encoding process consists of the following three stages, illustrated in Fig. 3:

1. E1

Each incoming sample $s[i]$ is quantized into the discrete scalar value $Q(s[i])$. Since this same identical mapping is applied to each incoming sample in a serial manner, it can be implemented using conventional ADCs. 2. E2

The encoder uses the index of the discrete value $Q(s[i])$ to select a codeword from the $i$th bin as follows: If $q_{j}=Q(s[i])$, then the selected codeword is $\tilde{{\boldsymbol{c}}}_{i}={\boldsymbol{c}}_{j,i}\in\mathcal{B}_{i}$. 3. E3

The encoder output ${{\boldsymbol{y}}}_{i}$, which is initialized such that ${{\boldsymbol{y}}}_{0}$ is the all-zero vector, is updated by taking its Boolean OR with the selected codeword $\tilde{{\boldsymbol{c}}}_{i}$, i.e.,

[TABLE]

Consequently, the encoder output ${{\boldsymbol{y}}}_{T}$ is given by

[TABLE]

Note that only the discrete index of the quantized $Q(s[i])$, and not its actual value, affects the selection of the encoder output ${{\boldsymbol{y}}}_{T}$. Nonetheless, in Subsection III-D we show that the selection of the output of $Q(\cdot)$, i.e., the values of $\{q_{j}\}$, and not only its partition of $\mathcal{R}$ into decision regions, affect the overall MSE of SQuaTS. Additionally, the formulation of ${{\boldsymbol{y}}}_{T}$ via (3) implies that it can be represented using a single register of $b$ bits, which is updated using logical operations on each incoming sample. Consequently, while the encoder assigns a $b$ bits codeword to each incoming sample, the overall output size is $b$ and not $T\cdot b$, thus the quantization rate is $R=\frac{b}{T}$.

III-C Decoder Structure

The recovery of the digital representation $\{\hat{s}[i]\}_{i\in\mathcal{T}}$ from the output of the encoder ${{\boldsymbol{y}}}_{n}\in\{0,1\}^{b}$ is based on maximum likelihood (ML) decoding. In this decoding scheme, the most likely set of $k$ codewords are selected, from which the digital representation is obtained. To formulate the decoding process, recall that the set $\mathcal{T}$ has exactly ${T\choose k}$ possible subsets of size $k$, representing the possible sets of non-zero entries of ${{\boldsymbol{s}}}$. We use $\{\mathcal{X}_{w}\}_{w\in\{1,\ldots,{T\choose k}\}}$ to denote these subsets. The SQuaTS decoder implements the following steps:

1. D1

For a given encoder output ${{\boldsymbol{y}}}_{T}$, the decoder recovers a collection of $k$ codewords $\hat{{\boldsymbol{C}}}_{\mathcal{X}_{w}}=\{{\boldsymbol{c}}_{j_{i},i}\}_{i\in\mathcal{X}_{w}}$, each one taken from a separate bin, for which ${{\boldsymbol{y}}}_{T}$ is most likely, namely,

[TABLE]

The decoder looks for both the set of $k$ bins $\mathcal{X}_{w}$ as well as the selection of the codeword for each bin, i.e., the selection of codeword index $j_{i}$ within the $i$th bin, $i\in\mathcal{X}_{w}$, which maximize the conditional probability (5). 2. D2

The decoder recovers $\{\hat{s}[i]\}$ from $\hat{{\boldsymbol{C}}}_{\mathcal{X}_{w}}=\{{\boldsymbol{c}}_{j_{i},i}\}_{i\in\mathcal{X}_{w}}$ by setting its $i$th entry, denoted $\hat{s}[i]$, to be $\hat{s}[i]=q_{j_{i}}$ for each $i\in\mathcal{X}_{w}$ and $\hat{s}[i]=q_{0}$ for $i\notin\mathcal{X}_{w}$.

The ML decoder scans $\binom{T}{k}(l)^{k}$ possible subsets of codewords in the codebook, i.e., the $\binom{T}{k}$ possible bins corresponding to indexes which may contain non-zero values, and the $l$ codewords in each such bin. For every scanned subset of codewords, the decoder compares the Boolean OR of each subset which contains $k$ codewords to the quantized register ${{\boldsymbol{y}}}_{T}$. Since the length of each codeword is $b$, the computational complexity is on the order of \mathcal{O}\big{(}\binom{T}{k}l^{k}kb\big{)} operations.

While the decoding process described above may be computationally complex, it essentially implements a one-to-one mapping from ${{\boldsymbol{y}}}_{T}$ to $\{\hat{s}[i]\}$, and can thus be implemented using a standard look-up table. In Section IV, we present a sub-optimal low-complexity SQuaTS decoder.

In the following subsection we study the achievable performance, in terms of the tradeoff between quantization rate and distortion, of the proposed SQuaTS system.

III-D Achievable Performance

In order to study the achievable performance, we first note that the SQuaTS encoder and decoder are designed to recover the output of the scalar quantizer $Q(\cdot)=Q_{{l+1}}(\cdot)$. Therefore, when the SQuaTS decoder detects the correct set of codewords, the distortion is determined by the scalar quantizer and its resolution, which is dictated by the parameter $l$.

To formulate this distortion, define the overall average MSE of the scalar quantizer via

[TABLE]

The average MSE (6) is determined by the scalar quantization mapping $Q(\cdot)$ and the distribution of the time sequence $\{s[i]\}$. It represents the accuracy of applying $Q(\cdot)$ directly to the sequence $\{s[i]\}$ without using any additional processing, thus operating at quantization rate of $\log(l+1)$ bits per input sample. SQuaTS with rate $R$, which, as we show next, can be much smaller than $\log(l+1)$, is capable of achieving this average MSE when its decoder successfully recovers the correct set of codewords. A sufficient condition for successful recovery in the limit of asymptotically large inputs, and thus for (6) to be achievable, is stated in the following theorem:

Theorem 1**.**

The SQuaTS system applied to a sparse signal $\{s[i]\}_{i\in\mathcal{T}}$ with support size $k=\mathcal{O}(1)$ achieves the average MSE $D_{T}(l)$ given in (6) in the limit $T\rightarrow\infty$ when for some $\varepsilon>0$, the quantization rate $R$ satisfies:

[TABLE]

Proof: The proof is given in the appendix.

Theorem 1 implies that, as $T$ increases, if the number of bits is $b=R\cdot T$ where $R$ satisfies (7), then the average error probability in detecting the SQuaTS codewords approaches zero (decaying exponentially with $T$) and thus the SQuaTS system achieves the average MSE $D_{T}(l)$ given in (6).

Note that the average MSE $D_{T}(l)$ and the corresponding quantization rate $R_{\varepsilon}(l)$ both depend on the auxiliary parameter $l$. The dependence of $D_{T}(l)$ on $l$ is obtained from the quantization mapping used, as well as the distribution of the input $\{s[i]\}$. For example, when the samples of $\{s[i]\}$ are identically distributed with probability density function (PDF) $f_{s}(\cdot)$, then, using the Panter-Dite approximation [46], the optimal (non-uniform) scalar quantizer in the fine quantization regime achieves the following average MSE:

[TABLE]

The average MSE in (8) implies that the achievable distortion using scalar quantizers, including conventional architectures such as uniform quantization mappings, can be made arbitrarily small by increasing the resolution $\log(l+1)$.

While the average MSE directly depends on the quantization mapping, the rate $R_{\varepsilon}(l)$ is invariant to the setting of $Q(\cdot)$, and is obtained as the maximal value of the right hand side of (7). To avoid the need to search for the maximal value in (7), we state an upper bound on $R_{\varepsilon}(l)$ in the following corollary:

Corollary 1**.**

The quantization rate $R_{\varepsilon}(l)$ in (7) satisfies

[TABLE]

Proof:

The corollary is obtained by substituting in (7) the upper bound $\log\binom{T-k}{i}\leq i\log T$, which follows from Stirling’s approximation [2, 47]. ∎

We note that when $k=\mathcal{O}(1)$, the upper bound (9) tends to zero as $T$ grows for any fixed $l$. Consequently, for large $T$, SQuaTS requires significantly smaller quantization rates to achieve $D_{T}(l)$ compared to directly applying $Q(\cdot)$ to $\{s[i]\}$, which requires a rate of $\log(l+1)$ to achieve the same average MSE. This gain, which demonstrates the ability of SQuaTS to exploit the underlying sparsity of $\{s[i]\}$, is also observed in the simulations study presented in Section VI.

Corollary 1 can be used to determine the quantization rate for achieving a desirable MSE for a given family of scalar quantization mappings: The parameter $l$ is set to the minimal value for which $D_{T}(l)$ is not larger than the desirable distortion. Next, using the resulting $l$, the quantization rate can be obtained using the right-hand side of (9). Theorem 1 guarantees that, for large input size $T$, the desirable distortion is achievable when using SQuaTS with the selected quantization rate. In fact, in the numerical study presented in Section VI we demonstrate that, by properly tuning $l$, the proposed system can achieve substantial MSE gains over previously proposed approaches for quantizing sparse time sequences.

The bound on the quantization rate required to approach $D_{T}(l)$ given in Corollary 1 can also be used to characterize the asymptotic growth rate of the number of quantization bits used by the SQuaTS system, $b$, as stated in the following corollary:

Corollary 2**.**

The MSE $D_{T}(l)$ can be approached as $T$ increases when the number of quantization bits $b$ grows as

[TABLE]

Corollary 2 implies that, besides the obvious linear dependence in $k\log l$, the required number of bits grows proportionally to a logarithmic factor of $T$, which depends on the sparsity pattern size $k$. A similar asymptotic growth in the number of bits, i.e., proportional to $k\log T$, was also shown to be sufficient to achieve a given distortion when using CS-based methods in [27, Thm. 2]. However, our numerical study presented in Section VI demonstrates that despite the similarity in the asymptotic growth, when $b$ is fixed, SQuaTS achieves improved reconstruction accuracy compared to CS-based techniques.

Substituting (10) in the ML decoding complexity in Subsection III-C allows us to characterize the computational burden of the ML decoder, as stated in the following corollary:

Corollary 3**.**

SQuaTS* with the ML decoder detailed in Subsection III-C is capable of achieving the MSE $D_{T}(l)$ (6) in the limit $T\rightarrow\infty$ with a computational complexity on the order of $\mathcal{O}\left(\binom{T}{k}l^{k}k^{2}\log T+\binom{T}{k}l^{k}k^{2}\log l\right)$ operations.*

The complexity of the SQuaTS decoder is significantly affected by the size of the sparsity pattern $k$ in a much more dominant manner compared to its dependence on the signal size $T$, and the resolution of the scalar quantizer $l$. While this implies that the SQuaTS system is most computationally efficient for highly sparse inputs, the proposed mechanism is applicable for any size of the sparsity pattern.

We note that SQuaTS is geared towards low rate and low resolution scenarios, where one typically has much to gain by incorporating coding schemes in quantization over merely utilizing serial scalar ADCs. As shown in (10), the codeword length scales logarithmically with $l$ and $T$. If the codewords are too long due to design issues, e.g., high values of $l$ yielding fine resolution quantization are desired, fragmentation as proposed in [44] can be used. That is, in the SQuaTS system, fragmentation by dividing the set of input bits $T$ or set the levels $l$ into two or more groups. This operation reduces the memory size and the decoding complexity as needed. Furthermore, when the sparsity level is approximately identical among the groups, as is the case for large groups with i.i.d. inputs or in the presence of prior knowledge of structured sparsity, fragmentation does not compromise the performance of the proposed SQuaTS system, as we numerically demonstrate in Fig. 10 and Fig. 14 in Section VI.

III-E Discussion

We next discuss the practical aspects of this method and its rationale. In particular, we first detail the benefits which stem from the SQuaTS architecture and compare it to related schemes for quantizing sparse signals, such as direct application of scalar quantizers as well as compress-and-quantize [29, 27, 28, 30, 33]. Then, we elaborate on the relationship between SQuaTS and group testing theory.

III-E1 Practical benefits and comparison with related schemes

SQuaTS is specifically designed to utilize scalar ADCs in a serial manner. The resulting structure can be therefore naturally implemented using practical ADC architectures [3]. Moreover, SQuaTS is tailored to exploit an underlying sparsity of the input signal. Straight-forward application of a serial scalar ADC requires $T\cdot\log(l+1)$ bits to achieve the distortion $D_{T}(l)$ in (6). Our proposed SQuaTS, which exploits the sparsity of the input by further encoding the ADC output in a serial manner, requires $b=\mathcal{O}\left(k\log(Tl)\right)$ bits to achieve the same MSE, as follows from Corollary 2. This implies that for highly sparse signals, i.e., when $k\ll T$, SQuaTS significantly reduces the number of bits while utilizing scalar ADCs for acquisition, by introducing an additional encoding applied in a serial manner at its output. The resulting approach thus bears some similarity to previously proposed universal quantization methods which are based on applying entropy coding to the output of a quantizer. See [48] for scalar quantizers and [49] for vector quantizers. Indeed, since the codewords representing the quantized value are generated according to a Bernoulli distribution with mean value $\frac{\ln(2)}{k}$ and the outcome ${{\boldsymbol{y}}}_{T}$ is the Boolean OR of $k$ inputs, it can be shown that its entries approach being independent and equally distributed on the set $\{0,1\}$ for large values of $k$, namely, the optimal lossless encoded representation, as achieved using entropy coding [2, Ch. 5]. Nonetheless, to apply conventional entropy coding, one must first quantize all the entries of the input (or at least a large block of input entries) before applying the encoding process, requiring a large number of bits to store and represent this quantized block. SQuaTS, which is specifically designed to operate in a serial manner, updates the same $b$-bits register on each incoming sample, thus avoiding the need to store the output of the serial scalar ADC $Q(\cdot)$ prior to its encoding.

Arguably the most common approach considered in the literature for quantization of sparse signals is based on CS techniques. In these methods, a sensing matrix is used to linearly combine the sparse signal into a lower-dimensional vector, which is then quantized, either using optimal vector quantization, as in [31], or more commonly, via some scalar continuous-to-discrete mapping, as in [29, 27, 28, 30]. When the input signal is a sequentially acquired time sequence, as considered here, such CS based techniques need to store the incoming samples in the analog domain prior to their combining using the sensing matrix333One may also store only the lower-dimension compressed vector and update its entries on each incoming input sample. Yet, this approach still requires the storage of a large amount of samples in analog as quantization can only be carried out once the complete signal is compressed.. This requirement, which does not exist for our proposed SQuaTS, limits the applicability of these proposed schemes, especially for long time sequences, i.e., in the regime typically considered in the literature. It should be stressed that CS-based methods assumes a simple acquisition, i.e. conventionally linear, at the expense of a more complex decoding process. SQuaTS on the other hand, is a more involved encoding scheme which is tailored to the task of serial acquisition of sparse signals.

An additional benefit of SQuaTS compared to CS-based methods, stems from its usage of binary codebooks for compression. SQuaTS operates directly on $\{s[i]\}$, and not on its lower dimensional projections as in CS, assign to binary codewords which originate from group testing theory, based on the value of $\{s[i]\}$, and more precisely, on $\{Q(s[i])\}$. By doing so, SQuaTS achieves improved immunity to measurement errors compared to operating over fields of higher cardinality. This benefit is translated to more accurate digital representations, as numerically demonstrated in Section VI.

Our analysis of SQuaTS is carried out assuming that the value of $k$ used in the design of the quantization is the true sparsity level, as detailed in Section II-B. However, SQuaTS is applicable and its performance guarantees hold also when only an upper bound on $k$ is known, and the actual sparsity pattern is smaller. This is a common assumption in the group testing literature [50], and it was shown that $k$ can be estimated in real-time with $O(\log T)$ bits [51, 52]. In general, the sparsity assumption, i.e., $k\ll T$, allows the incorporation of group testing tools to yield accurate and computationally feasible serial quantization. When the number of non-zero samples is higher than the value of $k$ used to design the code, the error probability will increase. However, if the quantization rate of the code defined is higher than the sufficiency rate (given in Theorem 1), the decoder will not fail drastically. Only some degradation in the MSE results are obtained, as we numerically demonstrate in Section VI-A.

The codebook used by the proposed SQuaTS system, as defined in Section III-A, is generated randomly. When $b$ is large, the probability of repetition is small [53]. In the case that $b$ is small, one can choose in the codebook generation stage ”typical codewords”, namely, only codewords without repetition to avoid errors in the recovery at the decoding process. We note that the numbers of non-zeros bits in the codewords is dependent on $b$ rather than on the sparsity level $k$. On average, the $nTl$ required codewords in SQuaTS system have $p\cdot b\approx\frac{\ln 2}{k}\cdot(1+\varepsilon)k\log_{2}nTl\triangleq|c(1)|$ non-zeros bits. Hence, the number of possible codewords is given by, $\binom{b}{|c(1)|}\geq\left(\frac{b}{|c(1)|}\right)^{|c(1)|}$, and thus to be able to generate sufficient codewords without repetitions in the codebook, it is required that

[TABLE]

for any $k,n,l,T$ and some $\varepsilon>0$. Rearranging terms in the inequality results in

[TABLE]

where $\varepsilon=\varepsilon^{\prime}+\varepsilon^{\prime\prime}$ and $0\leq\varepsilon^{\prime}<\varepsilon$. In Fig. 4 it is numerically demonstrated that for any $k\neq 1$ there are sufficient possible codewords for any $n,l,T$ with $\varepsilon^{\prime}=0$. For $k=1$ it is required to increase the size of $b$ by $\varepsilon^{\prime}$ to obtain sufficient possible codewords without repetitions.

A possible drawback of SQuaTS compared to CS-schemes stems from the fact that SQuaTS is designed assuming that the signal $\{s[i]\}$ is sparse, i.e., that at most $k$ of its entries are non-zero. CS methods are commonly capable of reliably recovering signals which are sparse in an alternative domain, namely, when there exists a non-singular matrix ${\boldsymbol{P}}$ such that ${\boldsymbol{P}}{{\boldsymbol{s}}}$ is sparse, where ${{\boldsymbol{s}}}$ is the $T\times 1$ vector representation of $\{s[i]\}$. It is noted though that SQuaTS can still be applied to such signals by first projecting the signal using the matrix ${\boldsymbol{P}}$, resulting in a sparse signal which can be represented using SQuaTS. Such application however requires the entire signal to be first acquired, as is the case with conventional CS methods.

Finally, we note that while CS based techniques typically require the ratio between the sparsity pattern size $k$ and the input dimensionality $T$ to be upper bounded, our proposed SQuaTS can be applied for any ratio between $k$ and $T$.

III-E2 Relationship to group testing theory

As mentioned in Subsection III-A, the SQuaTS code construction is inspired by codebooks designed for group testing. Group testing first originated from the need to identify a small subset $k$ of infected draftees with syphilis from a large set of a population with size $T$, using as few pool measurements $b$ as possible. Thus, group testing measurements, i.e., codewords, are designed such that given an outcome vector of size $b$, one should be able to identify the defective items, namely the non-zero inputs. As discussed in Subsection III-A, a fundamental difference between our setup and conventional group testing stems from the fact that while in group testing the inputs are represented over the binary field, in our setting the inputs are the quantized values $\{Q(s[i])\}_{i=1}^{T}$ whose alphabet size is $l+1$. Our code construction overcomes this difference by exploiting recent code designs targeting extended group testing models, and in particular, those considered in [44] and in the secure group testing framework [41].

The resulting group testing based code design leads to a compact and accurate digital representation. In particular, due to the binning structure of the code suggested, when $k$ inputs are different from zero there are only $\binom{T}{k}l^{k}$ possible subsets of codewords from which the output of the encoder is selected. For comparison, in a naive codebook which assigns a different codeword to each quantized input value without binning, there are $\binom{Tl}{k}$ possible subsets. This significantly reduces the number of bits required in the outcome vector.

Finally, we note that the construction of the suggested code does not depended on the distribution of the input signal, which is similar to universal quantization methods [48]. In fact, the distribution of $\{s[i]\}$ only affects the MSE induced by the serial scalar quantizer $Q(\cdot)$. The codebook presented in Subsection III-A is designed to allow reliable reconstruction under the worst case scenario, i.e., the setting in which $\{Q(s[i])\}_{i=1}^{T}$ are i.i.d. uniformly distributed. Intuitively, the quantization rate required to achieve the MSE $D_{T}(l)$ can be further reduced by exploiting a-priori information on the input distribution. This approach was considered for the original group testing problem with, e.g., Poisson priors in [54]. We leave investigation of this approach for future study.

IV Efficient Decoding Algorithm

The SQuaTS system detailed in the previous section and its performance analysis rely on ML decoding. In particular, the decoder uses an ML approach to identify the sub-set of $k$ samples of the sparse signal which may be non-zero, along with their corresponding quantized values. Such a decoding algorithm suffers from high computational complexity, as noted in Corollary 3. To overcome this drawback of SQuaTS, in this section we propose a decoding algorithm based on the Column Matching (CoMa) method used in [47] for group testing setups. The proposed algorithm is presented in Subsection IV-A. In Subsection IV-B we analyze the algorithm performance and discuss its benefits.

IV-A CoMa Decoder

As mentioned above, our proposed decoding algorithm is based on the CoMa method [47]. Broadly speaking, unlike the ML decoder, which looks for the set of $k$ codewords from different code bins which are most likely to correspond to the binary vector ${{\boldsymbol{y}}}_{T}$, CoMa decoder attempts to match a codeword from each bin to ${{\boldsymbol{y}}}_{T}$ separately. Replacing the joint search for a set of codewords with a separate examination of each codeword significantly reduces the computational burden, at the cost of degraded decoding accuracy, as we show in Subsection IV-B. The resulting decoder operates with the same code construction and encoder mapping as described for SQuaTS in Subsections III-A and III-B, respectively, thus maintaining the sequential operation and natural implementation with practical ADCs of SQuaTS.

In particular, given an encoder output ${{\boldsymbol{y}}}_{T}$, the CoMa decoder consists of two stages: First, it scans the codebook $\{\mathcal{B}_{i}\}=\{{\boldsymbol{c}}_{j,i}\}$, removing all codewords which could not have resulted in ${{\boldsymbol{y}}}_{T}$. Since the encoding procedure, and specifically, step E3 detailed in Subsection III-B, is based on a logical OR operation between the selected codewords from each bin, any codeword which has a non-zero entry in an index of zero entry of ${{\boldsymbol{y}}}_{T}$ could not have been used in the encoding of ${{\boldsymbol{y}}}_{T}$. Once this elimination stage is concluded, the resulting set of possible codewords, which we denote by $\mathcal{C}$, is used to generate the digital representation $\{\hat{s}[i]\}$. Specifically, for every remaining codeword ${\boldsymbol{c}}_{j,i}$ in $\mathcal{C}$, the decoder sets $\hat{s}[i]$ to be the quantized value assigned to ${\boldsymbol{c}}_{j,i}$, i.e., $\hat{s}[i]=q_{j}$. The time instances $i\in\mathcal{T}$ for which there is no codeword in $\mathcal{C}$ are assumed to have originated from the zero codeword ${\boldsymbol{c}}_{0}$, and are thus set to $q_{0}$. If the remaining set of codewords $\mathcal{C}$ contains several codewords from the same bin, i.e., $\exists j_{1}\neq j_{2}$ such that ${\boldsymbol{c}}_{j_{1},i}\in\mathcal{C}$ and ${\boldsymbol{c}}_{j_{2},i}\in\mathcal{C}$, then one of these codewords is randomly selected as the one used to generate the corresponding recovered sample $\hat{s}[i]$. The decoding method is summarized as Algorithm 1.

IV-B Analysis and Discussion

The CoMa decoder is based on examining each codeword one-by one, which is less computationally complex compared to the straight forward ML evaluation, at the cost of reduced performance. As we show next, Algorithm 1 requires a larger quantization rate $R$ to guarantee that the MSE $D_{T}(l)$ is achievable for any fixed $l$ compared to the ML decoder detailed in Subsection III-C, thus trading computational burden for quantization rate. The performance of the SQuaTS system using the CoMa decoder is stated in the following proposition:

Proposition 1**.**

The MSE $D_{T}(l)$ is achievable by SQuaTS with the CoMa decoder in the limit $T\rightarrow\infty$ when for some $\varepsilon>0$, the quantization rate $R$ satisfies:

[TABLE]

where $e$ is the base of the natural logarithm. For finite and large $T$, the probability of the MSE being larger than $D_{T}(l)$ is at most $T^{-\varepsilon}$.

Proof.

The proof directly follows using similar arguments as in [47], where instead of $T$ possible codewords, in the SQuaTS system there are $T\cdot l$ possible codewords. ∎

Comparing (12) and Corollary 1, which states an upper bound on the corresponding achievable quantization rate when using the ML decoder $R_{\varepsilon}(l)$, indicates that $\bar{R}_{\varepsilon}(l)>R_{\varepsilon}(l)$, i.e., the CoMa decoder requires larger quantization rates to achieve the MSE $D_{T}(l)$ than the ML decoder. However, Proposition 1 implies that, as the sequence length $T$ increases, the asymptotic growth in the number of bits, $b=RT$, is $b=\mathcal{O}(k\log T+k\log l)$, i.e., the same as the growth rate characterized in Corollary 2 for SQuaTS with the ML decoder. Furthermore, Proposition 1 holds for any sparsity pattern size $k$, while the corresponding analysis of the ML decoder requires $k$ not to grow with the sequence length, i.e., $k=\mathcal{O}(1)$.

Based on the characterization of the asymptotic growth of the number of bits $b$, we obtain the complexity of Algorithm 1. The computational burden under which SQuaTS is capable of achieving the MSE $D_{T}(l)$ when using the CoMa decoder is stated in the following Corollary:

Corollary 4**.**

SQuaTS* with the CoMa decoder achieves the MSE $D_{T}(l)$ in the limit $T\rightarrow\infty$ with complexity on the order of $\mathcal{O}(T\cdot l\cdot k\log(T\cdot l))$ operations.*

Proof:

Algorithm 1 essentially scans over all the $T\cdot l$ codewords, comparing each to the $b$-bits binary ${{\boldsymbol{y}}}_{T}$. Consequently, its number of operations is on the order of $\mathcal{O}(Tlb)$. Combining this with the observation that for $b=\mathcal{O}(k\log T+k\log l)$, SQuaTS with the CoMa decoder achieves the MSE $D_{T}(l)$ in the limit $T\rightarrow\infty$ proves the corollary. ∎

Comparing the complexity of the CoMa method in Corollary 4 to that of the ML decoder in Corollary 3 reveals the computational gains of Algorithm 1. Focusing on highly sparse setups where $T\gg k$, and recalling that $\binom{T}{kk}>\frac{T^{k}}{k^{k}}$, Corollary 3 indicates that the complexity of the ML decoder is larger than a term which is dominated by $T^{k}\log T$. Consequently, the ML decoder becomes infeasible as $k$ grows. The corresponding computational complexity of the CoMa decoder in Corollary 4 is dominated by the term $T\log T$, implying that it can be implemented for practically any sparsity pattern satisfying $T\gg k$. In fact, unlike the ML decoder, Algorithm 1 is invariant to the value of $k$, which is required here only for setting the distribution of the codewords, as explained in Subsection III-A, and for determining the quantization rate under which a desired MSE level is achievable with sufficiently high probability.

Finally, we note that the CoMa method detailed here is only one example of an efficient algorithm given in the literature that can be leveraged by the SQuaTS system decoder. In fact, it is possible to use several efficient algorithms proposed initially designed for the purpose of traditional group testing, see e.g., [55, 56], or even modify the coding scheme to be based on systematic codes, as studied in [57]. An additional decoding scheme which can be combined with SQuaTS is the recently proposed two-stage multi-level decoding method recently proposed in [58] for COVID-19 pooled testing. We leave the analysis of SQuaTS with these alternative decoding methods to future investigation.

V SQuaTS for Distributed Quantization

In distributed quantization, a set of signals are acquired individually and jointly recovered. Such setups correspond, e.g., to sensor arrays, where each bit-constrained sensor observes a different time sequence, and all the measured sequences should be recovered by some centralized server. Each sensor may have a direct link to the server, resulting in a single hop structure, or must convey its quantized measurements over a route with intermediate nodes, representing a multi-hop topology. In this section we show how SQuaTS can be naturally applied to distributed quantization setups. We first formulate the system model for distributed quantization in Subsection V-A. Then in Subsection V-B we adapt SQuaTS to distributed quantization over single-hop networks, and discuss how to it can applied to multi-hop networks in Subsection V-C.

V-A Distributed Quantization System Model

We consider distributed acquisition and centralized reconstruction of $n$ analog time sequences. The sequences, denoted $\{s_{m}[i]\}_{m=1}^{n}$ are separately observed over the period $i\in\mathcal{T}$, representing, e.g., sources measured at distinct physical locations. The signals are jointly sparse with joint support size $k$ [35]. We focus on two models for the joint sparsity of $\{s_{m}[i]\}$:

Overall sparsity

Here, the ensemble of all $n$ signals over the observed duration is $k$-sparse, namely, the set $\{s_{m}[i]\}_{m\in\mathcal{N},i\in\mathcal{T}}$, with $\mathcal{N}\triangleq\{1,\ldots,n\}$, contains at most $k$ non-zero entries. This model, in which no structure is assumed on the sparsity pattern of each signal, coincides with the general joint-sparse model of [35] without a shared component.

Structured sparsity

In the second model the signals are sparse in both time and space. Specifically, for each $m\in\mathcal{N}$, the signal $\{s_{m}[i]\}_{i\in\mathcal{T}}$ is $k_{t}$-sparse, while for any $i\in\mathcal{T}$, the set $\{s_{m}[i]\}_{m\in\mathcal{N}}$ is $k_{s}$-sparse. This setup is a special case of overall sparsity with $k=k_{s}k_{t}$ with an additional structure which can facilitate recovery.

Each time sequence $\{s_{m}[i]\}_{i\in\mathcal{T}}$ is encoded into a $b$-bits vector denoted ${{\boldsymbol{x}}}_{m}\in\{0,1\}^{b}$. The encoding stage is carried out in a distributed manner, namely, each ${{\boldsymbol{x}}}_{m}$ is determined only by its corresponding time sequence $\{s_{m}[i]\}_{i\in\mathcal{T}}$ and is not affected by the remaining sequences. The binary vectors $\{{{\boldsymbol{x}}}_{m}\}$ are conveyed to a single centralized decoder over a network, possibly undergoing several links over multi-hop routes. We consider a binary network model, such that each link can be either broken or error-free. The centralized decoder maintains links with $P$ nodes. The $P$ network outputs, denoted $\{{{\boldsymbol{z}}}_{m}\}_{m=1}^{P}$, are collected by the decoder into a $b$-bits vector ${{\boldsymbol{y}}}\in\{0,1\}^{b}$, which is decoded into a digital representation of $\{{s}_{m}[i]\}$, denoted $\{\hat{s}_{m}[i]\}$, as illustrated in Fig. 5. The accuracy is measured by the MSE $\sum_{i}\sum_{m}\mathbb{E}\left[(s_{m}[i]\!-\!\hat{s}_{m}[i])^{2}\right]$ and the quantization rate is $R=\frac{b}{nT}$ here.

V-B Single-Hop Networks

We next show how SQuaTS, proposed in Section III for the quantization of a single time sequence, can be adapted to distributed quantization setups. We begin here with single-hop networks, where each encoder has a direct error-free link to the centralized decoder. In particular, the applicability of SQuaTS to distributed quantization stems from the fact that its encoding procedure, and specifically step E3, is based on applying a logical OR operation to a set of codewords, selected according to the quantized values of each observed sample. The associative property of the logical OR operation implies that an SQuaTS encoder can be applied to an ensemble of $n$ sequences by separately encoding each sequence with an SQuaTS encoder using a different codebook.

In particular, in order to apply SQuaTS in a distributed quantization method, one must simply generate a codebook for the ensemble of sequences, i.e., a total of $nT$ codewords, via the generation procedure detailed in Subsection III-A. Then, the generated codewords are distributed among the $n$ encoders, and each encoder of index $m\in\{1,\ldots,n\}$ uses these codewords to apply the SQuaTS encoding method detailed in Subsection III-B to its corresponding sequence $\{s_{m}[i]\}$. The output of the $m$-th encoder at time instance $T$, i.e., after its sequence is acquired, is used as the binary vector ${{\boldsymbol{x}}}_{m}$ conveyed to the centralized receiver. Each node in the network performs a Boolean OR operation of all incoming input vectors ${{\boldsymbol{x}}}_{m^{\prime}},m^{\prime}\in\{1,\ldots,P^{\prime}\}$. The output of each node given $P^{\prime}$ inputs vectors is thus $\vee_{m=1}^{P^{\prime}}\textbf{x}_{m}$. We denote the output vector of the last nodes before the centralized decoder by $\textbf{z}_{m}$. For a single-hop network, the bit vectors received by the decoder, $\{{{\boldsymbol{z}}}_{m}\}_{m=1}^{P}$, are given by ${{\boldsymbol{z}}}_{m}={{\boldsymbol{x}}}_{m}$ and $P=n$. Consequently, by the associativity of the logical operator, the centralized decoder can recover the output of applying an SQuaTS encoder to the ensemble of sequences, denoted ${{\boldsymbol{y}}}_{T}$, from the outputs of the separate encoders, via

[TABLE]

Using ${{\boldsymbol{y}}}_{T}$, the centralized decoder can recover the estimate of the ensemble of time sequences $\{\hat{s}_{m}[i]\}$, via conventional SQuaTS decoding, i.e., ML decoding detailed in Subsection III-C or the CoMa method presented in Section IV.

The fact that the proposed distributed adaptation of SQuaTS effectively implements the application of SQuaTS to the ensemble of sequences implies that the achievable performance guarantees of SQuaTS, derived in Subsections III-D and IV-B for the ML and CoMa decoders, respectively, hold also in the distributed setup. For example, by letting $D_{T,n}(l)$ be the MSE achieved when $\hat{s}_{m}[i]=Q(s_{m}[i])$, namely, a desirable MSE determined only by the ADC resolution, it follows from Theorem 1 that $D_{T,n}(l)$ is achievable by the distributed quantization scheme when its rate satisfies the condition stated in the following proposition:

Proposition 2**.**

SQuaTS* adapted to distributed quantization using the ML decoder achieves the MSE $D_{T,n}(l)$ in the limit $nT\rightarrow\infty$ with $k=\mathcal{O}(1)$ when the quantization rate $R$ satisfies the following inequality for some $\varepsilon>0$:*

[TABLE]

The parameter $\vartheta$ and the set $\mathcal{I}(k)$ depend on the type of joint sparsity: for overall sparsity, $\vartheta=\binom{nT}{k}$ and $\mathcal{I}(k)=\{1,\ldots,k\}$, while for structured sparsity $\vartheta={n\choose k_{t}}{T\choose k_{s}}$, and $\mathcal{I}(k=k_{s}k_{t})=\{u_{t}u_{s}:1\leq u_{t}\leq k_{t},1\leq u_{s}\leq k_{s}\}$.

Proof:

The proposition follows by repeating the proof of Theorem 1 given in the Appendix, while setting the length of the sequence to be $nT$, i.e., the length of the ensemble of signals, instead of $T$, and noting that the joint sparsity affects the number of possible codeword combinations. In particular, $\vartheta$ is the number of possible sets of non-zero entries in the ensemble of signals over which the ML decoder searches, used in Lemma 1 in the Appendix. ∎

Proposition 2 indicates that, as expected, SQuaTS can exploit structures in the joint sparse nature of the observed signals to improve performance, namely, to utilize less bits while guaranteeing that a desired MSE $D_{T,n}(l)$ is achievable.

V-C Multi-Hop Networks

We now generalize our scheme to a multi-hop network, in which multiple directed links relate the distributed encoders and the centralized decoder. The intermediate nodes in the networks, which act as helpers or relays, can perform basic operations on their input from incoming links. For the sake of space and exposition, we consider a simplified model for this communication network, in which links are assumed to support $b$-bits of information without errors, or result in a complete erasure. We also assume that the transmission is synchronized, i.e., the encoders and intermediate nodes all transmit in sync across their outgoing links, and that the network is acyclic. Note that despite its simplicity, this model is reminiscent of several network models used in the literature, e.g., [15].

The operation of the encoders and the decoder in the multi hop setup is identical to that discussed for single hop networks in Subsection V-B. The only addition is in the network policy, as depicted in Fig. 6: At each intermediate node, we perform a Boolean OR operation of all incoming input vectors (which is the same mathematical operation performed by the encoder and decoders in Subsection V-B), and transmit the result length $b$-vector on all outgoing links. The network outputs are collected in ${{\boldsymbol{y}}}_{T}$ via (13).

Clearly, the resulting bit sequence at the decoder ${{\boldsymbol{y}}}_{T}$ is identical to the one in Subsection V-B, as long as there exist at least one path in the network from each encoder to the centralized decoder. Note that this is in contrast with the previous literature on distributed CS over networks, where it is typical to impose conditions on the network topology that guarantee a successful description [38]. Consequently, the structures of the encoders and the decoder are invariant to whether the encoders communicate with the decoder directly or over multi-hop networks, and the achievable performance of SQuaTS stated in, e.g., Proposition 2, hold in such multi-hop networks. Additionally, the scheme we propose is robust to link failures: as long as there exist at least one path from all encoders to the decoder, any number of link failures in the network still leads to the same received vector at the decoder, i.e., the coding scheme can achieve the min-cut max-flow bound of the network [15, 59].

It follows from the above discussion that the presence of a multi-hop network does not affect the operation of the distributed adaptation of SQuaTS or its achievable performance, and only requires a simplified network policy to be carried out by the intermediate network nodes. While our analysis assumes that each encoder has at least a single path to the decoder, it can be shown that the presence of missing paths for some encoders does not affect the recovery of the remaining signals. In particular, by treating the output of a broken link as the zero vector, if the $m$th encoder has no path to the decoder, the recovery of $\{s_{j}[i]\}_{j\neq m}$ remains intact, while $\hat{s}_{m}[i]$ is estimated as being all zeros.

VI Numerical Evaluations

In this section, we evaluate the performance of the proposed SQuaTS scheme using various simulations, for a fixed and finite signal size $T$. We first numerically evaluate SQuaTS for the quantization of a single sparse signal in Subsection VI-A. Then, in Subsection VI-B we study the resiliency of SQuaTS to noisy digital representations. Finally, in Subsection VI-C, we evaluate its extension to distributed quantization setups, as discussed in Section V.

VI-A Single Sparse Signal

We begin by numerically evaluating SQuaTS used for quantizing a single sparse signal $\{s[i]\}$, $i\in\mathcal{T}$. To this end, we consider two sparse sources with sizes $T\in\{100,50\}$ and support sizes $k\in\{3,2\}$, respectively. To generate each signal, we randomly select $k$ indexes, denoted $\{i_{j}\}_{j=1}^{k}$, and then choose the values of $\{s[i_{j}]\}$ to be i.i.d. zero-mean unit-variance Gaussian random variables, while the remaining entries are set to zero.

Each of the generated signals is quantized and represented in digital form using each of the following methods:

• •

SQuaTS system with l={\color[rgb]{0.00,0.00,0.00}\max\left(\lfloor\frac{1}{T}2^{\frac{TR}{k(1+\epsilon)}}\rfloor,2\right)} following (9), where $\epsilon$ is selected empirically for each point in the range $\epsilon\in[0.8,1.3]$ to maximize the performance of the ML decoder, where we select the value which achieves the minimal MSE among four different values of $\epsilon$ uniformly placed in this region. Here, the continuous-to-discrete mapping $Q(\cdot)$ implements uniform quantization over the region $[-2,2]$. We consider three different SQuaTS decoders: the ML decoder detailed in Subsection III-C the CoMa-based reduced complexity decoder proposed in Section IV which is tuned with the same value of $\epsilon$ as that used by the ML decoder; and the CoMa decoder with $\epsilon$ optimized by fine search separately for each quantization rate.

• •

A uniform scalar quantizer with support $[-2,2]$ applied separately to each sample of $\{s[i]\}$, mapping every decision region to its centroid. This system, which models the direct application of a serial scalar ADC to the sparse signal $\{s[i]\}$, can be utilized only when the quantization rate satisfies $R\geq 1$, as the quantizers require at least one bit.

• •

A compress-and-quantize system which first compresses ${{\boldsymbol{s}}}=[s[1],\dots,s[T]]^{T}$ into $\mathcal{R}^{m}$, where $m$ is selected in the range $[6k,20k]$ to minimize the MSE. The compression is carried out using a sensing matrix ${\boldsymbol{A}}\in\mathcal{R}^{m\times T}$ whose entires are i.i.d. zero-mean unit variance Gaussian RVs. The compressed signal ${\boldsymbol{A}}{{\boldsymbol{s}}}$ is quantized using a uniform scalar quantizer with support $[-2,2]$. The digital representation $\hat{{{\boldsymbol{s}}}}=[\hat{s}[1],\dots,\hat{s}[T]]^{T}$ is then recovered using the quantized iterative hard thresholding (QIHT) method [60] as well as fast iterative soft thresholding algorithm (FISTA) [61].

All of the above schemes are compared with the same number of bits $b=R\cdot T$, and the MSE is computed by averaging the squared error over $100$ Monte Carlo simulations.

The empirically evaluated MSEs of the considered quantization systems versus the quantization rate $R$ are depicted in Figs. 7-8 for the setups with $(T,k)=(100,3)$ and $(T,k)=(50,2)$, respectively. Observing Figs. 7-8, it is noted that the proposed SQuaTS system achieves superior representation accuracy and that its resulting MSE is not larger than $10^{-4}$ for quantization rates $R\geq 0.8$. For comparison, directly applying a scalar quantizer to the sparse signal is feasible only for $R\geq 1$, and its achievable MSE is only slightly less than $1$. This degraded performance of directly applying scalar quantizers stems from the fact that for the considered rates $R\in[1,2)$, this quantization mapping implements a one-bit sign quantization of $\{s[i]\}$. Since most of the samples of $\{s[i]\}$ are zero, this quantization rule induces substantial distortion.

The MSE performance of the CS-based quantization scheme improves much less dramatically with the quantization rate $R$ compared to SQuaTS. For example, for the scenario depicted in Fig. 7, the SQuaTS system achieves MSE of $2\cdot 10^{-3}$ for $R=0.5$, while the CS-based systems achieve an MSEs of $1.2\cdot 10^{-2}$ and $1.4\cdot 10^{-2}$ for the QIHT and FISTA decoders, respectively, i.e., a gap of approximately $7$ dB. However, for quantization rate of $R=1$, the corresponding MSE values are $1.7\cdot 10^{-6}$, $2\cdot 10^{-3}$, and $9\cdot 10^{-3}$, for the SQuaTS system, CS with QIHT recovery, and CS with FISTA recovery, respectively, namely, performance gaps of $30-37$ dB in MSE. For all considered scenarios, the QIHT recovery scheme, which is specifically designed for reconstructing sparse signals from compressed and quantized measurements, outperforms the FISTA method which considers general sparse recovery.

Furthermore, it is observed in Figs. 7-8 that the SQuaTS encoder combined with the reduced complexity CoMa detector proposed in Section IV outperforms CS-based quantizers as the quantization rate $R$ increases when using both a fixed $\epsilon$ as well as an optimized setting of this parameter. In particular, for the scenario whose results are depicted in Fig. 7, SQuaTS with the CoMa decoder without $\epsilon$ optimization outperforms compress-and-quantize with QIHT and FISTA recovery for rates $R>0.6$ and $R>1$, respectively. The corresponding rate thresholds for the scenario in Fig. 8 are $R>1.1$ and $R>1.4$, respectively. However, when optimizing $\epsilon$ specifically for the CoMa decoder in a fine manner, which can be carried out without substantial overhead due to its reduced computational complexity, its performance is notably improved. In fact, in high quantization rates, we observe that the optimization of $\epsilon$ allows the CoMa decoder to outperform the ML decoder, for which $\epsilon$ is merely selected out of four possible candidates.

The ability of SQuaTS with the CoMa decoder to notably outperform CS-based approaches is also demonstrated for higher resolution quantization regimes in Fig. 9. This is achieved by increasing the quantization rate to satisfy the sufficiency rate in Proposition 1 and selecting appropriate $l$. Fig. 10 numerically evaluates SQuaTS combined with fragmentation under the scenario presented in the bottom panel of Fig. 9. In particular, fragmentation is carried out here to reduce the computational burden by dividing the signal into ten groups, as discussed in Section III-D, resulting in $T=5$ and $k=1$. We observe that CoMa achieves improved performance also for various quantization rates by properly setting $l$, as demonstrated in Fig. 14. These results indicate that SQuaTS, which acquires the sparse signal in a serial manner, is capable of achieving superior performance even when using sub-optimal reduced complexity decoders, compared to CS-based methods which must observe the complete signal before it is quantized.

Fig 11 numerically evaluates the performance obtained for the scenario discussed in Section III-E, in which the value of $k$ used in designing the SQuaTS system differs from the true sparsity level. Here, we designed the system for $k=3$, while evaluating its performance when the actual number of the non-zero samples varies in the proximity of $k=3$. For lower non-zero samples case, we observe in Fig. 11 that SQuaTS still exploits the reduced sparsity level, allowing it to be translated into improved performance, despite the fact that it was designed with higher values of $k$. When the true value of $k$ is larger than that used in design, we observe some graceful degradation in the MSE accuracy of SQuaTS, indicating its ability to maintain reliable operation in such scenarios. In summary, if the number of non-zeros input samples is higher or lower than the a-priori $k$ selected, the codebook designed for $k$ can still be applied. In such cases, when the mismatch is not too large, only a minor degradation in the quantization distortion is observed. However, when this number of non-zeros inputs samples is much different than the a-priori assumption, one will need different codebook designs, and a look up table if this is used to reduce the decoding complexity with ML algorithm.

Finally, Figs. 12 and 13 show the trade-off between the quantization rate required for achieving a desired distortion level $D_{T}(l)$, computed via (7), with the parameters $k$ and $l$, for $T=50$ and $T=100$, respectively. The numerical results in Figs. 12 and 13 demonstrate how the quantization rate scales with $l$ and $k$. These observed curves settle with the characterization in Corollary 1, which noted that the rate scales logarithmically with $l$, and linearly with $k$. The results depicted in Figs. 12 and 13 demonstrate the ability of SQuaTS to exploit the signal sparsity and to benefit from small values of the support $k$, as the required quantization rate is reduced substantially as $k$ decreases To demonstrate that the performance gains of SQuaTS persist in such regimes, we numerically evaluate its performance for a scenario with $T=50$ and $k=5$. This scenario, for which the empirical performance is depicted in Fig. 14, is equivalent to the scenario of $T=100$ and $k=10$, in which fragmentation into two groups is used to reduce the decoding complexity. as suggested in Section III-D. Note that by performing fragmentation, the quantization rate is divided by the number of groups. Fig 14 also presents the performance with CoMa decoding for a low quantization rate when $\epsilon$ is optimized for this decoder. In practice, for high $k$ using ML decoder, fragmentation is needed due to the computational complexity. In the regime we simulated using CoMA, fragmentation is not needed. We observe in Fig. 14 that the expected gains of SQuaTS are indeed maintained here.

VI-B Noisy Digital Representation

Next, we carry out a set of experiments whose goal is to demonstrate how SQuaTS can be adapted to deal with noisy digital representations. To that aim, we consider the setup in which the value of the bits register ${\boldsymbol{y}}_{T}$ experiences independent random bit-flips, i.e., each bit $y_{i,T}$ is flipped from 0 to 1 with probability $0\leq q<1/2$ (positive flip), and flipped from 1 to 0 with probability $0\leq u<1$ (negative flip). This noisy model represents corruption of the codeword in the digital domain.

By making use of existing results in noisy group testing theory, we propose to increase the length of the codewords by some factor which depends on $u$ and $q$, while keeping the encoding identical. A trivial adaptation of the results in [53] to our setup, reveals that increasing the length of the codeword $b$ by a factor $\frac{1}{1-q}\cdot\frac{1}{(1-u)^{2}}$ is sufficient444While this factor is relatively loose, it is preferred here over the complex yet more precise expression that can be found in [62] due to its simple formulation.. On the side of the decoder, the ML scheme operates identically. The efficient CoMa method however, must be modified to deal with this noise in the system. While a precise discussion of this is outside of the scope of this paper, we refer the interested reader to exisiting efficient algorithms for noisy group testing such as Noisy-CoMa [63], as possible ways to tweak the CoMa decoding scheme of Subsection IV-A to account for the presence of digital noise.

Figs. 15 and 16 show the empirical MSE of the adapted scheme under the same signal settings considered in the previous subsection, in the presence of digital noise with parameters $(q=0.1,u=0.1)$ and $(q=0.4,u=0.1)$, respectively. We observe in Figs. 15- 16 that the quantization rate $R$ required to achieve a given MSE level is increased compared to the noiseless case in Figs. 7-8 – quantifying the additional bits which enable SQuaTS to be robust to digital noise. More precisely, to achieve an average MSE of $10^{-3}$ with the proposed scheme, the quantization rate must be increased from $.7$, required in the absence of digital noise, to about $1.6$ and $6$, when $q=0.1$ and $q=0.4$, respectively. As observed, this loss in performance is much more dramatic when $q$ grows, revealing that positive flips are more costly than negative flips. In either cases, the performance of the adapted SQuaTS scheme outperforms significantly the CS-based approaches, which fail to breach under the MSE of $10^{-2}$, even for large quantization rates.

VI-C Distributed Quantization

We end this section with a numerical study of the extension of SQuaTS to distributed quantization setups, detailed in Section V. To that aim, we numerically compute the achievable MSE of SQuaTS applied to $n=10$ jointly sparse signals of size $T=100$ with joint support size of $k=3$ in a single hop network. In Fig. 17 we compare the MSE of distributed SQuaTS to distributed CS [35], in which the quantized values of compressed projections are aggregated by and recovered the central decoder. We consider the cases where the decoder recovers the set of signals using the QIHT method [60] as well as FISTA [61]. While more advanced schemes combining distributed CS and vector quantization were proposed in [40], their complexity grows rapidly when $n>2$, and thus we focus on conventional distributed CS with scalar quantization.

Observing Fig. 17, we note that the proposed distributed quantization scheme notably outperforms techniques based on distributed CS. In particular, our method is shown to improve substantially the accuracy of the overall digital representation as the quantization rate increases, while distributed quantized CS is demonstrated to meet an error floor around $4\cdot 10^{-2}$ for FISTA and $9\cdot 10^{-3}$ for QIHT.

Finally, we demonstrate how the minimal quantization rate grows with the resolution $l$. To that aim, we compute in Fig. 18 the minimal rate versus $l$. The setup evaluated here consists of $n=10$ sequences of $T=90$ samples each, for both overall sparsity with $k\in\{6,12,24,36\}$ as well as structured sparsity with the same overall sparsity level and $k_{s}=3$. Observing Fig. 18, we note that structured sparsity allows to use lower quantization rates, i.e., fewer bits, to achieve the same level of distortion, due to the additional structure. We also note that the quantization rate grows slowly with $l$, indicating that a minor increase in the quantization rate can allow the scheme to utilize ADCs of much higher resolution, while maintaining the guaranteed performance. Observing Fig. 18, we note that structured sparsity allows to use lower quantization rates, i.e., fewer bits, to achieve the same level of distortion.

The results presented in this section demonstrate the potential of SQuaTS as a quantization scheme for sparse signals which is both accurate as well as suitable for implementation with conventional serial scalar ADCs, Our results also demonstrate the ability of SQuaTS to implement distributed quantization and its robustness to digital noise.

VII Conclusions

In this paper we proposed SQuaTS, a quantization system designed for representing sparse signals acquired in a sequential manner. SQuaTS combines code structures from group testing theory with the limitations and characteristics of conventional ADCs. We derived the achievable MSE of the proposed scheme in the asymptotic signal size regime and characterized its complexity. We proposed a reduced complexity decoding method for SQuaTS which trades performance for computational burden, while maintaining the sequential acquisition property of SQuaTS, and showed how SQuaTS can be extended to distributed setups. Our simulation study demonstrates the substantial performance gain of SQuaTS compared to directly applying a serial scalar ADC, as well as to CS-based methods.

To prove the Theorem 1, we first provide a reliability bound which guarantees accurate reconstruction of the quantized representation $\{Q(s[i])\}_{i=1}^{T}$ from ${{\boldsymbol{y}}}_{T}$. Then, we show that this bound results in the condition on the quantization rate stated in Theorem 1. An achievability bound on the required number of bits is stated in the following lemma:

Lemma 1**.**

If for some $\varepsilon>0$ independent of $T$ and $k$, the number of bits used for digital representation satisfies

[TABLE]

then, under the code construction of Section III, as $T\rightarrow\infty$ the average error probability to recover $\{Q(s[i])\}_{i=1}^{T}$, given by $\frac{1}{T}\sum\limits_{i=1}^{T}\Pr\left(\hat{s}_{i}\neq Q(s[i])\right)$, approaches zero exponentially.

Proof.

The Lemma follows from [44, Lemma 1], whose proof is based on [41, Lemma 2]. ∎

We note that the corresponding bound in [53, Theorem III.1], which studied group testing over a binary field, can be considered as a special case of Lemma 1 with $l=1$, i.e., using one bit quantizers. In particular, since we consider quantizers with arbitrary resolution, the bound in Lemma 1 must account for the fact that the codewords have to be selected from different bins, as $l$ can be larger than one.

Now, Lemma 1 yields a sufficient condition for the digital representation $\{\hat{s}[i]\}$ to approach the directly quantized $\{{s}[i]\}$, for which the MSE is $D_{T}(l)$ given in (6). Since the quantization rate is given by $R=\frac{b}{T}$, the condition (15) becomes

[TABLE]

proving the theorem. ∎