Signal Recovery From 1-Bit Quantized Noisy Samples via Adaptive Thresholding

TL;DR

This paper introduces a novel adaptive thresholding method for signal recovery from 1-bit noisy measurements, capable of handling arbitrary noise covariance, including colored noise, and recovering both fixed and time-varying signals.

Contribution

It presents the first unified framework for 1-bit signal recovery in noisy environments using adaptive quadratic programming, accommodating arbitrary noise covariance structures.

Findings

Effective recovery of signals from 1-bit noisy samples demonstrated

Handles both white and colored noise with arbitrary covariance matrices

Capable of recovering fixed and time-varying parameters

Abstract

In this paper, we consider the problem of signal recovery from 1-bit noisy measurements. We present an efficient method to obtain an estimation of the signal of interest when the measurements are corrupted by white or colored noise. To the best of our knowledge, the proposed framework is the pioneer effort in the area of 1-bit sampling and signal recovery in providing a unified framework to deal with the presence of noise with an arbitrary covariance matrix including that of the colored noise. The proposed method is based on a constrained quadratic program (CQP) formulation utilizing an adaptive quantization thresholding approach, that further enables us to accurately recover the signal of interest from its 1-bit noisy measurements. In addition, due to the adaptive nature of the proposed method, it can recover both fixed and time-varying parameters from their quantized 1-bit samples.

Equations

$${\mathbf{z}}^{(k)}={\theta }^{(k)}\text{mathds}1+{\mathbf{v}}^{(k)},$$

$$r_i^{(k)}=\text{sgn}\left(z_i^{(k)}-{\tau }_i^{(k)}\right),$$

$${\mathbf{r}}^{(k)}=\text{sgn}\left({\mathbf{z}}^{(k)}-{\mathbf{\tau }}^{(k)}\right)=\text{sgn}\left({\theta }^{(k)}\text{mathds}1+{\mathbf{v}}^{(k)}-{\mathbf{\tau }}^{(k)}\right),$$

$${\mathbf{\Omega }}^{(k)}\left({\mathbf{z}}^{(k)}-{\mathbf{\tau }}^{(k)}\right)⪰0,$$

$${\hat{\theta }}^{(k)}=[\text{mathds}{1}^T{\mathbf{\Sigma }}^{-1}\text{mathds}1{]}^{-1}\text{mathds}{1}^T{\mathbf{\Sigma }}^{-1}{\mathbf{z}}^{(k)},$$

$$\mathcal{Q}(\mathbf{z},\theta ):=||\mathbf{z}-\text{mathds}1\theta |{|}_{{\mathbf{\Sigma }}^{-1}}^2={\left(\mathbf{z}-\text{mathds}1\theta \right)}^H{\mathbf{\Sigma }}^{-1}\left(\mathbf{z}-\text{mathds}1\theta \right),$$

$$\mathcal{Q}(\mathbf{z})=\mathcal{Q}(\mathbf{z},{\hat{\theta }}^{(k)})={\mathbf{z}}^H(\mathbf{I}-\text{mathds}1{\mathbf{\eta }}^H{)}^H{\mathbf{\Sigma }}^{-1}(\mathbf{I}-\text{mathds}1{\mathbf{\eta }}^H)\mathbf{z},$$

$$\mathbf{\eta }\triangleq \frac{{\mathbf{\Sigma }}^{-1}\text{mathds}1}{\text{mathds}{1}^T{\mathbf{\Sigma }}^{-1}\text{mathds}1}.$$

$${\hat{\mathbf{z}}}^{(k)}=$$

$$\text{subject to}{\mathbf{\Omega }}^{(k)}(\mathbf{z}-{\mathbf{\tau }}^{(k)})⪰0,$$

$$\underset{\theta }{\mathrm{argmin}}\mathcal{Q}({\hat{\mathbf{z}}}^{(k)},\theta ),$$

$$p_{\theta }\triangleq P_{r|\theta }(1|\theta )=\text{Pr}(\theta +v_i>{\tau }_i)=Q\left(\frac{{\tau }_i-\theta }{{\sigma }_v}\right),$$

$$I(\theta ;r)$$

$$=-\sum _{i=0}^1{P}_r(i){\text{log}}_2{P}_r(i)-{\int }_{\text{mathds}R}{f}_{\theta }({\theta }^{\prime })\mathcal{H}(r|\theta ={\theta }^{\prime })d{\theta }^{\prime },$$

$$\mathcal{H}(r|\theta ={\theta }^{\prime })=p_{{\theta }^{\prime }}{\text{log}}_2{p}_{{\theta }^{\prime }}+(1-p_{{\theta }^{\prime }}){\text{log}}_2(1-p_{{\theta }^{\prime }}).$$

$$P_r(i)={\int }_{\text{mathds}R}{f}_{\theta }({\theta }^{\prime })P_{r|\theta }(i|\theta ={\theta }^{\prime })d{\theta }^{\prime }.$$

$${\mathbf{\tau }}^{(k+1)}={\hat{\theta }}^{(k)}\text{mathds}1+{\mathbf{w}}_{\tau }^{(k)},$$

$$\text{NMSE}\triangleq \text{mathds}E\left\{\frac{||\mathbf{x}-\hat{\mathbf{x}}|{|}_2^2}{||\mathbf{x}|{|}_2^2}\right\}.$$

Full text

Copyright 2018 IEEE. Published in the IEEE 2018 Asilomar Conference on Signals, Systems, and Computers (Asilomar 2018), scheduled for 28-31 October 2018 in Asilomar, Pacific Grove, CA, USA. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE. Contact: Manager, Copyrights and Permissions / IEEE Service Center / 445 Hoes Lane / P.O. Box 1331 / Piscataway, NJ 08855-1331, USA. Telephone: + Intl. 908-562-3966.

I Introduction

Wireless sensor networks (WSNs) present significant potential for usage in spatially wide-scale detection and estimation. However, there exist several practical constraints such as a low power consumption, a low cost of manufacturing, and more importantly, limited computational and transmission capabilities that must be addressed in implementing such networks [1]. As a result, it is highly desirable to develop distributed estimation frameworks with which the nodes can reliably communicate with the fusion center (FC) while satisfying the power and computational constraints. Furthermore, quantization of signal of interest is a necessary first step in many digital signal processing applications such as spectrum sensing, radar, and wireless communications. Analog-to-digital converters (ADCs) play a central role in most modern digital systems as they bridge the gap between the analog world and its digital counterpart. Yet, ADCs are a key implementation bottleneck in WSNs and many other Internet-of-Things (IoT) applications due to the significant accrued power consumption and cost when they are used in large numbers.

In addition, the system bandwidth in a WSN is limited and thus is a fundamental constraint that must be considered. Hence, it is important to use a proper quantization scheme to reduce the communicated bits prior to transmission to address this limitations. Sampling at high data rates with high resolution ADCs would also dramatically increase the manufacturing cost of these electronic components. An immediate solution to such challenges is to use low-resolution, and specifically 1-bit, analog-to-digital converters (ADCs) [2, 3, 4, 5]. Therefore, the problem of recovering a signal from 1-bit measurements has attracted a great deal of interest over the past few years in a wide range of applications—see, e.g., [6, 7, 8, 9, 10, 11, 12], and the references therein. To name a few, the authors in [13, 14, 15, 16, 17, 18, 19], have investigated this problem from a classical statistical viewpoint. Particularly, 1-bit sampling and signal recovery have been extensively studied in the context of recently introduced one-bit Compressive Sensing (CS) problem as well [20, 21, 22, 23, 24, 25, 26, 27, 28]. More specifically, the task of recovering the frequency and phase of temporal and spatial sinusoidal signals utilizing only 1-bit information with fixed quantization thresholds has been extensively investigated in [16] and [17], respectively. On the other hand, the recovery of general signals with high-dimensional parameters from sign comparison information were considered in [18] and [19]. In the context of CS, it was shown that sparse signals can be accurately recovered with high probability from 1-bit data when sufficient number of measurements is obtained [21, 22]. However, most of the CS literature have only considered the case of comparing the signal of interst with zero, which makes it impossible to recover the amplitude of the signal of interest.

In this paper, we examine the most extreme case of quantization, i.e. 1-bit case, and propose an efficient signal estimation and threshold design algorithm which can perform the task of signal recovery from its 1-bit noisy measurements under the scenarios of white, colored or correlated noise. Furthermore, the proposed method can handle the task of parameter estimation for both cases of time-varying and fixed signals.

II System Model

We consider a wireless sensor network with $N$ spatially distributed single-antenna nodes each of which observing an unknown deterministic parameter $\theta\in\mathds{R}$, at the time index $k$, according to the linear observation model of $z_{i}^{(k)}=\theta^{(k)}+v_{i}^{(k)}$, where $i$ denotes the sensor index, and $v_{i}^{(k)}$ is additive zero-mean Gaussian observation noise with a Normal distribution, e.g., $v_{i}^{(k)}\sim\mathcal{N}(0,\sigma_{v}^{2})$. Hence, the observed signal at all nodes can be compactly formulated as

[TABLE]

where $\mathds{1}$ denotes the all-one vector.

In order to satisfy the inherent bandwidth and power budget constraints in WSNs, we assume in this case that each node utilizes a 1-bit quantization scheme to encode its observation into 1 bit of information which will be transmitted to the fusion center for further processing. Namely, the $i$th node applies the following quantization function on its observed data $z_{i}^{(k)}$ prior to transmission:

[TABLE]

where $\mbox{sgn}(\cdot)$ is the sign function, and $\tau_{i}^{(k)}$ denotes the quantization threshold at the $i$th node, for the time index $k$. Assuming that each node can reliably transmit 1 bit of information to the FC, the aggregated received data from all nodes at the $k$-th sampling time can be expresses as

[TABLE]

where the sign function is applied element-wise, and $\mathbf{v}^{(k)}~{}=~{}[v_{1}^{(k)},\dots,v_{N}^{(k)}]^{T}$ is the combined observation noise vector with covariance matrix $\boldsymbol{\Sigma}^{(k)}$, and $\boldsymbol{\tau}^{(k)}~{}=~{}[\tau_{1}^{(k)},\dots,\tau_{N}^{(k)}]$ is the quantization threshold vector. Next, the FC utilizes the received 1-bit information $\boldsymbol{r}^{(k)}$ to first construct an estimation of the unknown parameter $\theta^{(k)}$, and then, to further design the next quantization threshold for each node accordingly.

An important observation which we use is that given a set of quantization thresholds $\boldsymbol{\tau}^{(k)}$, the corresponding vector of 1-bit measurements $\boldsymbol{r}^{(k)}$ defined in (3) represents a limitation on the geometric location of the unquantized data $\mathbf{z}^{(k)}$. Particularly, one can capture this geometric knowledge on $\mathbf{z}^{(k)}$ through the following linear inequality:

[TABLE]

where $\boldsymbol{\Omega}^{(k)}=\text{\text{Diag}}\{\mathbf{\boldsymbol{r}^{(k)}}\}$, and $\text{Diag}\{\cdot\}$ is the diagonalization operator, and $\mathbf{0}$ denotes the all-zero vector.

III Centralized 1-Bit Signal Recovery via Quadratic Programming

We lay the ground for our 1-bit statistical and inference model using the weighted least square (WLS) method. Let $\boldsymbol{\Sigma}=\mathds{E}\{\mathbf{v}^{(k)}\mathbf{v}^{(k)^{H}}\}$ denote the covariance matrix of the noise vector $\mathbf{v}^{(k)}$. Note that if the unquantized information vector $\mathbf{z}^{(k)}$ was available at the FC, then the maximum likelihood (ML) estimation of the unknown parameter $\theta^{(k)}$ given $\mathbf{z}^{(k)}$ can be expressed as:

[TABLE]

where $\hat{\theta}^{(k)}$ denotes the ML estimate of the unknown parameter according to the vector of observations $\mathbf{z}^{(k)}$. Furthermore, it is well-known that the variance of the ML estimation in (5) is given by $\text{Var}(\hat{\theta}^{(k)})=[\mathds{1}^{T}\boldsymbol{\Sigma}^{-1}\mathds{1}]^{-1}$. Alternatively, one can obtain the maximum-likelihood estimator of the unknown parameter by minimizing the following WLS criterion,

[TABLE]

where $\mathcal{Q}(\mathbf{z},\theta)$ is our cost function to be minimized over the parameters $(\mathbf{z},\theta)$. A natural approach to obtain an estimate of $\theta^{(k)}$ using the 1-bit quantization vector $\boldsymbol{r}^{(k)}$, is to use an alternating optimization approach and further exploit the limitation on the geometric location of $\mathbf{z}^{(k)}$ imposed by (4); in other words, to first obtain an estimate of $\mathbf{z}^{(k)}$ by fixing the variable $\theta^{(k)}$ and then to recover the unknown parameter $\theta^{(k)}$ using (5), given by $\text{\emph{argmin}}_{\theta}\mathcal{Q}(\mathbf{z},\theta)$. Interestingly, for a fixed parameter $\mathbf{z}$, the optimal $\theta^{*}$ that minimizes $\mathcal{Q}(\mathbf{z},\theta)$ coincides with that of the MLE of $\theta^{(k)}$ given in (5).

We can further substitute the optimal $\hat{\theta}^{(k)}$ of (5) into (6) to simplify the objective function $\mathcal{Q}(\mathbf{z},\theta)$ in terms of the parameter $\mathbf{z}$, viz.

[TABLE]

where we define $\boldsymbol{\eta}$ as,

[TABLE]

Consequently, one can cast the problem of recovering the unquantized vector $\mathbf{z}$ from 1-bit noisy measurements $\boldsymbol{r}^{(k)}$ as the following constrained quadratic program (CQP):

[TABLE]

where the inequality in (10) is applied element-wise (equivalent to $N$ scalar inequality constraints). Note that the constraint (10) ensures the consistency between the received 1-bit quantized data $\boldsymbol{r}^{(k)}$ (as incorporated in $\boldsymbol{\Omega}^{(k)}$) and the solution $\hat{\mathbf{z}}^{(k)}$. Moreover, note that the matrix $\mathbf{M}~{}=~{}(\mathbf{I}~{}-~{}\mathds{1}\boldsymbol{\eta}^{H})^{H}\boldsymbol{\Sigma}^{-1}(\mathbf{I}~{}-~{}\mathds{1}\boldsymbol{\eta}^{H})$ at the core of this CQP is positive semi-definite. Therefore, the CQP problem in (9) is convex and can be solved efficiently using standard numerical methods (e.g., the interior point method [29]).

Having obtained $\hat{\mathbf{z}}^{(k)}$ via solving (9), we can then easily estimate the unknown parameter $\theta^{(k)}$ by solving the program,

[TABLE]

whose closed-form solution $\hat{\theta}^{(k)}$ is given in (5).

IV Quantization Threshold Design

So far, we discussed our inference framework to estimate an unknown deterministic signal from its 1-bit noisy measurements and the corresponding quantization thresholds. In order to further facilitate the estimation process and to restore the exact signal model (both amplitude and phase), it is of essence to design a proper adaptive quantization thresholding scheme. In this section, we devise a stochastic adaptive thresholding method for our estimation algorithm, which thanks to its adaptive nature, empowers the proposed inference framework to accurately recover time-varying signals as well. We first investigate the performance of a 1-bit quantizer of the form (2) in the presence of additive noise from an information theoretic viewpoint, and further show that the presence of noise is indeed improving the performance of a 1-bit quantization scheme. We then propose our threshold design strategy accordingly.

We first consider the case that the unknown parameter is a random variable with a known distribution, and that the observation noise $v\sim\mathcal{N}(0,\sigma_{v}^{2})$. Clearly, each 1-bit sample $r=\text{sgn}(\theta+v-\tau)$, (at a given time index and quantization threshold value $\tau$) can be seen as a random variable that follows a Bernoulli distribution $\mathcal{B}(p_{\theta})$, whose parameter $p_{\theta}$ is given by

[TABLE]

where $P_{r|\theta}$ represents the conditional probability of receiving $r=1$ given the parameter $\theta$, and $Q(\cdot)$ denotes the standard Q-function. The mutual information [30] between the unknown parameter $\theta$ and the obtained 1-bit sample $r_{i}$ can be expressed as

[TABLE]

where $P_{r}(i)$ for $i\in\{0,1\}$, is the probability mass function (pmf) of the discrete Bernoulli random variable $r$, and $f_{\theta}$ is the probability density function (pdf) of the parameter of interest $\theta$, and $\mathcal{H}(\cdot)$ denotes the entropy function of the argument random variable. Moreover, the conditional entropy $\mathcal{H}(r|\theta=\theta^{\prime})$ in (13) can be further simplified in terms of the noise distribution and according to (12) as follows:

[TABLE]

Moreover, the pmf of the Bernoulli random variable $r$ can be recast as

[TABLE]

Eventually, one can easily calculate the mutual information of the form (13) between the unknown parameter and the observed 1-bit samples by utilizing (12)-(15).

Figure 1 (a) illustrates the mutual information $I(\theta;r)$ versus the Gaussian noise standard deviation $\sigma_{v}$ for two cases: (1) $\theta$ is uniformly distributed and (2) $\theta$ follows a Normal distribution. Surprisingly, as the noise power increases, the mutual information shows a non-monotonically behavior in the presence of noise in both cases. Namely, the mutual information between the input signal, and the output of the 1-bit quantizer $r$, first achieves a global maximum and then dampens. This implies that a moderate amount of noise can indeed provide signal processing benefits in the case of 1-bit quantization schemes. Henceforth, this motivates us to employ a stochastic threshold design method in which we artificially induce noise to the system through the thresholds $\boldsymbol{\tau}^{(k)}$, to exploit this non-monotonic behavior of mutual information $I(\theta;r)$. In addition, we further consider the current knowledge of the unknown parameter at each time index to tune the quantization thresholds of each node for the next observation period, which enables us to achieve an even more accurate estimate of the parameter of interest.

— Stochastic Threshold Design: Herein, we propose our adaptive threshold design strategy for the task of 1-bit signal recovery. It can be shown that the optimal threshold given the Bernoulli observations $\boldsymbol{r}^{(k)}$ is indeed equal to the unknown parameter; i.e., $\tau^{(k)}_{opt}=\theta^{(k)}$ at each time index. Nevertheless, we cannot use the optimal threshold in that it is a function of the unknown parameter at each sampling period, and therefore, cannot be used in practice. Hence, a natural approach to determine the next quantization threshold is to exploit our current knowledge of the unknown parameter to set the next quantization thresholds. Namely, the fusion center should first obtain an estimate of the parameter based on the received quantized information $\boldsymbol{r}^{(k)}$ from the nodes, and then set the next quantization threshold for each node according to the obtained estimate of the unknown parameter (note that $\hat{\mathbf{\theta}}^{(k)}$ is our best estimate of the real value of the unknown parameter $\theta^{(k)}$, at the time index $k$). With the current estimate of the unknown parameter at hand, the FC samples the next quantization threshold for each node from a Normal distribution with mean $\hat{\theta}^{(k)}$, and variance $\sigma_{\tau}^{2}$. In other words, the fusion center adds $N$ realizations of a zero-mean random variable $w_{\tau}\sim\mathcal{N}(0,\sigma_{\tau}^{2})$ to the current estimate of the unknown parameter obtained from (11) in order to choose the next thresholds. Namely, after obtaining $\hat{\theta}^{(k)}$ at the $k$th cycle, the FC chooses the next quantization thresholds $\boldsymbol{\tau}^{(k+1)}$ according to the following model,

[TABLE]

where $\mathbf{w}_{\tau}^{(k)}=[w_{1}^{(k)},\dots,w_{N}^{(k)}]^{T}$ are $N$ independent samples drawn from the normal distribution of $\mathcal{N}(0,\sigma_{\tau}^{2})$. Note that, by using a stochastic threshold design strategy, we are able to introduce an artificial noise whose variance can be controlled in such a way to not only maximize the mutual information but also incorporate the current estimate of the unknown parameter in the design. Furthermore, the variance of the random variable $w_{\tau}$ can be chosen according to the observation noise variance to maximize the mutual information. More generally, we can model $\boldsymbol{\tau}^{(k+1)}$ as a multivariate Gaussian random vector with mean vector $\boldsymbol{\mu}^{(k+1)}=\theta^{(k)}\mathds{1}$, and covariance matrix $\boldsymbol{\Sigma}_{\tau}$, e.g., $\boldsymbol{\tau}^{(k+1)}\sim\mathcal{N}(\boldsymbol{\mu}^{(k+1)},\boldsymbol{\Sigma}_{\tau})$.

The proposed signal recovery and threshold design method is summarized in Table 1.

V Numerical Results

In this section, we evaluate the performance of the two proposed algorithms for the task of 1-bit signal recovery. We define the normalized mean square error (NMSE) of an estimate $\hat{\mathbf{x}}$ of a signal $\mathbf{x}$ as

[TABLE]

Each data point presented in the numerical results is averaged over $100$ independent samples and realizations of the problem parameters. In the following, we analyze the performance of our proposed algorithms in different scenarios. In addition, we compare the performance of our method in the presence of white Gaussian noise (WGN) with the modified mean estimator (MME) proposed in [31]. It must be noted that our algorithm can handle the task of parameter estimation in the presence of both white or colored (correlated) noise. However, the MME method of [31] can only handle the scenario of white Gaussian noise.

Fig. 1(b) demonstrates the normalized mean square error (NMSE) vs. the total number of nodes $N$, for the white Gaussian noise (WGN) scenario when $\theta^{(k)}=10sin(2\pi fk)$ where $f=200\mbox{Hz}$. It can be seen that in the presence of WGN, our proposed method significantly outperforms the MME method of [31] and attains a very high accuracy for estimating the unknown parameter.

In Fig. 2 we consider the presence of (correlated) colored Gaussian noise at the time of observation, where each node observes a time-varying parameter of the form $\theta^{(k)}=10sin(2\pi fk)$ with $f=50\mbox{Hz}$. Particularly, Fig. 2(a) illustrates the performance of our proposed method versus the total power of colored noise $P_{tot}=\text{Tr}(\boldsymbol{\Sigma})$, and it can be seen that in the presence of colored noise, the proposed method can accurately recover the unknown time-varying parameter. On the other hand, Fig. 2(b) demonstrates the NMSE vs. the total number of nodes assuming that $P_{tot}=5$. It can be seen that as the number of nodes $N$ (1-bit information) increases, the accuracy of our proposed method improves, and in most of the cases, the NMSE attains values that are virtually zero (note that we used logarithmic scale in our illustrations).

VI Conclusion

In this paper, we assumed the most extreme case of quantization, i.e. the 1-bit case, and proposed an efficient signal recovery and threshold design method which can perform the task of signal recovery from its 1-bit noisy measurements under both scenarios of the presence of white and colored Gaussian noise. Moreover, the proposed algorithms can accurately recover fixed as well as time-varying unknown parameters (e.g., a sinusoidal signal).