Multidimensional Sampling of Isotropically Bandlimited Signals

TL;DR

This paper derives a new lower bound on the reconstruction error for multidimensional isotropically bandlimited signals sampled on arbitrary lattices, linking optimal sampling lattices to sphere covering and packing properties.

Contribution

It introduces a precise lower bound for error variance in multidimensional sampling and establishes a rigorous connection between optimal sampling lattices and sphere covering/packing.

Findings

Lower bound is exact at high and low sampling rates.

Optimal lattice at low rates is dual of the best sphere-covering lattice.

Optimal lattice at high rates is dual of the best sphere-packing lattice.

Abstract

A new lower bound on the average reconstruction error variance of multidimensional sampling and reconstruction is presented. It applies to sampling on arbitrary lattices in arbitrary dimensions, assuming a stochastic process with constant, isotropically bandlimited spectrum and reconstruction by the best linear interpolator. The lower bound is exact for any lattice at sufficiently high and low sampling rates. The two threshold rates where the error variance deviates from the lower bound gives two optimality criteria for sampling lattices. It is proved that at low rates, near the first threshold, the optimal lattice is the dual of the best sphere-covering lattice, which for the first time establishes a rigorous relation between optimal sampling and optimal sphere covering. A previously known result is confirmed at high rates, near the second threshold, namely, that the optimal lattice is the dual of the best sphere-packing lattice. Numerical results quantify the performance of various lattices for sampling and support the theoretical optimality criteria.

Figures

Tables

Table 1. TABLE I: The reconstruction error variance deviates from its lower bound when r / ω 0 𝑟 subscript 𝜔 0 r/\omega_{0} is between these two thresholds, where the first depends on the dual lattice’s covering radius and the second on its packing radius.

Lattice $\Lambda \left(𝑩\right)$	$d$	${\left(vol\Omega \left(𝑨\right)\right)}^{1/d}/R\left(𝑨\right)$	${\left(vol\Omega \left(𝑨\right)\right)}^{1/d}/\rho \left(𝑨\right)$
Integers $\mathbb{Z}$	$1$	$2$	$2$
Square ${\mathbb{Z}}^2$	$2$	$\sqrt{2}=1.41$	$2$
Hexagonal $A_2$	$2$	$3^{3/4}/\sqrt{2}=1.61$	$3^{1/4}\cdot \sqrt{2}=1.86$
Cubic ${\mathbb{Z}}^3$	$3$	$2/\sqrt{3}=1.15$	$2$
BCC $A_3^{\ast }$	$3$	$2^{1/3}=1.26$	$2^{5/6}=1.78$
FCC $A_3$	$3$	$2^{5/3}/\sqrt{5}=1.42$	$2^{5/3}/\sqrt{3}=1.83$
${\mathbb{Z}}^4$	$4$	$1$	$2$
$D_4$	$4$	$2^{1/4}=1.19$	$2^{3/4}=1.68$
$A_4$	$4$	$5^{3/8}/\sqrt{2}=1.29$	$5^{3/8}=1.83$
${\mathbb{Z}}^8$	$8$	$1/\sqrt{2}=0.71$	$2$
$E_8$	$8$	$1$	$\sqrt{2}=1.41$
$A_8$	$8$	$3^{11/8}/\sqrt{20}=1.01$	$3^{7/8}/\sqrt{2}=1.85$

Equations

$$r$$

$$=(\mathrm{vol}\Omega (\mathbf{A}){)}^{1/d}.$$

$$S(\mathbf{\omega })=\{\begin{array}{ll}{S}_0,& \mathbf{\omega }\in \Delta ,\\ 0,& \mathbf{\omega }\notin \Delta ,\end{array}$$

$$\Delta \triangleq \{\mathbf{\omega }\in {\mathbb{R}}^d:\Vert \mathbf{\omega }\Vert \le {\omega }_0\}$$

$${\sigma }^2$$

$$=\frac{S_0}{(2\pi {)}^d}\mathrm{vol}\Delta ,$$

$$\mathrm{vol}\Delta =\frac{{\pi }^{d/2}{\omega }_0^d}{\Gamma \left(\frac{d}{2}+1\right)}.$$

$${{\sigma }_{\text{e}}}^2={\sigma }^2\frac{\mathrm{vol}\left(\Delta \setminus \Omega (\mathbf{A})\right)}{\mathrm{vol}\Delta }.$$

$${\sigma }_{\text{e}}={\sigma }^2-\frac{1}{(2\pi {)}^d}{\int }_{\Omega (\mathbf{A})}\frac{{\sum }_{\mathbf{\lambda }\in \Lambda (\mathbf{A})}{S}^2(\mathbf{\omega }+\mathbf{\lambda })}{{\sum }_{\mathbf{\lambda }\in \Lambda (\mathbf{A})}S(\mathbf{\omega }+\mathbf{\lambda })}d\mathbf{\omega }$$

$${\sigma }_{\text{e}}={\sigma }^2-\frac{1}{(2\pi {)}^d}{\int }_{\Omega (\mathbf{A})\cap \Delta }\frac{{\sum }_{\mathbf{\lambda }\in \Lambda (\mathbf{A})}{S}^2(\mathbf{\omega }+\mathbf{\lambda })}{{\sum }_{\mathbf{\lambda }\in \Lambda (\mathbf{A})}S(\mathbf{\omega }+\mathbf{\lambda })}d\mathbf{\omega }.$$

$${\sigma }_{\text{e}}$$

$$={\sigma }^2-\frac{S_0}{(2\pi {)}^d}\mathrm{vol}(\Omega (\mathbf{A})\cap \Delta )$$

$$={\sigma }^2-\frac{S_0}{(2\pi {)}^d}\left[\mathrm{vol}\Delta -\mathrm{vol}(\Delta \setminus \Omega (\mathbf{A}))\right]$$

$$={\sigma }^2\frac{\mathrm{vol}(\Delta \setminus \Omega (\mathbf{A}))}{\mathrm{vol}\Delta },$$

$${{\sigma }_{\text{lb}}}^2\triangleq {\sigma }^2\max \left\{0,1-\frac{\Gamma \left(\frac{d}{2}+1\right)}{{\pi }^{d/2}}{\left(\frac{r}{{\omega }_0}\right)}^d\right\}.$$

$${{\sigma }_{\text{e}}}^2\ge {\sigma }^2\max \left\{0,1-\frac{\mathrm{vol}\Omega (\mathbf{A})}{\mathrm{vol}\Delta }\right\}.$$

$${{\sigma }_{\text{e}}}^2=\{\begin{array}{ll}{\sigma }^2-\frac{r{\sigma }^2}{2{\omega }_0},& \text{if }r<2{\omega }_0,\\ 0,& \text{if }r\ge 2{\omega }_0,\end{array}$$

Full text

I Introduction

By generalizing the classical sampling theorem to multiple dimensions, it has been proven that the Nyquist rate for isotropically bandlimited signals, i.e., the lowest sampling rate that allows error-free reconstruction, is determined by the densest sphere-packing lattice [1, 2]. In three dimensions, for example, the body-centered cubic (BCC) lattice is optimal for sampling, because its dual, the face-centered cubic (FCC) lattice, is the optimal sphere packing [3, Ch. 1].

In practice, however, it is not always possible to satisfy the Nyquist criterion. In this case, the spectrum of the original signal is replicated around the points of the dual lattice in the frequency domain such that an overlapping between the replicas cannot be avoided. This overlapping causes the typical prealiasing effects, for example, in volume visualization applications [4]. Several papers [5],[6, Sec. 4.1], [7] rely on a conjecture that an optimal sphere-covering lattice ensures minimal overlap in the frequency domain. As the BCC lattice is optimal for three-dimensional sphere covering [3, Ch. 2], its dual, the FCC lattice, is supposed to be optimal for sampling isotropically bandlimited signals below the Nyquist rate in the spatial domain. However, Vad et al. [8] demonstrated that the overlap between the replicas of the spectrum depends very much on the sampling rate, which results in different optimality ranges for the BCC and FCC lattices.

In this paper, we present an exact expression for the reconstruction error variance for signals with constant, isotropically bandlimited spectrum using any sampling lattice, in any dimension, and at any rate. A lower bound on the error variance is derived, which is tight at high and low rates. Comparing the error variance with the lower bound allows us to identify optimal lattices for sampling at any rate. It is shown that the dual of the best sphere-packing lattice is optimal not only just above the Nyquist rate, but also in a range below the Nyquist rate. The dual of the best sphere-covering lattice is optimal at significantly lower sampling rates, where the reconstruction variance is high. To our best knowledge, this is the first time a mathematically precise relationship is established between optimal low-rate sampling and the sphere-covering problem.

Finally, the reconstruction error variance is numerically computed for lattices in $2$, $3$, $4$, and $8$ dimensions, supporting the theoretical results and establishing rate intervals for the optimality of various well-known lattices.

II Definitions and problem statement

Let ${\boldsymbol{B}}\in{\mathbb{R}}^{d\times n}$ and ${\boldsymbol{A}}\in{\mathbb{R}}^{d\times n}$ be matrices such that $n\geq d$ and all elements of ${\boldsymbol{A}}{\boldsymbol{B}}^{T}/(2\pi)$ are integers. We denote with $\Lambda({\boldsymbol{B}})$ the $d$-dimensional lattice generated by ${\boldsymbol{B}}$, and its dual, scaled by $2\pi$, is $\Lambda({\boldsymbol{A}})$. The Voronoi cell, packing radius, and covering radius of $\Lambda({\boldsymbol{B}})$ are denoted by $\Omega({\boldsymbol{B}})$, $\rho({\boldsymbol{B}})$, and $R({\boldsymbol{B}})$, resp.

The lattice $\Lambda({\boldsymbol{B}})$ is used for sampling a stationary stochastic process. We will refer to the process as being a function of (multidimensional) time, although the theory applies equally well to spatial or other processes. Thus, the domain where $\Lambda({\boldsymbol{B}})$ resides is time and the domain of $\Lambda({\boldsymbol{A}})$ is (angular) frequency. The lattice density $1/\mathop{\mathrm{vol}}\nolimits\Omega({\boldsymbol{B}})$, where $\mathop{\mathrm{vol}}\nolimits$ denotes volume, gives the number of samples per unit volume and can be regarded as a multidimensional analogy of the sampling rate. We therefore define the sampling rate in rad/s as

[TABLE]

An isotropically bandlimited stochastic process is characterized by a power spectral density being uniform over a $d$-dimensional sphere

[TABLE]

where

[TABLE]

and $\omega_{0}$ is the (angular) bandwidth of the process. The process variance is

[TABLE]

where we recall that the volume of a $d$-dimensional sphere is

[TABLE]

This paper deals with the problem of minimizing the average reconstruction error variance ${\sigma_{\text{e}}}^{2}$ with the best linear interpolator [1, Sec. VI], [9], for given $\omega_{0}$ and ${\boldsymbol{B}}$.

III The reconstruction error

The main results in this paper are an exact expression for the average reconstruction error variance when the signal has a constant, isotropically bandlimited spectrum (3) and a lower bound thereon, given by Theorems 1 and 2, resp.

Theorem 1

When an isotropically bandlimited process is sampled on the lattice $\Lambda({\boldsymbol{B}})$ and reconstructed using the best linear interpolator, the error variance is

[TABLE]

Proof:

The theorem can be derived as a special case of the last equation in [10, Sec. III-B]. More precisely, it follows from [10, Eq. (13)] that

[TABLE]

where a value of “0/0” should be interpreted as 0. Consider first any pair of points ${\boldsymbol{\omega}}\in\Omega({\boldsymbol{A}})\setminus\Delta$ and ${\boldsymbol{\lambda}}\in\Lambda({\boldsymbol{A}})$. Their sum satisfies $\|{\boldsymbol{\omega}}+{\boldsymbol{\lambda}}\|\geq\|{\boldsymbol{\omega}}\|>\omega_{0}$, where the first inequality follows from the definition of $\Omega({\boldsymbol{A}})$ and the second from the definition (4) of $\Delta$. Thus, by (3)–(4), $S({\boldsymbol{\omega}}+{\boldsymbol{\lambda}})=0$ and the numerator and denominator of (9) are both [math]. It follows that points ${\boldsymbol{\omega}}\in\Omega({\boldsymbol{A}})\setminus\Delta$ do not contribute to the integral, and (9) can therefore be rewritten as

[TABLE]

Since $S({\boldsymbol{\omega}})$ is uniform, $S^{2}({\boldsymbol{\omega}})=S_{0}S({\boldsymbol{\omega}})$ for all ${\boldsymbol{\omega}}$ and

[TABLE]

where the last step follows from (6). ∎

In the next section, we will evaluate (8) as a function of the sampling rate for different lattices, which in general involves $d$-dimensional numerical integration. A closed-form lower bound, which depends only on the normalized sampling rate $r/\omega_{0}$, can be derived as follows.

Theorem 2

For any lattice, the reconstruction error variance satisfies ${\sigma_{\text{e}}}^{2}\geq{\sigma_{\text{lb}}}^{2}$, where

[TABLE]

Further, ${\sigma_{\text{e}}}^{2}={\sigma_{\text{lb}}}^{2}$ if and only if $\omega_{0}\leq\rho({\boldsymbol{A}})$ or $\omega_{0}\geq R({\boldsymbol{A}})$.

Proof:

Bounding the numerator of (8) using $\mathop{\mathrm{vol}}\nolimits(\Delta\setminus\Omega({\boldsymbol{A}}))\geq 0$ and $\mathop{\mathrm{vol}}\nolimits(\Delta\setminus\Omega({\boldsymbol{A}}))\geq\mathop{\mathrm{vol}}\nolimits\Delta-\mathop{\mathrm{vol}}\nolimits\Omega({\boldsymbol{A}})$ yields

[TABLE]

The right-hand side of (16) can be evaluated using (7) and (2), which yields (15). To establish the “if and only if” conditions for ${\sigma_{\text{e}}}^{2}={\sigma_{\text{lb}}}^{2}$, we observe that $\omega_{0}\leq\rho({\boldsymbol{A}})\Leftrightarrow\Delta\subseteq\Omega({\boldsymbol{A}})\Leftrightarrow\Delta\setminus\Omega({\boldsymbol{A}})=\varnothing$ and $\omega_{0}\geq R({\boldsymbol{A}})\Leftrightarrow\Delta\supseteq\Omega({\boldsymbol{A}})\Leftrightarrow\mathop{\mathrm{vol}}\nolimits(\Delta\setminus\Omega({\boldsymbol{A}}))=\mathop{\mathrm{vol}}\nolimits\Delta-\mathop{\mathrm{vol}}\nolimits\Omega({\boldsymbol{A}})$. ∎

Example 1

As a sanity check of Theorem 1, we consider the one-dimensional case. The spectrum is flat in $\Delta=[-\omega_{0},\omega_{0}]$ and the sampling instants are $\Lambda({\boldsymbol{B}})=\{\ldots,0,2\pi/r,4\pi/r,\ldots\}$. The dual lattice is $\Lambda({\boldsymbol{A}})=\{\ldots,0,r,2r,\ldots\}$ with $\Omega({\boldsymbol{A}})=[-r/2,r/2]$. It is easily verified that $\mathop{\mathrm{vol}}\nolimits(\Delta\setminus\Omega({\boldsymbol{A}}))=2\omega_{0}-r$ if $r<2\omega_{0}$ and 0 otherwise. Hence, Theorem 1 yields

[TABLE]

which is as expected from the standard (one-dimensional) sampling theorem. The lower bound in Theorem 2 is tight everywhere in the one-dimensional case, since $\rho({\boldsymbol{A}})=R({\boldsymbol{A}})=r/2$.

Theorem 2 explains why multidimensional sampling of isotropically bandlimited processes enables perfect reconstruction also for some rates below the standard (one-dimensional) Nyquist rate. The reconstruction error is zero whenever $\omega_{0}\leq\rho({\boldsymbol{A}})$, in other words when $r/\omega_{0}\geq(\mathop{\mathrm{vol}}\nolimits\Omega({\boldsymbol{A}}))^{1/d}/\rho({\boldsymbol{A}})$. This threshold rate is listed the right column of Table I for various lattices. It equals $2$ for the cubic lattice in any dimension, which agrees with the standard (one-dimensional) sampling theorem. It is lower for several other lattices in $d$ dimensions, reflecting the fact that their Voronoi cells are more spherical than the $d$-dimensional cube. The minimum threshold rate over all possible lattices in a given dimension can be regarded as the Nyquist rate in that dimension. For example, whereas $r\geq 2\omega_{0}$ is required if $d=1$, $r\geq 1.86\omega_{0}$ is sufficient if $d=2$ and only $r\geq 1.41\omega_{0}$ if $d=8$.

Finding the maximum $\rho({\boldsymbol{A}})$ for a given dimension $d$ and a given volume $\mathop{\mathrm{vol}}\nolimits({\boldsymbol{A}})$ is known as the sphere-packing problem in the lattice literature, and the optimal lattices in dimensions $2$, $3$, $4$, and $8$ are $A_{2}$, $A_{3}$, $D_{4}$, and $E_{8}$ [3, Ch. 1]. Note, however, that Table I lists the lattices $\Lambda({\boldsymbol{B}})$ used for sampling in the time domain, whereas the sphere-packing problem applies to their duals $\Lambda({\boldsymbol{A}})$ in the frequency domain. This analogy between sampling isotropically bandlimited processes in $d$ dimensions at relatively high rates (defined as rates near the Nyquist rate) was observed already in [1, 11]111Some of the results in [1] were credited to Miyakawa, whose paper [2] is unfortunately unaccessible to us..

Theorem 2 furthermore indicates the existence of a lower threshold rate, which is also included in Table I. Whenever $\omega_{0}\geq R({\boldsymbol{A}})$ or, equivalently, when $r/\omega_{0}\leq(\mathop{\mathrm{vol}}\nolimits\Omega({\boldsymbol{A}}))^{1/d}/R({\boldsymbol{A}})$, the reconstruction error variance is the minimal possible for any lattice. At these lower rates, the optimal sampling lattice is therefore the dual of the lattice with the minimum $R({\boldsymbol{A}})$. This connection between multidimensional low-rate sampling and the sphere-covering problem has not, to our best knowledge, been reported previously.

In summary, the optimal sampling lattices for signals with isotropically bandlimited spectra at high and low rates are the duals of the best sphere-packing lattice and sphere-covering lattice, resp. These results are somewhat unexpected in view of [10], where asymptotically optimal lattices for signals with isotropical, exponentially decaying spectra were derived. The optimal high-rate lattice is in both cases the dual of the best sphere-packing lattice, but at low rates, the optimal lattice for the scenario of [10] was found to be the best sphere-packing lattice. This lattice is in general different from the dual of the best sphere-covering lattice [3, Ch. 1, Tab. 1.1], which as shown above is optimal in the bandlimited case.

IV Numerical results

The reconstruction error variance according to Theorem 1 was numerically calculated for selected low-dimensional lattices. To this end, $\mathop{\mathrm{vol}}\nolimits(\Delta\setminus\Omega({\boldsymbol{A}}))$ needed to be estimated for bandwidths $\rho({\boldsymbol{A}})\leq\omega_{0}\leq R({\boldsymbol{A}})$. We are particularly interested in bandwidths near these two thresholds, where the gap to the lower bound is expected to be small. Regular Monte-Carlo integration did not give sufficient accuracy, due to the intricate geometry of multidimensional polytopes. In particular, the vertices of the Voronoi cells, which determine the covering radius $R({\boldsymbol{A}})$, resemble narrow “spikes” and account for a negligible fraction of the total volume in high dimensions [9, Sec. 13.5.3]. To address this problem, we generated random vectors uniformly on a $d$-dimensional sphere of a given radius $\omega$ and decoded these vectors using well-known lattice decoding algorithms [12] to determine the fraction of the vectors that belonged to $\Omega({\boldsymbol{A}})$. Repeating this process for an appropriately chosen sequence of $\omega$ values enabled accurate estimation of $\mathop{\mathrm{vol}}\nolimits(\Delta\setminus\Omega({\boldsymbol{A}}))$ for a given $\Lambda({\boldsymbol{A}})$, also for bandwidths $\omega_{0}$ near $\rho({\boldsymbol{A}})$ and $R({\boldsymbol{A}})$.

The results of Theorem 1 are shown in Fig. 1 as functions of the normalized sampling rate $r/\omega_{0}$. This normalization is chosen because $\Omega({\boldsymbol{A}})$ in (8) scales proportionally to $r$ and $\Delta$ scales proportionally to $\omega_{0}$. The lower bound in Theorem 2 is also shown. As expected from theory, all lattices yield zero error for high enough $r/\omega_{0}$. When $r/\omega_{0}$ is decreased below the second threshold in Table I, aliasing occurs and the error variance begins to increase. For very low $r/\omega_{0}$, below the first threshold in Table I, the lattices will eventually have the same performance again, following the lower bound exactly.

To better illustrate the range of sampling rates for which the error variance differs from the lower bound, Fig. 2 shows the normalized difference ${\sigma_{\text{e}}}^{2}-{\sigma_{\text{lb}}}^{2}$ in logarithmic scale. The theoretical thresholds are shown as dashed vertical lines and it is clearly seen that ${\sigma_{\text{e}}}^{2}\neq{\sigma_{\text{lb}}}^{2}$ only when $\rho({\boldsymbol{A}})<\omega_{0}<R({\boldsymbol{A}})$.

In dimension $2$, sampling on the hexagonal lattice is clearly superior to the square lattice for any rate. In dimension $3$, the BCC lattice $A^{*}_{3}$ is better at high rates and the FCC lattice $A_{3}$ at low rates. This reflects the well-known fact that the dual of the BCC lattice (which is FCC) is the optimal $3$-dimensional packing lattice and the dual of the FCC lattice is the optimal covering lattice. The crossover point lies at $r/\omega_{0}=1.59$. The situation is similar in dimension $4$, where $D_{4}$ is optimal at high rates and $A_{4}$ at low rates, again due to the optimality of their duals for sphere packing and covering, resp. In dimension 8, the optimal sphere covering is not known, but it is conjectured to be $A^{*}_{8}$ [3, Chs. 1, 4]. Its covering radius $R({\boldsymbol{A}})$ is however only about $1\%$ smaller than that of $E_{8}$ (see Table I), which leaves some doubts about the best lattice for low-rate sampling. The numerical results in Figs. 1–2 show that for practical purposes, $E_{8}$ should be the preferred sampling lattice at any rate, despite its marginally weaker covering radius. Finally, for any dimension $d\geq 2$, cubic sampling is significantly weaker than sampling on the optimal lattice, being more than $3$ times farther away from the lower bound for all rates.

V Conclusions

The new lower bound on the reconstruction error variance for sampling multidimensional, isotropically bandlimited signals is exact for sufficiently low and high sampling rates. From this attractive property, two optimality criteria are derived for the choice of sampling lattices. The theoretical and numerical results confirm that the optimal lattice for multidimensional sampling at high rates (near the multidimensional Nyquist rate, as defined in Sec. III) is the dual of the best sphere-packing lattice [1]. The optimal lattice at low rates turns out to be the dual of the best sphere-covering lattice, which was conjectured by Entezari et al. [5],[6, Sec. 4.1], [7] but, to our best knowledge, has never before been established mathematically.