Sampling and Reconstruction in Distinct Subspaces Using Oblique Projections

TL;DR

This paper investigates optimal reconstruction operators in Hilbert spaces that balance stability and accuracy, with applications to nonuniform Fourier sampling, by analyzing oblique projections and their properties.

Contribution

It introduces methods to construct and interpolate between reconstruction operators optimized for stability and quasi-optimality in subspace sampling.

Findings

Reconstruction operator with minimal operator norm is most stable.

Operator with minimal quasi-optimality constant offers robustness against systematic errors.

Continuous variation between operators allows trading stability for accuracy.

Abstract

We study reconstruction operators on a Hilbert space that are exact on a given reconstruction subspace. Among those the reconstruction operator obtained by the least squares fit has the smallest operator norm, and therefore is most stable with respect to noisy measurements. We then construct the operator with the smallest possible quasi-optimality constant, which is the most stable with respect to a systematic error appearing before the sampling process (model uncertainty). We describe how to vary continuously between the two reconstruction methods, so that we can trade stability for quasi-optimality. As an application we study the reconstruction of a compactly supported function from nonuniform samples of its Fourier transform.

Tables

Table 1. (a) m = 10 𝑚 10 m=10 , φ 𝒯 , 𝒰 = 1.7734 ​ e − 08 ± 1.3455 ​ e − 08 subscript 𝜑 𝒯 𝒰 plus-or-minus 1.7734 𝑒 08 1.3455 𝑒 08 \varphi_{\mathcal{T},\mathcal{U}}=1.7734e-08\pm 1.3455e-08

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB	${\Vert Q_{\lambda }\Vert }_{\mathrm{op}}$	$\mu \left(Q_{\lambda }\right)$	$\kappa \left({\Sigma }_{\lambda }^{-\frac{1}{2}}{U}^{\ast }T\right)$
$Q_0$	$0.067\pm 0.000$	$0.088\pm 0.043$	$0.184\pm 0.165$	$8.288\pm 10.884$	$1.000\pm 0.000$	$1.000\pm 0.000$
$Q_{0.1}$	$0.068\pm 0.005$	$0.075\pm 0.011$	$0.118\pm 0.045$	$4.024\pm 3.907$	$1.078\pm 0.221$	$1.747\pm 1.233$
$Q_{0.2}$	$0.069\pm 0.010$	$0.076\pm 0.013$	$0.118\pm 0.045$	$3.936\pm 3.807$	$1.160\pm 0.382$	$2.256\pm 1.842$
$Q_{0.3}$	$0.071\pm 0.014$	$0.077\pm 0.016$	$0.118\pm 0.046$	$3.880\pm 3.744$	$1.245\pm 0.525$	$2.724\pm 2.369$
$Q_{0.4}$	$0.073\pm 0.019$	$0.079\pm 0.020$	$0.119\pm 0.048$	$3.838\pm 3.699$	$1.336\pm 0.662$	$3.190\pm 2.881$
$Q_{0.5}$	$0.076\pm 0.024$	$0.081\pm 0.024$	$0.120\pm 0.050$	$3.804\pm 3.662$	$1.434\pm 0.802$	$3.682\pm 3.412$
$Q_{0.6}$	$0.079\pm 0.028$	$0.084\pm 0.028$	$0.121\pm 0.053$	$3.776\pm 3.632$	$1.544\pm 0.950$	$4.224\pm 3.996$
$Q_{0.7}$	$0.082\pm 0.033$	$0.086\pm 0.033$	$0.123\pm 0.056$	$3.752\pm 3.608$	$1.670\pm 1.112$	$4.848\pm 4.672$
$Q_{0.8}$	$0.085\pm 0.039$	$0.089\pm 0.038$	$0.125\pm 0.060$	$3.733\pm 3.587$	$1.820\pm 1.298$	$5.603\pm 5.503$
$Q_{0.9}$	$0.089\pm 0.045$	$0.093\pm 0.044$	$0.127\pm 0.065$	$3.718\pm 3.572$	$2.011\pm 1.523$	$6.581\pm 6.616$
$Q_1$	$0.094\pm 0.052$	$0.098\pm 0.051$	$0.131\pm 0.071$	$3.712\pm 3.566$	$2.276\pm 1.824$	$7.982\pm 8.319$

Table 2. (a) m = 10 𝑚 10 m=10 , φ 𝒯 , 𝒰 = 1.7734 ​ e − 08 ± 1.3455 ​ e − 08 subscript 𝜑 𝒯 𝒰 plus-or-minus 1.7734 𝑒 08 1.3455 𝑒 08 \varphi_{\mathcal{T},\mathcal{U}}=1.7734e-08\pm 1.3455e-08

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB	${\Vert Q_{\lambda }\Vert }_{\mathrm{op}}$	$\mu \left(Q_{\lambda }\right)$	$\kappa \left({\Sigma }_{\lambda }^{-\frac{1}{2}}{U}^{\ast }T\right)$
$Q_0$	$0.067\pm 0.000$	$0.088\pm 0.043$	$0.184\pm 0.165$	$8.288\pm 10.884$	$1.000\pm 0.000$	$1.000\pm 0.000$
$Q_{0.1}$	$0.068\pm 0.005$	$0.075\pm 0.011$	$0.118\pm 0.045$	$4.024\pm 3.907$	$1.078\pm 0.221$	$1.747\pm 1.233$
$Q_{0.2}$	$0.069\pm 0.010$	$0.076\pm 0.013$	$0.118\pm 0.045$	$3.936\pm 3.807$	$1.160\pm 0.382$	$2.256\pm 1.842$
$Q_{0.3}$	$0.071\pm 0.014$	$0.077\pm 0.016$	$0.118\pm 0.046$	$3.880\pm 3.744$	$1.245\pm 0.525$	$2.724\pm 2.369$
$Q_{0.4}$	$0.073\pm 0.019$	$0.079\pm 0.020$	$0.119\pm 0.048$	$3.838\pm 3.699$	$1.336\pm 0.662$	$3.190\pm 2.881$
$Q_{0.5}$	$0.076\pm 0.024$	$0.081\pm 0.024$	$0.120\pm 0.050$	$3.804\pm 3.662$	$1.434\pm 0.802$	$3.682\pm 3.412$
$Q_{0.6}$	$0.079\pm 0.028$	$0.084\pm 0.028$	$0.121\pm 0.053$	$3.776\pm 3.632$	$1.544\pm 0.950$	$4.224\pm 3.996$
$Q_{0.7}$	$0.082\pm 0.033$	$0.086\pm 0.033$	$0.123\pm 0.056$	$3.752\pm 3.608$	$1.670\pm 1.112$	$4.848\pm 4.672$
$Q_{0.8}$	$0.085\pm 0.039$	$0.089\pm 0.038$	$0.125\pm 0.060$	$3.733\pm 3.587$	$1.820\pm 1.298$	$5.603\pm 5.503$
$Q_{0.9}$	$0.089\pm 0.045$	$0.093\pm 0.044$	$0.127\pm 0.065$	$3.718\pm 3.572$	$2.011\pm 1.523$	$6.581\pm 6.616$
$Q_1$	$0.094\pm 0.052$	$0.098\pm 0.051$	$0.131\pm 0.071$	$3.712\pm 3.566$	$2.276\pm 1.824$	$7.982\pm 8.319$

Table 3. (b) m = 20 𝑚 20 m=20 , φ 𝒯 , 𝒰 = 2.0805 ​ e − 08 ± 1.2298 ​ e − 08 subscript 𝜑 𝒯 𝒰 plus-or-minus 2.0805 𝑒 08 1.2298 𝑒 08 \varphi_{\mathcal{T},\mathcal{U}}=2.0805e-08\pm 1.2298e-08

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB	${\Vert Q_{\lambda }\Vert }_{\mathrm{op}}$	$\mu \left(Q_{\lambda }\right)$	$\kappa \left({\Sigma }_{\lambda }^{-\frac{1}{2}}{U}^{\ast }T\right)$
$Q_0$	$0.048\pm 0.000$	$0.096\pm 0.060$	$0.259\pm 0.201$	$11.927\pm 12.724$	$1.000\pm 0.000$	$1.000\pm 0.000$
$Q_{0.1}$	$0.049\pm 0.003$	$0.066\pm 0.017$	$0.147\pm 0.056$	$5.330\pm 4.392$	$1.097\pm 0.201$	$2.103\pm 1.391$
$Q_{0.2}$	$0.050\pm 0.006$	$0.067\pm 0.018$	$0.146\pm 0.055$	$5.249\pm 4.320$	$1.194\pm 0.349$	$2.833\pm 2.088$
$Q_{0.3}$	$0.052\pm 0.009$	$0.068\pm 0.019$	$0.146\pm 0.055$	$5.199\pm 4.277$	$1.293\pm 0.480$	$3.499\pm 2.694$
$Q_{0.4}$	$0.054\pm 0.012$	$0.069\pm 0.021$	$0.146\pm 0.055$	$5.162\pm 4.245$	$1.398\pm 0.607$	$4.164\pm 3.286$
$Q_{0.5}$	$0.056\pm 0.015$	$0.070\pm 0.022$	$0.146\pm 0.056$	$5.133\pm 4.221$	$1.510\pm 0.735$	$4.867\pm 3.906$
$Q_{0.6}$	$0.058\pm 0.018$	$0.072\pm 0.024$	$0.147\pm 0.057$	$5.110\pm 4.201$	$1.634\pm 0.869$	$5.644\pm 4.591$
$Q_{0.7}$	$0.061\pm 0.021$	$0.074\pm 0.026$	$0.148\pm 0.057$	$5.090\pm 4.184$	$1.776\pm 1.014$	$6.546\pm 5.387$
$Q_{0.8}$	$0.064\pm 0.025$	$0.076\pm 0.029$	$0.148\pm 0.059$	$5.074\pm 4.171$	$1.945\pm 1.180$	$7.650\pm 6.372$
$Q_{0.9}$	$0.067\pm 0.028$	$0.079\pm 0.031$	$0.150\pm 0.060$	$5.062\pm 4.162$	$2.157\pm 1.378$	$9.102\pm 7.695$
$Q_1$	$0.071\pm 0.032$	$0.082\pm 0.035$	$0.151\pm 0.063$	$5.057\pm 4.158$	$2.451\pm 1.638$	$11.233\pm 9.726$

Table 4. (c) m = 30 𝑚 30 m=30 , φ 𝒯 , 𝒰 = 3.5152 ​ e − 08 ± 7.3768 ​ e − 09 subscript 𝜑 𝒯 𝒰 plus-or-minus 3.5152 𝑒 08 7.3768 𝑒 09 \varphi_{\mathcal{T},\mathcal{U}}=3.5152e-08\pm 7.3768e-09

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB	${\Vert Q_{\lambda }\Vert }_{\mathrm{op}}$	$\mu \left(Q_{\lambda }\right)$	$\kappa \left({\Sigma }_{\lambda }^{-\frac{1}{2}}{U}^{\ast }T\right)$
$Q_0$	$0.039\pm 0.000$	$0.122\pm 0.084$	$0.373\pm 0.292$	$17.371\pm 17.536$	$1.000\pm 0.000$	$1.000\pm 0.000$
$Q_{0.1}$	$0.040\pm 0.003$	$0.069\pm 0.021$	$0.180\pm 0.066$	$6.452\pm 4.861$	$1.129\pm 0.213$	$2.431\pm 1.552$
$Q_{0.2}$	$0.042\pm 0.006$	$0.069\pm 0.021$	$0.180\pm 0.066$	$6.369\pm 4.793$	$1.246\pm 0.373$	$3.348\pm 2.324$
$Q_{0.3}$	$0.044\pm 0.008$	$0.070\pm 0.022$	$0.178\pm 0.066$	$6.318\pm 4.752$	$1.363\pm 0.514$	$4.180\pm 2.998$
$Q_{0.4}$	$0.046\pm 0.011$	$0.071\pm 0.023$	$0.178\pm 0.066$	$6.282\pm 4.723$	$1.482\pm 0.649$	$5.008\pm 3.658$
$Q_{0.5}$	$0.048\pm 0.013$	$0.072\pm 0.023$	$0.178\pm 0.066$	$6.254\pm 4.701$	$1.610\pm 0.785$	$5.885\pm 4.350$
$Q_{0.6}$	$0.050\pm 0.015$	$0.073\pm 0.024$	$0.179\pm 0.066$	$6.232\pm 4.683$	$1.745\pm 0.928$	$6.856\pm 5.115$
$Q_{0.7}$	$0.052\pm 0.017$	$0.074\pm 0.025$	$0.179\pm 0.067$	$6.213\pm 4.667$	$1.900\pm 1.085$	$7.987\pm 6.007$
$Q_{0.8}$	$0.054\pm 0.020$	$0.075\pm 0.027$	$0.179\pm 0.068$	$6.198\pm 4.655$	$2.082\pm 1.265$	$9.378\pm 7.113$
$Q_{0.9}$	$0.056\pm 0.022$	$0.077\pm 0.028$	$0.180\pm 0.068$	$6.187\pm 4.646$	$2.310\pm 1.483$	$11.223\pm 8.600$
$Q_1$	$0.059\pm 0.024$	$0.079\pm 0.030$	$0.181\pm 0.069$	$6.183\pm 4.642$	$2.621\pm 1.775$	$13.966\pm 10.885$

Table 5. (d) m = 40 𝑚 40 m=40 , φ 𝒯 , 𝒰 = 5.4254 ​ e − 08 ± 1.4382 ​ e − 08 subscript 𝜑 𝒯 𝒰 plus-or-minus 5.4254 𝑒 08 1.4382 𝑒 08 \varphi_{\mathcal{T},\mathcal{U}}=5.4254e-08\pm 1.4382e-08

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB	$\Vert Q_{\lambda }\Vert$	$\mu \left(Q_{\lambda }\right)$	$\kappa \left({\Sigma }_{\lambda }^{-\frac{1}{2}}{U}^{\ast }T\right)$
$Q_0$	$0.034\pm 0.000$	$0.870\pm 3.781$	$2.550\pm 9.917$	$180.730\pm 790.710$	$1.000\pm 1.000$	$1.000\pm 1.000$
$Q_{0.1}$	$0.036\pm 0.004$	$0.075\pm 0.025$	$0.209\pm 0.076$	$7.322\pm 5.272$	$1.185\pm 0.298$	$2.701\pm 1.705$
$Q_{0.2}$	$0.038\pm 0.007$	$0.075\pm 0.025$	$0.208\pm 0.073$	$7.229\pm 5.194$	$1.327\pm 0.469$	$3.758\pm 2.537$
$Q_{0.3}$	$0.040\pm 0.009$	$0.076\pm 0.026$	$0.207\pm 0.072$	$7.179\pm 5.153$	$1.456\pm 0.610$	$4.713\pm 3.263$
$Q_{0.4}$	$0.041\pm 0.011$	$0.076\pm 0.026$	$0.206\pm 0.072$	$7.146\pm 5.127$	$1.580\pm 0.739$	$5.667\pm 3.978$
$Q_{0.5}$	$0.043\pm 0.012$	$0.077\pm 0.027$	$0.206\pm 0.071$	$7.121\pm 5.107$	$1.707\pm 0.865$	$6.677\pm 4.729$
$Q_{0.6}$	$0.044\pm 0.014$	$0.077\pm 0.027$	$0.206\pm 0.071$	$7.102\pm 5.093$	$1.840\pm 0.993$	$7.800\pm 5.562$
$Q_{0.7}$	$0.046\pm 0.015$	$0.078\pm 0.028$	$0.206\pm 0.071$	$7.088\pm 5.081$	$1.983\pm 1.129$	$9.110\pm 6.535$
$Q_{0.8}$	$0.048\pm 0.017$	$0.079\pm 0.028$	$0.206\pm 0.071$	$7.076\pm 5.073$	$2.147\pm 1.281$	$10.729\pm 7.744$
$Q_{0.9}$	$0.049\pm 0.018$	$0.080\pm 0.029$	$0.206\pm 0.071$	$7.068\pm 5.067$	$2.344\pm 1.460$	$12.888\pm 9.374$
$Q_1$	$0.051\pm 0.019$	$0.081\pm 0.029$	$0.207\pm 0.072$	$7.065\pm 5.065$	$2.601\pm 1.689$	$16.127\pm 11.882$

Table 6. (a) m = 10 𝑚 10 m=10

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB
$Q_0$	$0.067\pm 0.000$	$0.075\pm 0.003$	$0.127\pm 0.020$
$Q_{0.1}$	$0.068\pm 0.003$	$0.076\pm 0.004$	$0.128\pm 0.020$
$Q_{0.2}$	$0.069\pm 0.007$	$0.077\pm 0.007$	$0.129\pm 0.021$
$Q_{0.3}$	$0.071\pm 0.011$	$0.079\pm 0.011$	$0.131\pm 0.022$
$Q_{0.4}$	$0.073\pm 0.015$	$0.081\pm 0.015$	$0.132\pm 0.024$
$Q_{0.5}$	$0.075\pm 0.019$	$0.083\pm 0.018$	$0.134\pm 0.027$
$Q_{0.6}$	$0.078\pm 0.023$	$0.086\pm 0.022$	$0.136\pm 0.030$
$Q_{0.7}$	$0.081\pm 0.027$	$0.088\pm 0.027$	$0.139\pm 0.034$
$Q_{0.8}$	$0.084\pm 0.032$	$0.092\pm 0.031$	$0.142\pm 0.038$
$Q_{0.9}$	$0.088\pm 0.037$	$0.095\pm 0.037$	$0.146\pm 0.044$
$Q_1$	$0.093\pm 0.043$	$0.100\pm 0.043$	$0.151\pm 0.051$

Table 7. (a) m = 10 𝑚 10 m=10

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB
$Q_0$	$0.067\pm 0.000$	$0.075\pm 0.003$	$0.127\pm 0.020$
$Q_{0.1}$	$0.068\pm 0.003$	$0.076\pm 0.004$	$0.128\pm 0.020$
$Q_{0.2}$	$0.069\pm 0.007$	$0.077\pm 0.007$	$0.129\pm 0.021$
$Q_{0.3}$	$0.071\pm 0.011$	$0.079\pm 0.011$	$0.131\pm 0.022$
$Q_{0.4}$	$0.073\pm 0.015$	$0.081\pm 0.015$	$0.132\pm 0.024$
$Q_{0.5}$	$0.075\pm 0.019$	$0.083\pm 0.018$	$0.134\pm 0.027$
$Q_{0.6}$	$0.078\pm 0.023$	$0.086\pm 0.022$	$0.136\pm 0.030$
$Q_{0.7}$	$0.081\pm 0.027$	$0.088\pm 0.027$	$0.139\pm 0.034$
$Q_{0.8}$	$0.084\pm 0.032$	$0.092\pm 0.031$	$0.142\pm 0.038$
$Q_{0.9}$	$0.088\pm 0.037$	$0.095\pm 0.037$	$0.146\pm 0.044$
$Q_1$	$0.093\pm 0.043$	$0.100\pm 0.043$	$0.151\pm 0.051$

Table 8. (b) m = 20 𝑚 20 m=20

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB
$Q_0$	$0.048\pm 0.000$	$0.067\pm 0.005$	$0.159\pm 0.023$
$Q_{0.1}$	$0.049\pm 0.002$	$0.068\pm 0.006$	$0.160\pm 0.022$
$Q_{0.2}$	$0.050\pm 0.005$	$0.069\pm 0.007$	$0.161\pm 0.023$
$Q_{0.3}$	$0.052\pm 0.008$	$0.071\pm 0.009$	$0.162\pm 0.023$
$Q_{0.4}$	$0.054\pm 0.011$	$0.073\pm 0.011$	$0.163\pm 0.024$
$Q_{0.5}$	$0.056\pm 0.014$	$0.074\pm 0.013$	$0.164\pm 0.025$
$Q_{0.6}$	$0.059\pm 0.017$	$0.076\pm 0.016$	$0.166\pm 0.027$
$Q_{0.7}$	$0.061\pm 0.020$	$0.079\pm 0.019$	$0.168\pm 0.029$
$Q_{0.8}$	$0.064\pm 0.023$	$0.081\pm 0.022$	$0.170\pm 0.032$
$Q_{0.9}$	$0.067\pm 0.026$	$0.084\pm 0.026$	$0.173\pm 0.037$
$Q_1$	$0.071\pm 0.030$	$0.088\pm 0.030$	$0.176\pm 0.043$

Table 9. (c) m = 30 𝑚 30 m=30

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB
$Q_0$	$0.039\pm 0.000$	$0.070\pm 0.006$	$0.189\pm 0.023$
$Q_{0.1}$	$0.041\pm 0.003$	$0.071\pm 0.007$	$0.189\pm 0.023$
$Q_{0.2}$	$0.042\pm 0.006$	$0.072\pm 0.008$	$0.190\pm 0.023$
$Q_{0.3}$	$0.044\pm 0.008$	$0.073\pm 0.009$	$0.191\pm 0.023$
$Q_{0.4}$	$0.046\pm 0.011$	$0.075\pm 0.010$	$0.192\pm 0.024$
$Q_{0.5}$	$0.048\pm 0.013$	$0.076\pm 0.012$	$0.193\pm 0.024$
$Q_{0.6}$	$0.050\pm 0.015$	$0.078\pm 0.014$	$0.194\pm 0.025$
$Q_{0.7}$	$0.052\pm 0.017$	$0.079\pm 0.016$	$0.195\pm 0.026$
$Q_{0.8}$	$0.055\pm 0.020$	$0.081\pm 0.018$	$0.197\pm 0.027$
$Q_{0.9}$	$0.057\pm 0.022$	$0.083\pm 0.020$	$0.199\pm 0.029$
$Q_1$	$0.060\pm 0.025$	$0.086\pm 0.023$	$0.202\pm 0.033$

Table 10. (d) m = 40 𝑚 40 m=40

	relative error $\frac{{\Vert Q_{\lambda }h-f\Vert }_{L^2}}{{\Vert f\Vert }_{L^2}}$
	$\text{SNR}=\infty$	$\text{SNR}=20$dB	$\text{SNR}=10$dB
$Q_0$	$0.034\pm 0.000$	$0.075\pm 0.007$	$0.215\pm 0.023$
$Q_{0.1}$	$0.036\pm 0.003$	$0.076\pm 0.007$	$0.216\pm 0.023$
$Q_{0.2}$	$0.038\pm 0.006$	$0.077\pm 0.008$	$0.217\pm 0.023$
$Q_{0.3}$	$0.040\pm 0.008$	$0.078\pm 0.008$	$0.218\pm 0.023$
$Q_{0.4}$	$0.041\pm 0.010$	$0.079\pm 0.009$	$0.218\pm 0.023$
$Q_{0.5}$	$0.043\pm 0.011$	$0.080\pm 0.010$	$0.219\pm 0.024$
$Q_{0.6}$	$0.045\pm 0.013$	$0.082\pm 0.011$	$0.220\pm 0.024$
$Q_{0.7}$	$0.046\pm 0.014$	$0.083\pm 0.013$	$0.221\pm 0.025$
$Q_{0.8}$	$0.048\pm 0.016$	$0.084\pm 0.014$	$0.222\pm 0.026$
$Q_{0.9}$	$0.050\pm 0.017$	$0.085\pm 0.015$	$0.223\pm 0.027$
$Q_1$	$0.051\pm 0.019$	$0.087\pm 0.016$	$0.225\pm 0.029$

Equations

$$\tilde{f}=\underset{g\in \mathcal{T}}{\text{arg min}}\sum _{j\in \mathbb{N}}{w}_j{\left|⟨g,u_j{⟩}_{\mathcal{H}}-d_j\right|}^2.$$

$$\Vert f-Q((⟨f,u_j{⟩}_{\mathcal{H}}+l_j{)}_{j\in \mathbb{N}}){\Vert }_{\mathcal{H}}⩽\mu (Q)\Vert f-P_{\mathcal{T}}f{\Vert }_{\mathcal{H}}+\Vert Q{\Vert }_{\mathrm{op}}\Vert l{\Vert }_2.$$

$$Q_{1}d=\underset{g\in \mathcal{T}}{\text{arg min}}\sum _{j\in \mathbb{N}}{\left|⟨g,u_j{⟩}_{\mathcal{H}}-d_j\right|}^2=\underset{g\in \mathcal{T}}{\text{arg min}}\Vert U^{\ast }g-d{\Vert }_2.$$

$$Q_{0}d=\underset{g\in \mathcal{T}}{\text{arg min}}\Vert W{U}^{\ast }g-Wd{\Vert }_2.$$

$$Q_{\lambda }d=\underset{g\in \mathcal{T}}{\text{arg min}}\Vert {\Sigma }_{\lambda }^{-\frac{1}{2}}{U}^{\ast }g-{\Sigma }_{\lambda }^{-\frac{1}{2}}d{\Vert }_2.$$

$$A\Vert f{\Vert }_{\mathcal{H}}^2⩽\Vert U^{\ast }f{\Vert }_2^2⩽B\Vert f{\Vert }_{\mathcal{H}}^2.$$

$$C\Vert g{\Vert }_{\mathcal{H}}^2⩽\Vert T^{\ast }g{\Vert }_2^2⩽D\Vert g{\Vert }_{\mathcal{H}}^2\text{ for }g\in \mathcal{T}.$$

$$\Vert f-Q{U}^{\ast }f{\Vert }_{\mathcal{H}}⩽\mu \Vert f-P_{\mathcal{T}}f{\Vert }_{\mathcal{H}},\text{for all }f\in \mathcal{H}.$$

$$\Vert f-Q(U^{\ast }f+l){\Vert }_{\mathcal{H}}⩽\mu \Vert f-P_{\mathcal{T}}f{\Vert }_{\mathcal{H}}+\beta \Vert l{\Vert }_2.$$

$$Qc=\sum _{j\in \mathbb{N}}{c}_j{h}_j,$$

$$\cos ({\phi }_{\mathcal{T},\mathcal{U}})=\underset{\underset{\Vert g{\Vert }_{\mathcal{H}}=1}{g\in \mathcal{T}}}{\inf }\Vert P_{\mathcal{U}}g{\Vert }_{\mathcal{H}}=\underset{\underset{\Vert g{\Vert }_{\mathcal{H}}=1}{g\in \mathcal{T}}}{\inf }\underset{\underset{\Vert u{\Vert }_{\mathcal{H}}=1}{u\in \mathcal{U}}}{\sup }|⟨g,u{⟩}_{\mathcal{H}}|.$$

$$\Vert P_{\mathcal{T},\mathcal{W}}{\Vert }_{\mathrm{op}}=\frac{1}{\cos ({\phi }_{\mathcal{T},{\mathcal{W}}^{\perp }})}$$

$$\Vert f-P_{\mathcal{T}}f{\Vert }_{\mathcal{H}}⩽\Vert f-P_{\mathcal{T},\mathcal{W}}f{\Vert }_{\mathcal{H}}⩽\frac{1}{\cos ({\phi }_{\mathcal{T},{\mathcal{W}}^{\perp }})}\Vert f-P_{\mathcal{T}}f{\Vert }_{\mathcal{H}},$$

$$Q{U}^{\ast }g=\sum _{j\in \mathbb{N}}⟨g,P_{\mathcal{T}}{u}_j{⟩}_{\mathcal{H}}{h}_j=g.$$

$$\sum _{j\in \mathbb{N}}|⟨f,P_{\mathcal{T}}{u}_j{⟩}_{\mathcal{H}}{|}^2=\sum _{j\in \mathbb{N}}|⟨P_{\mathcal{T}}f,u_j{⟩}_{\mathcal{H}}{|}^2⩽B\Vert P_{\mathcal{T}}f{\Vert }_{\mathcal{H}}^2⩽B\Vert f{\Vert }_{\mathcal{H}}^2.$$

$$\Vert f-Q(U^{\ast }f+l){\Vert }_{\mathcal{H}}⩽\Vert P{\Vert }_{\mathrm{op}}\Vert f-P_{\mathcal{T}}f{\Vert }_{\mathcal{H}}+\Vert Q{\Vert }_{\mathrm{op}}\Vert l{\Vert }_2.$$

$$\mu (Q)=\Vert Q{U}^{\ast }{\Vert }_{\mathrm{op}}=\frac{1}{\cos ({\phi }_{\mathcal{T},\mathcal{W}})}.$$

$$\mathcal{N}(U^{\ast })\cap \mathcal{T}=\{0\}.$$

$$\Vert g{\Vert }_{\mathcal{H}}\cos ({\phi }_{\mathcal{T},\mathcal{U}})⩽\Vert P_{\mathcal{U}}g{\Vert }_{\mathcal{H}}\text{for all }g\in \mathcal{T}.$$

$$A\Vert g{\Vert }_{\mathcal{H}}^2{\cos }^2({\phi }_{\mathcal{T},\mathcal{U}})⩽A\Vert P_{\mathcal{U}}g{\Vert }_{\mathcal{H}}^2⩽\sum _{j\in \mathbb{N}}|⟨P_{\mathcal{U}}g,u_j{⟩}_{\mathcal{H}}{|}^2⩽B\Vert P_{\mathcal{U}}g{\Vert }_{\mathcal{H}}^2⩽B\Vert g{\Vert }_{\mathcal{H}}^2.$$

$$C_1\Vert g{\Vert }_{\mathcal{H}}^2⩽\sum _{j\in \mathbb{N}}|⟨g,P_{\mathcal{T}}{u}_j{⟩}_{\mathcal{H}}{|}^2=\sum _{j\in \mathbb{N}}|⟨P_{\mathcal{U}}g,u_j{⟩}_{\mathcal{H}}{|}^2⩽B\Vert P_{\mathcal{U}}g{\Vert }_{\mathcal{H}}^2.$$

$$(U^{\ast }T)(j,k)=⟨t_k,u_j{⟩}_{\mathcal{H}}=⟨t_k,P_{\mathcal{T}}{u}_j{⟩}_{\mathcal{H}},$$

$$(T^{\ast }Uc{)}_k=\sum _{j\in \mathbb{N}}⟨f,{\tilde{u}}_j{⟩}_{\mathcal{H}}⟨P_{\mathcal{T}}{u}_j,t_k{⟩}_{\mathcal{H}}=⟨f,t_k{⟩}_{\mathcal{H}}=(T^{\ast }f{)}_k.$$

$$\mathcal{R}(T^{\ast }U)=\mathcal{R}(T^{\ast }).$$

$$Q_{1}d=\underset{g\in \mathcal{T}}{\text{arg min}}\sum _{j\in \mathbb{N}}{\left|⟨g,u_j{⟩}_{\mathcal{H}}-d_j\right|}^2=\underset{g\in \mathcal{T}}{\text{arg min}}\Vert U^{\ast }g-d{\Vert }_2.$$

$$A_2{U}^{\ast }=P_{\mathcal{T},S(\mathcal{T}{)}^{\perp }}$$

$$A_{2}c=0\text{mbox}forc\in \mathcal{R}(U^{\ast }{)}^{\perp }.$$

$$K:=\underset{c\in {\ell }^2(\mathbb{N})}{\text{arg min}}\Vert U^{\ast }Tc-d{\Vert }_2.$$

$$Q_{1}d=\underset{g\in \mathcal{T}}{\text{arg min}}\sum _{j\in \mathbb{N}}|⟨g,u_j{⟩}_{\mathcal{H}}-d_j{|}^2=\underset{g\in \mathcal{T}}{\text{arg min}}\Vert U^{\ast }g-d{\Vert }_2^2.$$

$$A\cos ({\phi }_{\mathcal{T},\mathcal{U}})\Vert g{\Vert }_{\mathcal{H}}⩽A\Vert P_{\mathcal{U}}g{\Vert }_{\mathcal{H}}⩽\Vert S{P}_{\mathcal{U}}g{\Vert }_{\mathcal{H}}=\Vert Sg{\Vert }_{\mathcal{H}}\text{for }g\in \mathcal{T}.$$

Full text

Funding by the Austrian Science Fund (FWF) through grant NFN SISE (S10602) and P26273 - N25 and by the Vienna Science and Technology Fund (WWTF) through project VRG12-009 and project ICT15-119.

1. Introduction

1.1. The reconstruction problem

In this paper we treat the following sampling problem. Let $\mathcal{H}$ be a separable Hilbert space over $\mathbb{C}$ with inner product $\langle\cdot,\cdot\rangle_{\mathcal{H}}$.We assume that we are given linear measurements $(\langle f,u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}$, $u_{j}\in\mathcal{H}$, of an unknown function $f\in\mathcal{H}$. We call $(u_{j})_{j\in\mathbb{N}}$ the sampling frame and $\mathcal{U}:=\overline{\textnormal{span}}(u_{j})_{j\in\mathbb{N}}$ the sampling space. Our goal is to approximate the function $f$ by an element in the reconstruction space $\mathcal{T}:=\overline{\textnormal{span}}(t_{k})_{k\in\mathbb{N}}$ with $t_{k}\in\mathcal{H}$, by a series expansion $\tilde{f}=\sum_{k\in\mathbb{N}}c_{k}t_{k}$ from the given measurements. The main point is that in general the reconstruction space is distinct from the sampling space, whereas in classical frame theory these two spaces coincide.

1.2. Areas of application and related work

This type of sampling problem arises in many concrete applications and in the numerical modelling of infinite dimensional problems.

(i) Sampling of bandlimited functions. In [25] a bandlimited function is approximated from finitely many, nonuniform samples by means of a trigonometric polynomial. In this case the sampling space consists of the reproducing kernels $u_{j}(x)=\frac{\sin\pi(x-x_{j})}{\pi(x-x_{j})},j=1,\dots,n,$ and the reconstruction vectors are $t_{k}(x)=e^{2\pi ikx/(2M+1)}\chi_{[-M,M]}(x),|k|\leqslant M$.

(ii) Inverse Polynomial Reconstruction Method. In this method one tries to approximate an algebraic polynomial or an analytic function from its Fourier samples. Thus the sampling space consists of vectors $u_{j}(x)=e^{\pi ijx}\chi_{[-1,1]}(x),j=1,\dots,m$, and the reconstruction space consists of a suitable polynomial basis, usually the monomials $t_{k}(x)=x^{k},k=0,\dots,n$, or the Legendre polynomials. This method claims to efficiently mitigate the Gibbs phenomenon  [38, 28, 29, 30], and, indeed, the modified inverse polynomial reconstruction method [27] leads to a numerically stable reconstruction when $m\geqslant n^{2}$.

(iii) Fourier sampling. More generally, the goal is to approximate a compactly supported function in some smoothness class from its nonuniform Fourier samples $\hat{f}(\omega_{j})$. Thus the sampling space consists again of the functions $u_{j}(x)=e^{\pi i\omega_{j}x}\chi_{[-1,1]}(x)$. The reconstruction space depends on the signal model and on a priori information. If $f$ is smooth and belongs to a Besov space, then the reconstruction space may be taken to be a wavelet subspace. The problems of Fourier sampling have motivated Adcock and Hansen to revisit nonuniform sampling theory and to create the impressive and useful framework of generalized sampling [8, 7, 10, 6, 33].

(iv) Model reduction in parametric partial differential equations and the generalized empirical interpolation method. In general the solution manifold to a parametric partial differential equation is quite complicated, therefore it is approximated by finite-dimensional spaces $\mathcal{T}_{n}$. The Generalized Empirical Interpolation Method (GEIM) [34, 35] builds an interpolant in an $n$-dimensional space $\mathcal{T}_{n}$ based on the knowledge of $n$ physical measurements $(\langle f,u_{j}\rangle_{\mathcal{H}})_{j=1}^{n}$. In [36, 13] an extension based on a least squares method has been proposed, where the dimension $m$ of $\mathcal{T}_{m}$ is smaller than the number $n$ of the measurements $(\langle f,u_{j}\rangle_{\mathcal{H}})_{j=1}^{n}$. A further generalization to Banach spaces is contained in [18]. The focus in [36, 13] lies in minimizing the error caused by the model mismatch. This is done by using a correction term outside of the reconstruction space, which means that (in contrast to our work) the reconstruction is allowed to be located outside of the reconstruction space. This approach is optimal in the absence of measurement noise [13].

In all these problems the canonical approximation or reconstruction is by means of a least squares fit, namely

[TABLE]

The weights $w_{j}$ are usually chosen to be $w_{j}=1$, but in many contexts is has turned out to be useful to use weights as a kind of cheap preconditioners. The use of adaptive weights in sampling theory goes back at least to [23, 24], and has become standard in the recent work on (Fourier) sampling, see for example [5, 25, 26, 40, 1, 3, 2, 4, 11].

1.3. The reconstruction operators

In this paper we restrict ourselves to the case where the approximation $\tilde{f}=\sum_{k\in\mathbb{N}}c_{k}t_{k}$ of the unknown function $f\in\mathcal{H}$ is obtained by a linear and bounded reconstruction operator $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$. Thus the approximation $\tilde{f}$ from the data $\langle f,u_{j}\rangle_{j\in\mathbb{N}}$ is given by \tilde{f}=Q\big{(}\langle f,u_{j}\rangle_{j\in\mathbb{N}}\big{)}. We will use two quantities to measure the quality of such a reconstruction operator. As a measure of stability with respect to measurement noise we use the operator norm $\|Q\|_{\rm op}$. As a measure of stability with respect to model mismatch we follow [9] and use the so-called quasi-optimality constant $\mu(Q)$ (see Definition 2.1).

Let $P_{\mathcal{T}}$ denote the orthogonal projection onto $\mathcal{T}$, $f\in\mathcal{H}$ the target function, and $l\in\ell^{2}(\mathbb{N})$ be the noise vector. Then the input data are given by the sequence $(\langle f,u_{j}\rangle_{\mathcal{H}}+l_{j})_{j\in\mathbb{N}}$, the reconstruction is $\tilde{f}=Q((\langle f,u_{j}\rangle_{\mathcal{H}}+l_{j})_{j\in\mathbb{N}})$, and the error is bounded by

[TABLE]

1.4. Contributions

The error bound (2) raises several questions:

• •

Which operators admit an error bound of the form (2)?

• •

Under what circumstances does such an operator exist?

• •

Which operator has the smallest possible operator norm $\|Q\|_{\rm op}$?

• •

Which operator has the smallest possible quasi-optimality constant $\mu(Q)$?

• •

Is there a way to trade-off between quasi-optimality and operator norm?

Our objective is to answer these questions both in finite-dimensional and infinite dimensional Hilbert spaces. The results can be formulated conveniently in the language of frame theory.

(i) Characterization of all reconstruction operators. We characterize all reconstruction operators that admit an error estimate of the form (2). In fact, every dual frame of the set $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ yields a reconstruction satisfying (2). Conversely, every reconstruction operator subject to (2) is the synthesis operator of a dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$. Note that (2) implies that such a reconstruction operator $Q$ is exact on the reconstruction space, i.e., $f=Q(\langle f,u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}$ for all $f\in\mathcal{T}$. Reconstruction operators fulfilling this property are called perfect. For a precise formulation see Theorem 2.2.

The important insight of  [9] is the connection between stability and the angle $\phi_{\mathcal{T},\mathcal{U}}$ between the sampling space and the reconstruction space. We will see that a perfect reconstruction operator exists if and only if $\cos(\phi_{\mathcal{T},\mathcal{U}})>0$. It should also be mentioned that the reconstruction operators considered in this paper are a special case of pseudoframes [32].

(ii) Least squares approximation. As already mentioned, the canonical approximation of the data $\langle f,u_{j}\rangle_{j\in\mathbb{N}}$ by a vector in $\mathcal{T}$ is by a least squares fit. Let $U^{*}f=(\langle f,u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}$ denote the analysis operator of the frame $(u_{j})_{j\in\mathbb{N}}$ and let $d\in\ell^{2}(\mathbb{N})$ denote the vector containing the noisy measurements $(d_{j})=(\langle f,u_{j}\rangle_{\mathcal{H}}+l_{j})_{j\in\mathbb{N}}=U^{*}f+l$. Let the reconstruction operator $Q_{1}$ be defined by the least squares fit

[TABLE]

It is folklore that the least squares solution (3) is optimal in the absence of additional information on $f$. Precise formulations of this optimality were proven in [9, Theorem 6.2.] (including even non-linear reconstructions) and in [12] (in abstract Hilbert space). We will show in addition (Theorem 3.1) that $Q_{1}$ is the synthesis operator of the canonical dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$. Using this property we derive a simple proof for the statement that the operator $Q_{1}$ has the smallest possible operator norm among all perfect reconstruction operators.

(iii) Minimizing the quasi-optimality constant. Let $W=G^{\frac{\dagger}{2}}:=(G^{\dagger})^{\frac{1}{2}}$ be the square root of the Moore-Penrose pseudoinverse of the Gramian $G=U^{*}U$ of the sampling frame $\mathcal{U}$ and consider the operator $Q_{0}$ defined by

[TABLE]

We will show (Theorem 3.3) that $Q_{0}$ has the smallest possible quasi-optimality constant. The reduction of the quasi-optimality constant is one of the motivations of weighted least squares, see [2, 1, 5, 3]. In (4) we go a step further and use the non-diagonal matrix $W=G^{\frac{\dagger}{2}}$ as a weight for the least squares problem. From the point of view of linear algebra, $W$ may be seen as a preconditioner.

In [2, 1, 5, 3] and also [11, 24, 25, 26, 4, 23] the stability with respect to a bias in the measured object is considered, i.e., the reconstruction from $U^{*}(f+\Delta f)=(\langle f+\Delta f,u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}$ (stated in terms of a frame inequality in the latter). In this context, $Q_{0}$ is the most stable operator with respect to biased objects, see the discussion in Section 3.6.

(iv) Trading stability and quasi-stability. It is natural to ask whether one can mix between the two least squares problems (3) and (4). Let \Sigma_{\lambda}=\big{(}\lambda I+(1-\lambda)U^{*}U\big{)} and $\lambda\in[0,1]$ and define $Q_{\lambda}$ by

[TABLE]

These reconstruction operators “interpolate” between $Q_{1}$ (most stable with respect to noise) and $Q_{0}$ (most stable with respect to model uncertainty). The parameter $\lambda$ can be seen as a regularization parameter, or alternatively the matrix $\Sigma_{\lambda}$ as version of the adaptive weights in sampling. In Theorem 3.7 and Lemma 3.8 we will study this class of reconstruction operators and derive several representations for $Q_{\lambda}$.

(v) Fourier resampling — numerical experiments. In the last part we carry out a numerical comparison of the various reconstruction operators on the basis of the so-called resampling problem. We approximate a function with compact support from finitely many, nonuniform samples of its Fourier transform and then resample the Fourier transform on a regular grid. For this problem we test the performance of the reconstruction operators $Q_{\lambda}$.

The paper is organized as follows: In Section 2 we introduce the frame theoretic background, discuss the angle between subspaces, and characterize all reconstruction operators satisfying the required stability estimate (2). In Section 3 we study the various least squares problems (3) and (4) and analyze several representations of the corresponding reconstruction operators. The section is complemented by general numerical considerations. Section 4 covers the numerical experiments on Fourier sampling. The brief appendix collects some standard facts about frames.

2. Classification of all reconstruction operators

We will use the language of frame theory throughout the whole paper. The Appendix contains a short list of basic definitions and well known facts from frame theory. For more details on this topic, see for instance [15].

Let us introduce some notation. To every set of measurement vectors $(u_{j})_{j\in\mathbb{N}}$ in a Hilbert space $\mathcal{H}$ (of finite or infinite dimension) we associate the synthesis operator $U$ defined formally by $Uc=\sum_{j\in\mathbb{N}}c_{j}u_{j}$ and the sampling space $\mathcal{U}=\overline{\textnormal{span}}(u_{j})_{j\in\mathbb{N}}$. The adjoint operator $U^{*}$ consists of the measurements $U^{*}f=(\langle f,u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}$ and is called the analysis operator. The frame operator is $S=UU^{*}$ and the Gramian is $G=U^{*}U$. With this notation, $(u_{j})_{j\in\mathbb{N}}$ is a frame for $\mathcal{U}=\overline{\textnormal{span}}(u_{j})_{j\in\mathbb{N}}$, if there exist constants $A,B>0$, such that for every $f\in\mathcal{U}$

[TABLE]

We always assume that $(u_{j})_{j\in\mathbb{N}}$ is a frame for $\mathcal{U}$, thus $U^{*}$ is bounded from $\mathcal{H}$ to $\ell^{2}(\mathbb{N})$ and $U^{*}$ has closed range in $\ell^{2}(\mathbb{N})$. We use $\mathcal{R}(A)$ for the range of an operator $A$ and $\mathcal{N}(A)$ for its kernel (null space).

Likewise we assume that $(t_{k})_{k\in\mathbb{N}}$ is a frame for the reconstruction space $\mathcal{T}=\overline{\textnormal{span}}(t_{k})_{k\in\mathbb{N}}$ with synthesis operator $T$ and analysis operator $T^{*}$. Thus

[TABLE]

Given a sequence of linear measurements $(\langle f,u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}=U^{*}f$, we try to find an approximation of $f$ in the subspace $\mathcal{T}$. Assuming that all occurring operartors are bounded, we investigate the class of reconstruction operators $Q:\ell^{2}\to\mathcal{T}$, such that $\tilde{f}=QU^{*}f$ is the desired reconstruction or approximation of $f$. We use two metrics to quantify the stability of a reconstruction operator ${Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}}$. As a measure for stability with respect to measurement noise we use the operator norm $\|Q\|_{\rm op}$. In order to measure how well $Q$ deals with the part of the function lying outside of the reconstruction space, we use the quasi-optimality constant from [9].

Definition 2.1**.**

Let $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ and $P_{\mathcal{T}}$ be the orthogonal projection onto $\mathcal{T}$. The quasi-optimality constant $\mu=\mu(Q)>0$ is the smallest number $\mu$, such that

[TABLE]

If $\mu(Q)<\infty$ we call $Q$ a quasi-optimal operator. Since $P_{\mathcal{T}}f$ is the element of $\mathcal{T}$ closest to $f$, the quasi-optimality constant $\mu$ is a measure of how well $QU^{*}$ performs in comparison to orthogonal projection $P_{\mathcal{T}}$. Note that for $f\in\mathcal{T}$ we have $QU^{*}f=f$, thus a quasi-optimal reconstruction operator is perfect.

The following theorem characterizes all bounded quasi-optimal operators.

Theorem 2.2**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be closed subspaces of $\mathcal{H}$, and $(u_{j})_{j\in\mathbb{N}}$ be a Bessel sequence spanning the closed subspace $\mathcal{U}$. For an operator $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ the following are equivalent.

1. (i)

There exist constants $0\leqslant\mu,\beta<\infty$, such that for $f\in\mathcal{H}$ and $l\in\ell^{2}(\mathbb{N})$

[TABLE] 2. (ii)

$QU^{*}g=g$* for $g\in\mathcal{T}$ and $Q$ is a bounded operator. * 3. (iii)

The sequence $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ is a frame for $\mathcal{T}$. Let $(h_{j})_{j\in\mathbb{N}}\subset\mathcal{T}$ be a dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$, then $Q$ is of the form

[TABLE]

i.e., $Q$ is the synthesis operator of some dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$. 4. (iv)

The operator $Q$ is bounded and $QU^{*}$ is a bounded oblique projection onto $\mathcal{T}$.

Theorem 2.2 sets up a bijection between the class of reconstruction operators and the class of all dual frames of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$.

To prove Theorem 2.2, we need the concept of subspace angles. Among the many different definitions of the angle between subspaces (see [41, 39]) the following definition is most suitable for our analysis.

Definition 2.3**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be closed subspaces of a Hilbert space $\mathcal{H}$. The subspace angle $\varphi_{\mathcal{T},\mathcal{U}}\in[0,\frac{\pi}{2}]$ between $\mathcal{T}$ and $\mathcal{U}$ is defined as

[TABLE]

We observe that in general $\cos(\varphi_{\mathcal{T},\mathcal{U}})\neq\cos(\varphi_{\mathcal{U},\mathcal{T}})$. For $\mathcal{T}\subset\mathcal{U}$, $\cos(\varphi_{\mathcal{T},\mathcal{U}})=1$ and therefore $\varphi_{\mathcal{T},\mathcal{U}}=0$. If $\mathcal{U}\subsetneq\mathcal{T}$, then $\cos(\varphi_{\mathcal{T},\mathcal{U}})=0$ and $\varphi_{\mathcal{T},\mathcal{U}}=\frac{\pi}{2}$.

The following lemma collects the main properties of oblique projections and angles between subspaces.

Lemma 2.4**.**

Assume that $\mathcal{T}$ and $\mathcal{W}$ are closed subspaces of a Hilbert space $\mathcal{H}$. Then

1. (i)

$\cos(\varphi_{\mathcal{T},\mathcal{W}^{\perp}})>0$* if and only if $\mathcal{T}\cap\mathcal{W}=\{0\}$ and the direct sum $\mathcal{T}\oplus\mathcal{W}$ (not necessarily orthogonal) is closed in $\mathcal{H}$.* 2. (ii)

If $\mathcal{T}\cap\mathcal{W}=\{0\}$ and $\mathcal{H}_{1}:=\mathcal{T}\oplus\mathcal{W}$ is a closed subspace of $\mathcal{H}$, then the oblique projection $P_{\mathcal{T},\mathcal{W}}:\mathcal{H}_{1}\rightarrow\mathcal{T}$ with range $\mathcal{T}$ and kernel $\mathcal{W}$ is well defined and bounded on $\mathcal{H}_{1}$. 3. (iii)

Let $\cos(\varphi_{\mathcal{T},\mathcal{W}^{\perp}})>0$, $\mathcal{H}_{1}:=\mathcal{T}\oplus\mathcal{W}$, and let $P_{\mathcal{T},\mathcal{W}}:\mathcal{H}_{1}\rightarrow\mathcal{T}$ be the oblique projection with range $\mathcal{T}$ and null space $\mathcal{W}$. Then

[TABLE]

and

[TABLE]

for all $f\in\mathcal{H}_{1}$. The upper bound in (7) is sharp.

Item (i) of Lemma 2.4 is stated in [42, Theorem 2.1], the proof of (ii) can be found in [14, Theorem 1], and for (iii) see [41], [14], and [9, Corollary 3.5].

Proof of Theorem 2.2 (i) $\,\Rightarrow\,$ (ii). Set $l=0$ and choose $f\in\mathcal{T}$. Then (5) implies $QU^{*}f=f$, since otherwise $\mu=\infty$. Setting $f=0$ in (5) implies that $Q$ is bounded.

(ii) $\,\Rightarrow\,$ (iii) Let $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ be a bounded operator with $QU^{*}g=g$ for $g\in\mathcal{T}$. Let $(e_{j})_{j\in\mathbb{N}}$ be the standard basis of $\ell^{2}(\mathbb{N})$ and let $h_{j}=Qe_{j}$. Then $Qc=\sum_{j\in\mathbb{N}}c_{j}h_{j}$. In particular for $g\in\mathcal{T}$,

[TABLE]

Since $Q$ is bounded, $(h_{j})_{j\in\mathbb{N}}$ is a Bessel sequence in $\mathcal{T}$. By assumption $(u_{j})_{j\in\mathbb{N}}$ is a Bessel sequence in $\mathcal{U}$ with Bessel bound $B$ and consequently

[TABLE]

Therefore $(h_{j})_{j\in\mathbb{N}}$ is a dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$.

(iii) $\,\Rightarrow\,$ (iv) Let $(h_{j})_{j\in\mathbb{N}}$ be a dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ and define $Q$ by $Qc=\sum_{j\in\mathbb{N}}c_{j}h_{j}$ and $P:=QU^{*}$. Since the range of $P$ is contained in $\mathcal{T}$ and $QU^{*}g=g$ for $g\in\mathcal{T}$, it follows that $P$ is onto $\mathcal{T}$ and that $P^{2}=QU^{*}QU^{*}=QU^{*}=P$. Since both $Q$ and $U^{*}$ are bounded, $P$ is bounded.

(iv) $\,\Rightarrow\,$ (i). Let $Q$ be a bounded operator, and let $P:=QU^{*}$ be a bounded oblique projection onto $\mathcal{T}$. Lemma 2.4(iii) implies that $\|f-Pf\|_{\mathcal{H}}\leqslant\|P\|_{\rm op}\|f-P_{\mathcal{T}}f\|_{\mathcal{H}}$, and consequently

[TABLE]

This finishes the proof. $\Box$

As a direct consequence of Theorem 2.2 and Lemma 2.4, (iii), we obtain the following characterization of the quasi-optimality constant.

Corollary 2.5**.**

If $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ is a bounded and perfect reconstruction operator, then $P=QU^{*}$ is a bounded oblique projection onto $\mathcal{T}$. If $\mathcal{W}^{\perp}$ denotes the null-space of $P$, then

[TABLE]

In the following we always use the assumption that the angle between the reconstruction and sampling space fulfills ${\cos(\varphi_{\mathcal{T},\mathcal{U}})>0}$. The following lemma shows that this assumption is equivalent to $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ forming a frame for $\mathcal{T}$ for every frame $(u_{j})_{j\in\mathbb{N}}$ for $\mathcal{U}$. By Theorem 2.2 (iii) this is necessary for the existence of a quasi-optimal operator. In finite dimensions, for a basis $(u_{j})_{j=1}^{n}$for $\mathcal{U}$, $(P_{\mathcal{T}}u_{j})_{j=1}^{n}$ can only be a spanning set for $\mathcal{T}$ if $\dim(\mathcal{U})\geqslant\dim(\mathcal{T})$. This means that by the assumption ${\cos(\varphi_{\mathcal{T},\mathcal{U}})>0}$ we restrict ourselves to an oversampled regime.

Lemma 2.6**.**

If $\mathcal{T}$ and $\mathcal{U}$ are closed subspaces of $\mathcal{H}$, then the following are equivalent:

1. (i)

$\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$. 2. (ii)

For every frame $(u_{j})_{j\in\mathbb{N}}$ for $\mathcal{U}$ with frame bounds $A$ and $B$, the projection $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ is a frame for $\mathcal{T}$ with frame bounds $A\cos^{2}(\varphi_{\mathcal{T},\mathcal{U}})$ and $B$.

If one of these conditions is satisfied, then the following property holds: 3. (iii)

$\mathcal{R}(T^{*}U)=\mathcal{R}(T^{*})$, therefore both $\mathcal{R}(T^{*}U)$ and $\mathcal{R}(U^{*}T)$ are closed subspaces and $U^{*}T$ is pseudo-invertible. Furthermore,

[TABLE]

Proof.

(i) $\,\Rightarrow\,$ (ii) Let $(u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{U}$ with frame bounds $A$ and $B$. The assumption $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$ and the definition of $\varphi_{\mathcal{T},\mathcal{U}}$ imply that

[TABLE]

In particular, for $g\in\mathcal{T}$ we obtain with (9)

[TABLE]

The identity $\langle P_{\mathcal{U}}g,u_{j}\rangle_{\mathcal{H}}=\langle g,u_{j}\rangle_{\mathcal{H}}=\langle g,P_{\mathcal{T}}u_{j}\rangle_{\mathcal{H}}$ for $g\in\mathcal{T}$ and $j\in\mathbb{N}$ now shows that $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ is a frame for $\mathcal{T}$ with frame bounds $A\cos^{2}(\varphi_{\mathcal{T},\mathcal{U}})$ and $B$.

(ii) $\,\Rightarrow\,$ (i) Let $(u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{U}$ with upper frame bound $B$ and let $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{T}$ with lower frame bound $C_{1}>0$. Since $\langle g,P_{\mathcal{T}}u_{j}\rangle_{\mathcal{H}}=\langle P_{\mathcal{U}}g,u_{j}\rangle_{\mathcal{H}}$ for $g\in\mathcal{T}$, we obtain

[TABLE]

This implies that $\cos(\varphi_{\mathcal{T},\mathcal{U}})=\underset{\underset{\|g\|_{\mathcal{H}}=1}{g\in\mathcal{T}}}{\inf}\|P_{\mathcal{U}}g\|_{\mathcal{H}}\geqslant\sqrt{\frac{C_{1}}{B}}>0$.

(ii) $\,\Rightarrow\,$ (iii) Since $(u_{j})_{j\in\mathbb{N}}$ and $(t_{k})_{k\in\mathbb{N}}$ are Bessel sequences, both $U^{*}$ and $T$ are bounded, and therefore $U^{*}T$ is also bounded. The entries of $U^{*}T$ are given by

[TABLE]

and $U^{*}T$ is a cross-Gramian of two frames for $\mathcal{T}$. Let $(\tilde{u}_{j})_{j\in\mathbb{N}}$ be a dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$. Setting $c_{j}=\langle f,\tilde{u}_{j}\rangle_{\mathcal{H}}$ we obtain, for $f\in\mathcal{T}$,

[TABLE]

It follows that

[TABLE]

Since $(t_{k})_{k\in\mathbb{N}}$ is a frame for $\mathcal{T}$, $\mathcal{R}(T^{*})$ is closed in $\ell^{2}(\mathbb{N})$, and so are $\mathcal{R}(T^{*}U)$ and $\mathcal{R}(U^{*}T)$. This implies that both $T^{*}U$ and $U^{*}T$ possess a pseudoinverse (see Appendix 4.1).

To prove (8), let $g\in\mathcal{N}(U^{*})\cap\mathcal{T}$. Then $g=Tc$ for some $c\in\ell^{2}(\mathbb{N})$ and $U^{*}g=U^{*}Tc=0$. This means that $c\in\mathcal{N}(U^{*}T)=\mathcal{R}(T^{*}U)^{\perp}=\mathcal{R}(T^{*})^{\perp}=\mathcal{N}(T)$. Consequently, $g=Tc=0$, and $\mathcal{N}(U^{*})\cap\mathcal{T}=\{0\}$. ∎

3. The reconstruction operators

3.1. Least squares and the operator $Q_{1}$

We first consider the reconstruction operator $Q_{1}:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ corresponding to the solution of the least squares problem

[TABLE]

This approach is analyzed in detail in [9]. Least square approximation is by far the most frequent approximation method in applications and of fundamental importance, since it has the smallest operator norm among all perfect operators.

The following theorem reviews several representations of the operator $Q_{1}$. The connection of the operator $Q_{1}$ to the oblique projection $P_{\mathcal{T},S(\mathcal{T})^{\perp}}$ was already derived in [9, Section 4.1.] for finite dimensional space $\mathcal{T}$. Our new contribution is the connection to the canonical dual frame and the systematic discussion of the various representions of a least squares problem. As we will apply the statement several times, we include a streamlined proof. As usual, $A^{\dagger}$ denotes the Moore-Penrose pseudo-inverse of an operator $A$. For the existence of $A^{\dagger}$ it suffices to show that the range of $A$ is closed (see Appendix 4.1).

Theorem 3.1**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be closed subspaces of a Hilbert space $\mathcal{H}$ such that $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$. Let $(u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{U}$ with synthesis operator $U$ and frame operator $S$. Let $(t_{k})_{k\in\mathbb{N}}$ be a frame for $\mathcal{T}$ with synthesis operator $T$.

Consider the following operators:

1. (i)

$A_{1}=T\left(U^{*}T\right)^{\dagger}$. 2. (ii)

The operator $A_{2}$ is given on $\mathcal{R}(U^{*})$ by

[TABLE]

and on $\mathcal{R}(U^{*})^{\perp}$ by

[TABLE]

By (13) $A_{2}$ is independent of the particular choice of the reconstruction frame $(t_{k})_{k\in\mathbb{N}}$ for $\mathcal{T}$. 3. (iii)

Let $(h_{j})_{j\in\mathbb{N}}$ be the canonical dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ and $A_{3}c=\sum_{j\in\mathbb{N}}c_{j}h_{j}$ be the synthesis operator of $(h_{j})_{j\in\mathbb{N}}$. 4. (iv)

Let $d\in\ell^{2}(\mathbb{N})$ and let $\hat{c}=(\hat{c}_{k})_{k\in\mathbb{N}}$ be the unique minimal norm element of the set

[TABLE]

Let the operator $A_{4}$ be defined by $A_{4}d=\sum_{k=1}^{\infty}\hat{c}_{k}t_{k}=T\hat{c}$.

Then all four operators are equal, $Q_{1}:=A_{1}=A_{2}=A_{3}=A_{4}$ and provide the unique solution to the least squares problem

[TABLE]

Proof.

Step 1. First we check that each $A_{j},j=1,\dots,4$, is well defined from $\ell^{2}(\mathbb{N})$ to $\mathcal{T}$. For $A_{1}$ this is clear by virtue of Lemma 2.6.

For $A_{2}$ we need to show that the projection $P_{\mathcal{T},S(\mathcal{T})^{\perp}}$ is well defined and bounded on the whole space $\mathcal{H}$. According to Lemma 2.4(i) we need to verify that $S(\mathcal{T})$ is closed, $\cos(\varphi_{\mathcal{T},S(\mathcal{T})})>0$ and that $\mathcal{H}=\mathcal{T}\oplus S(\mathcal{T})^{\perp}$. For this we exploit the frame inequality (10) of $(u_{j})_{j\in\mathbb{N}}$, (9), and the fact that $S=SP_{\mathcal{U}}=P_{\mathcal{U}}SP_{\mathcal{U}}$, and we obtain

[TABLE]

The lower bound implies that $S(\mathcal{T})$ is closed. For the angle $\varphi_{\mathcal{T},S(\mathcal{T})}$ we obtain

[TABLE]

Since $\langle g,Sg\rangle=\langle P_{\mathcal{U}}g,SP_{\mathcal{U}}g\rangle\rangle\geqslant A\|P_{\mathcal{U}}g\|_{\mathcal{H}}^{2}$ and $\|Sg\|_{\mathcal{H}}=\|SP_{\mathcal{U}}g\|_{\mathcal{H}}\leqslant B\|P_{\mathcal{U}}g\|_{\mathcal{H}}$, we continue (17) as follows:

[TABLE]

It remains to prove that $\mathcal{T}\oplus S(\mathcal{T})^{\perp}=\mathcal{H}$, or, equivalently, that

[TABLE]

So assume that $g\in\mathcal{T}^{\perp}\cap S(\mathcal{T})$. Since $\mathcal{R}(T)=\mathcal{T}$, we may write every $t\in\mathcal{T}$ as $t=Td$ for some $d\in\ell^{2}(\mathbb{N})$. In particular, there exist $c\in\ell^{2}(\mathbb{N})$ and $v=Tc\in\mathcal{T}$, such that $g=Sv=STc$. Then for all $d\in\ell^{2}(\mathbb{N})$, the element $g\in\mathcal{T}^{\perp}\cap S(\mathcal{T})$ satisfies

[TABLE]

Setting $d=c$, we obtain $U^{*}Tc=0$. By Lemma 2.6(iii) $v=Tc\in\mathcal{T}\cap\mathcal{N}(U^{*})=\{0\}$, and thus $g=Sv=0$, which implies that $\mathcal{T}^{\perp}\cap S(\mathcal{T})=\{0\}$.

The operator $A_{3}$ is the synthesis operator with respect to the canonical dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ and is therefore bounded by general frame theory.

Now to $A_{4}$: By Lemma 2.6 the operator $U^{*}T$ has a closed range and therefore its Moore-Penrose pseudoinverse is well defined. It is well known that $\hat{c}=(U^{*}T)^{\dagger}d$ is the unique element of $K$ of minimal norm. Consequently, $A_{4}d=\sum_{k\in\mathbb{N}}\hat{c}_{k}t_{k}$ is bounded on $\ell^{2}(\mathbb{N})$.

Step 2. We next show that all these operators are equal.

Claim $A_{1}=A_{4}$. Since $\hat{c}=(U^{*}T)^{\dagger}d$ is the unique element of $K$ of minimal norm and $A_{4}d=T\hat{c}=T(U^{*}T)^{\dagger}d$, we have $A_{1}=A_{4}$.

Claim $A_{1}=A_{2}$. We define $R:=A_{1}U^{*}=T(U^{*}T)^{\dagger}U^{*}$ and show that $R^{2}=R$, $\mathcal{R}(R)=\mathcal{T}$ and $\mathcal{N}(R)=S(\mathcal{T})^{\perp}$. The equality $R^{2}=R$ follows from the identity $A^{\dagger}AA^{\dagger}=A^{\dagger}$ for the Moore-Penrose pseudoinverse applied to $A=U^{*}T$. Clearly $\mathcal{R}(R)\subseteq\mathcal{T}$. To prove the converse inclusion we show that $\mathcal{R}(RT)=\mathcal{T}$. Using $\mathcal{R}(T^{*}U)=\mathcal{R}(T^{*})$ from Lemma 2.6 and $A^{\dagger}A=P_{\mathcal{R}(A^{*})}$ we conclude that

[TABLE]

which proves $\mathcal{R}(R)=\mathcal{T}$.

Now let $f\in\mathcal{N}(R)$, then we have, for all $h\in\mathcal{H}$,

[TABLE]

Since $\mathcal{R}(T^{*})=\mathcal{R}(T^{*}U)$ by Lemma 2.6(iii), this means that for all $c\in\ell^{2}(\mathbb{N})$

[TABLE]

In other words, $f\in\mathcal{R}(U^{*}UT)^{\perp}=S(\mathcal{T})^{\perp}$, as claimed.

Claim $A_{1}=A_{3}$. We need to show that the operator $A_{1}=T\left(U^{*}T\right)^{\dagger}$ is the synthesis operator of the canonical dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$. The frame operator $\tilde{S}$ of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ can be written in the form

[TABLE]

By Definition 4.2(iv), the canonical dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ is given by $(\tilde{S}^{\dagger}P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ with synthesis operator

[TABLE]

where we used $A^{\dagger}=(A^{*}A)^{\dagger}A^{*}$ with $A=U^{*}P_{\mathcal{T}}$ for the last equality. Since we have already proved that $A_{1}=A_{2}$, we know that the operator $A_{1}$ is independent of the particular choice of a frame for $\mathcal{T}$. We may therefore use the frame $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$ with synthesis operator $P_{\mathcal{T}}U$ instead of $T$, and as a consequence obtain that $A_{1}=P_{\mathcal{T}}U(U^{*}P_{\mathcal{T}}U)^{\dagger}=(U^{*}P_{\mathcal{T}})^{\dagger}$, where now we use $A^{\dagger}=A^{*}(AA^{*})^{\dagger}$ with $A=U^{*}P_{\mathcal{T}}$. Comparing with (19), we have proved that $A_{3}=A_{1}$.

Step 3. Finally we show that each operator $A_{1}=\dots=A_{4}$ provides the unique solution to the least squares fit (12). Since $\mathcal{N}(U^{*})\cap\mathcal{T}=\{0\}$ by Lemma 2.6(iii), the solution $\tilde{f}\in\mathcal{T}$ of the least squares problem

[TABLE]

is unique. Since $\mathcal{R}(T)=\mathcal{T}$, there exists a $c\in\ell^{2}(\mathbb{N})$, such that $\tilde{f}=Tc$, and by (20) $\tilde{f}=Tc$ for every element $c\in K$ (cf. (15)). In particular, for the minimal norm element $\hat{c}=(U^{*}T)^{\dagger}d\in K$ used for the definition of the operator $A_{4}$, we obtain $\tilde{f}=T\hat{c}=T(U^{*}T)^{\dagger}d=A_{4}d=Q_{1}d$. ∎

Theorem 3.1 implies a simple proof for the statement that the operator $Q_{1}$ has the smallest possible operator norm among all perfect reconstruction operators. This has already been proven in [9, Theorem 6.2.] in a more general setup that includes non-linear reconstruction operators.

Theorem 3.2**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be two closed subspaces of a Hilbert space $\mathcal{H}$ such that $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$. If $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ is a perfect reconstruction operator ($QU^{*}g=g$ for $g\in\mathcal{T}$), then

[TABLE]

Proof.

Let $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ be a bounded and perfect operator. From Theorem 2.2 we infer that $Q$ is the synthesis operator of a dual frame of $(P_{\mathcal{T}}u_{j})_{j\in\mathbb{N}}$. From Lemma 4.5 (expansion coefficients with respect to the canonical dual frame have the minimum $\ell^{2}$-norm) we infer that for $g\in\mathcal{T}$

[TABLE]

Since $Q^{*}$ is the analysis operator of a frame for $\mathcal{T}$, we have $Q^{*}g^{\perp}=Q_{1}^{*}g^{\perp}=0$ for $g^{\perp}\in\mathcal{T}^{\perp}$. Therefore $\|Q^{*}\|_{\rm op}\geqslant\|{Q_{1}}^{*}\|_{\rm op}$, and consequently $\|Q\|_{\rm op}\geqslant\|Q_{1}\|_{\rm op}$. ∎

3.2. The operator $Q_{0}$

In the last section we analyzed the operator $Q_{1}$ with the smallest operator norm. We now introduce and study the operator $Q_{0}$ with the smallest quasi-optimality constant. In the following we write $G^{\frac{\dagger}{2}}=(G^{\dagger})^{1/2}$ when $G$ is a positive operator with a pseudoinverse. 111An early version of Theorem 3.3 was announced in our technical report https://arxiv.org/pdf/1312.1717.pdf (Theorem 2.6)

Theorem 3.3**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be closed subspaces of a Hilbert space $\mathcal{H}$ such that $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$. Let $(u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{U}$ with synthesis operator $U$ and Gramian $G=U^{*}U$, and let $(t_{k})_{k\in\mathbb{N}}$ be a frame for $\mathcal{T}$ with synthesis operator $T$. Consider the following operators:

1. (i)

$B_{1}:=T\left(G^{\frac{\dagger}{2}}U^{*}T\right)^{\dagger}G^{\frac{\dagger}{2}}$. 2. (ii)

The operator $B_{2}$ given on $\mathcal{R}(U^{*})$ by

[TABLE]

and on $\mathcal{R}(U^{*})^{\perp}$ by

[TABLE]

Consequently $B_{2}$ depends only on the subspace $\mathcal{T}$, but not on the particular choice of a frame $(t_{k})_{k\in\mathbb{N}}$ for $\mathcal{T}$. 3. (iii)

Let $d\in\ell^{2}(\mathbb{N})$ and $\hat{c}=(\hat{c}_{k})_{k\in\mathbb{N}}$ be the unique minimal norm element of the set

[TABLE]

Let the operator $B_{3}$ be defined by $B_{3}d=\sum_{k=1}^{\infty}\hat{c}_{k}t_{k}$.

Then the operators defined by (i)-(iv) are equivalent, $Q_{0}:=B_{1}=B_{2}=B_{3}$ and provide the unique solution of the least squares problem

[TABLE]

Proof.

Let $(S^{\frac{\dagger}{2}}u_{j})_{j\in\mathbb{N}}$ be the tight frame for $\mathcal{U}$ associated to $(u_{j})_{j\in\mathbb{N}}$. By Lemma 4.4 its analysis operator $L^{*}$ is given by

[TABLE]

We now apply Theorem 3.1 to the frames $(t_{k})_{k\in\mathbb{N}}$ for $\mathcal{T}$ and $(S^{\frac{\dagger}{2}}u_{j})_{j\in\mathbb{N}}$ for $\mathcal{U}$.

Since $(S^{\frac{\dagger}{2}}u_{j})_{j\in\mathbb{N}}$ is a tight frame for $\mathcal{U}$, its frame operator is $\tilde{S}=LL^{*}=P_{\mathcal{U}}$. As proven in Theorem 3.1, $\mathcal{H}=\mathcal{T}\oplus\tilde{S}(\mathcal{T})^{\perp}=\mathcal{T}\oplus P_{\mathcal{U}}(\mathcal{T})^{\perp}$ and the projection $P_{\mathcal{T},\tilde{S}(\mathcal{T})^{\perp}}=P_{\mathcal{T},P_{\mathcal{U}}(\mathcal{T})^{\perp}}$ is well defined and bounded.

Let us show that $B_{1}=B_{2}$. We set $A_{1}=T(L^{*}T)^{\dagger}$. The equivalence of the operators $A_{1}$ and $A_{2}$ of Theorem 3.1 says that $A_{1}L^{*}=P_{\mathcal{T},P_{\mathcal{U}}(\mathcal{T})^{\perp}}$ on $\mathcal{R}(L^{*})$ and $A_{1}=0$ on $\mathcal{R}(L^{*})^{\perp}$. Consequently, with (24), we obtain

[TABLE]

and

[TABLE]

In order to prove that (22) holds for $B_{1}$, we show that \mathcal{R}(U^{*})^{\perp}=\mathcal{N}\big{(}G^{\frac{\dagger}{2}}\big{)}. The set $\mathcal{R}(U^{*})$ is closed because $(u_{j})_{j\in\mathbb{N}}$ is a frame for $\mathcal{U}$, and therefore $\mathcal{R}(G)=\mathcal{R}(U^{*}U)=\mathcal{R}(U^{*})$. Since $\mathcal{N}(A^{\dagger})=\mathcal{N}(A^{*})$, we obtain

[TABLE]

That the operator $B_{1}$ has the representation (iii) follows from the fact that the minimal norm element of $K$ (cf. (23)) is obtained by the Moore-Penrose pseudoinverse, i.e. $\hat{c}=(G^{\frac{\dagger}{2}}U^{*}T)^{\dagger}G^{\frac{\dagger}{2}}d$.

By Theorem 3.1 $A_{1}=T(L^{*}T)^{\dagger}$ solves the following least squares problems: for every $\tilde{d}\in\ell^{2}$, in particular for $\tilde{d}=G^{\frac{\dagger}{2}}d$ with $d\in\ell^{2}(\mathbb{N})$, the element $\tilde{f}=A_{1}\tilde{d}=A_{1}G^{\frac{\dagger}{2}}d=B_{1}d$ solves

[TABLE]

∎

Remark 3.4**.**

The approximation or reconstruction of $f$ from $U^{*}f$ by means of $Q_{0}$ can be understood as a two-step procedure: first the input data $U^{*}f$ are preprocessed with $G^{\frac{\dagger}{2}}$, the result is $G^{\frac{\dagger}{2}}U^{*}f=L^{*}f=(\langle f,S^{\frac{\dagger}{2}}u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}$. Then $\tilde{f}$ is produced with $T(L^{*}T)^{\dagger}$, which is again the synthesis operator of a frame.* *

The next result shows that the operator $Q_{0}$ has the smallest possible quasi-optimality constant.

Theorem 3.5**.**

Let $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$ and let $Q_{0}$ be defined as in Theorem 3.3(i). If $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ is a perfect reconstruction operator, then

[TABLE]

*or, equivalently, $\|QU^{*}\|_{\rm op}\geqslant\|Q_{0}U^{*}\|_{\rm op}$. *

Proof.

We recall that $\mu(Q)$ is the smallest $\alpha$ such that for every $f\in\mathcal{H}$

[TABLE]

and we may assume that $\mu(Q)<\infty$. Let $g\in\mathcal{T}$ and $u^{\perp}\in\mathcal{U}^{\perp}$. Then inequality (25) implies that $QU^{*}g=g$ and $QU^{*}u^{\perp}=0$. This means that $QU^{*}f=P_{\mathcal{T},\mathcal{U}^{\perp}}f$ for $f\in\mathcal{T}\oplus\mathcal{U}^{\perp}$. Since by Corollary 2.5 the sharp upper bound is

[TABLE]

we conclude that $\alpha\geqslant\frac{1}{\cos(\varphi_{\mathcal{T},\mathcal{U}})}$.

It remains to prove that $\mu(Q_{0})=\frac{1}{\cos(\varphi_{\mathcal{T},\mathcal{U}})}$. Since $Q_{0}U^{*}=P_{\mathcal{T},P_{\mathcal{U}}(\mathcal{T})^{\perp}}$, Corollary 2.5 implies that $\mu(Q_{0})=\frac{1}{\cos(\varphi_{\mathcal{T},P_{\mathcal{U}}(\mathcal{T})})}$. We observe that

[TABLE]

using definition (6) for the first equality and last equality. Thus $\mu(Q)\geqslant\mu(Q_{0})$. ∎

3.3. Combinations of $Q_{0}$ and $Q_{1}$

The operators $Q_{0}$ and $Q_{1}$ optimize different performance metrics, specifically $Q_{1}$ is most stable with respect to noisy data, and $Q_{0}$ is optimal with respect to the deviation of the target function from the reconstruction space. It is natural to interpolate between these two operators and to try to define mixtures $Q_{\lambda}$ such that $\|Q_{1}\|_{\rm op}\leqslant\|Q_{\lambda}\|_{\rm op}\leqslant\|Q_{0}\|_{\rm op}$ and $\mu(Q_{0})\leqslant\mu(Q_{\lambda})\leqslant\mu(Q_{1})$, $\lambda\in(0,1)$. To do this, we procede as follows.

For $\lambda\in[0,1]$ we define

[TABLE]

and

[TABLE]

where $I$ denotes the identity operator on $\mathcal{H}$ and on $\ell^{2}(\mathbb{N})$ respectively, $S_{1}=UU^{*}$ the frame operator and $G_{1}:=U^{*}U$ the Gramian of the frame $(u_{j})_{j\in\mathbb{N}}$ for $\mathcal{U}$. For $\lambda>0$ $M_{\lambda}$ is invertible on $\mathcal{H}$, $\mathcal{U}$ is an invariant subspace of $M_{\lambda}$ and $\Sigma_{\lambda}$ is invertible on $\ell^{2}(\mathbb{N})$. We now set

[TABLE]

The next lemma describes the properties of the new frame $(u_{\lambda,j})_{j\in\mathbb{N}}$.

Lemma 3.6**.**

Let $(u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{U}$ with frame bounds $A$ and $B$. Fix $\lambda\in(0,1]$, and let $(u_{\lambda,j})_{j\in\mathbb{N}}$ be defined by (28).

Then $(u_{\lambda,j})_{j\in\mathbb{N}}$ is a frame for $\mathcal{U}$ with frame bounds $\frac{A}{\lambda+(1-\lambda)A}$ and $\frac{B}{\lambda+(1-\lambda)B}$, i.e., for every $f\in\mathcal{U}$

[TABLE]

Furthermore

[TABLE]

i.e., the operator $\Sigma_{\lambda}^{-\frac{1}{2}}U^{*}$ is the analysis operator of the frame $(u_{\lambda,j})_{j\in\mathbb{N}}$ for $\mathcal{U}$.

Proof.

Using $S_{1}=UU^{*}$ for the frame operator of $(u_{j})_{j\in\mathbb{N}}$, we obtain the following: for $f\in\mathcal{U}$

[TABLE]

Let f(x)=x\big{(}\lambda+(1-\lambda)x\big{)}^{-1} on $[0,\infty)$. Then $f$ is increasing and $S_{1}M_{\lambda}^{-1}=f(S_{1})$. Consider the restriction $S_{1}:\mathcal{U}\rightarrow\mathcal{U}$ to the subspace $\mathcal{U}$. If $\sigma(S_{1})\subseteq[A,B]\subseteq(0,\infty)$, then by the spectral theorem $\sigma(S_{1}M_{\lambda}^{-1})=\sigma(f(S_{1}))\subseteq[f(A),f(B)]$. Combining this with (31) implies the frame inequality (29).

Identity (30) is proven in Lemma 4.4. ∎

Theorem 3.7**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be closed subspaces of a separable Hilbert space $\mathcal{H}$ such that $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$. Let $(u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{U}$ and $(t_{k})_{k\in\mathbb{N}}$ be a frame for $\mathcal{T}$. For $0<\lambda\leqslant 1$ let $L_{\lambda}$ be the synthesis operator of the frame $(u_{\lambda,j}=M_{\lambda}^{-\frac{1}{2}}u_{j})_{j\in\mathbb{N}}$ for $\mathcal{U}$ and $S_{\lambda}=L_{\lambda}L_{\lambda}^{*}$ the corresponding frame operator.

Consider the following operators:

1. (i)

$C_{1}:=T(\Sigma_{\lambda}^{-\frac{1}{2}}U^{*}T)^{\dagger}\Sigma_{\lambda}^{-\frac{1}{2}}$. 2. (ii)

Let the operator $C_{2}$ be defined on $\mathcal{R}(U^{*})$ by

[TABLE]

and

[TABLE]

Consequently, $C_{2}$ is independent of the particular choice of the reconstruction frame $(t_{k})_{k\in\mathbb{N}}$ for $\mathcal{T}$. 3. (iii)

For $d\in\ell^{2}(\mathbb{N})$ set $C_{3}d=\sum_{k=1}^{\infty}\hat{c}_{k}t_{k}$ with $\hat{c}=(\hat{c}_{k})_{k\in\mathbb{N}}$ being the minimal norm element of the set

[TABLE]

Then these operators are equal, $Q_{\lambda}:=C_{1}=C_{2}=C_{3}$, and $\tilde{f}=Q_{\lambda}d$ is the unique solution of the least squares problem

[TABLE]

Proof.

We apply Theorem 3.1 to the frames $(u_{\lambda,j})_{j\in\mathbb{N}}$ for $\mathcal{U}$ and $(t_{k})_{k\in\mathcal{N}}$ for $\mathcal{T}$.

Let $L_{\lambda}^{*}$ be the analysis operator of $(u_{\lambda,j})_{j\in\mathbb{N}}$ and set $A_{1}=T(L_{\lambda}^{*}T)^{\dagger}$. Since $A_{1}$ has the equivalent representation (ii) of Theorem 3.1

[TABLE]

and $A_{1}=0$ on $\mathcal{R}(L_{\lambda}^{*})^{\perp}$. This means that

[TABLE]

Since $M_{\lambda}$ is invertible and $L_{\lambda}^{*}=U^{*}M_{\lambda}^{-1/2}$ we have $\mathcal{R}(L_{\lambda}^{*})=\mathcal{R}(U^{*}M_{\lambda}^{-1/2})=\mathcal{R}(U^{*})$ and (32) holds for $C_{1}$. To prove that $C_{1}=0$ on $\mathcal{R}(U^{*})^{\perp}$, we use three algebraic properties of kernels: If $A,B,C$ are bounded, $A$ pseudo-invertible and $C$ invertible, then $\mathcal{N}(A^{\dagger})=\mathcal{N}(A^{*})$, $\mathcal{N}(AB)\supseteq\mathcal{N}(B)$ and $\mathcal{N}(AC)=C^{-1}\mathcal{N}(A)$. Consequently the kernel of $C_{1}$ is

[TABLE]

Thus $C_{1}=0$ on $\mathcal{R}(U^{*})^{\perp}$, which is (33).

For showing $C_{1}=C_{3}$ and (34) we repeat the proof of Theorem 3.3 verbatim. ∎

The following Lemma gives a useful upper bound on the quasi-optimality constant of the operators $Q_{\lambda}$.

Lemma 3.8**.**

Let $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$ and let $Q_{\lambda}$ be defined as in Theorem 3.7. Then the quasi-optimality constant $\mu(Q_{\lambda})=\|Q_{\lambda}U^{*}\|_{\rm op}$ is bounded by

[TABLE]

Proof.

By Lemma 3.6 $(u_{j,\lambda})_{j\in\mathbb{N}}$ is a frame for $\mathcal{U}$ with frame bounds $\frac{A}{\lambda+(1-\lambda)A}$ and $\frac{B}{\lambda+(1-\lambda)B}$, and synthesis operator $L_{\lambda}=\Sigma_{\lambda}^{-\frac{1}{2}}U^{*}$. Using (18) for the frame $(u_{j,\lambda})_{j\in\mathbb{N}}$ we infer (35). ∎

We observe that the upper bound in (35) is decreasing for $\lambda\rightarrow 0$, and for $\lambda=0$ the upper bound coincides with the operator norm $\|Q_{0}U^{*}\|_{\rm op}=\frac{1}{\cos(\varphi_{\mathcal{T},\mathcal{U}})}$. Unfortunately we do not know yet how to obtain a meaningful bound on the operator norm of $Q_{\lambda}$.

Remark 3.9**.**

In [12] the authors consider a regularization term (Tikhonov regularization) . The reconstruction operators corresponding to such a regularized least squares fit do not fulfill $Q(\langle f,u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}=f$ for $f\in\mathcal{T}$ (see [12, equation (7)]), and therefore do not belong to the class of reconstruction operators analyzed in this paper.**

3.4. Numerical calculation of the coefficients

We now discuss how to calculate the coefficients of the reconstructions for finite sequences $(u_{j})_{j=1}^{n}$ and $(t_{k})_{k=1}^{m}$ in (a possibly infinite-dimensional space) $\mathcal{H}$. The reconstruction vectors $(t_{k})_{k=1}^{m}$ are assumed to be linearly independent. Let $d\in\mathbb{C}^{n}$ denote the vector consisting of the noisy measurements

[TABLE]

By Theorem 3.7 the approximation $\tilde{f}=Q_{\lambda}d$ of $f$ is given by the linear combination

[TABLE]

with expansion coefficients

[TABLE]

If we formulate this least squares problem in terms of the normal equations, we have to solve

[TABLE]

We observe that the cross-Gramian $U^{*}T\in\mathbb{C}^{n\times m}$ is the matrix with entries

[TABLE]

and the matrix $\Sigma_{\lambda}\in\mathbb{C}^{n\times n}$ is given by

[TABLE]

For the solution of an overdetermined least squares problem one may use a direct method, such as the QR decomposition with pivoting with an operation count of $\mathcal{O}(nm^{2})$. Alternatively, one may approximate the solution of (36) up to a given precision $\varepsilon>0$ by means of iterative methods, such as the conjugate gradient method applied to the normal equations with an operation count $\mathcal{O}(\log(\varepsilon)nm)$. A concrete realization is the LSQR algorithm, see [37].

The convergence of the conjugate gradient iteration depends fundamentally on the condition number $\kappa(R_{\lambda})$ of the matrix $R_{\lambda}=T^{*}U\Sigma_{\lambda}^{-1}U^{*}T$ in (37) (where $\kappa(A)=\|A\|_{\rm op}\,\|A^{-1}\|_{\rm op}$). The following lemma offers an estimate for the condition number under the additional condition that the reconstruction space is spanned by an orthonormal set. This is a common practice in many applications [27, 6, 9, 10, 11].

Lemma 3.10**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be closed subspaces of a separable Hilbert space $\mathcal{H}$ such that $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$. Let $(u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{U}$ with frame bounds $A$ and $B$, and let $(t_{k})_{k\in\mathbb{N}}$ be an orthonormal basis for $\mathcal{T}$. Set $R_{\lambda}=T^{*}U\Sigma_{\lambda}^{-1}U^{*}T$.

Then

[TABLE]

Proof.

From Lemma 3.6 we know that $(u_{j,\lambda}=M_{\lambda}^{-\frac{1}{2}}u_{j})_{j\in\mathbb{N}}$ is a frame for $\mathcal{U}$ with frame bounds $\frac{A}{\lambda+(1-\lambda)A}$ and $\frac{B}{\lambda+(1-\lambda)B}$, and the synthesis operator $L_{\lambda}=\Sigma_{\lambda}^{-\frac{1}{2}}U^{*}$. Using $\|Tc\|_{\mathcal{H}}=\|c\|_{2}$ and (10) for the frame $(u_{j,\lambda})_{j\in\mathbb{N}}$ instead of $(u_{j})_{j\in\mathbb{N}}$ we infer that

[TABLE]

Since $\kappa(R_{\lambda})=\kappa(L_{\lambda}^{*}L_{\lambda})=(\|L_{\lambda}\|_{\rm op}\,\|L_{\lambda}^{\dagger}\|_{\rm op})^{2}=\kappa(L_{\lambda})^{2}$, inequality (38) is now a direct consequence of (39). ∎

Remark 3.11**.**

1. We observe that the bound for $\kappa(R_{\lambda})$ on the right-hand side of (38) is increasing in $\lambda$ and we expect that also $\kappa(R_{\lambda_{1}})\leqslant\kappa(R_{\lambda_{2}})$ for $\lambda_{1}\leqslant\lambda_{2}$. This has been tested experimentally in Section 4.

2. Note that for $\lambda>0$ the solution of the original least squares problem $\underset{c\in\ell^{2}(\mathbb{N})}{\min}\|U^{*}Tc-d\|_{2}$ and of $\underset{c\in\ell^{2}(\mathbb{N})}{\min}\|\Sigma_{\lambda}^{-\frac{1}{2}}U^{*}Tc-\Sigma_{\lambda}^{-\frac{1}{2}}d\|_{2}$ are distinct in general. This is an important difference to classical preconditioning of square systems, where the solution of the original and the preconditioned system coincide.

3. One may interpret the introduction of $\Sigma_{\lambda}^{-1/2}$ as a form of preprocessing of the measurement vector $d$. In most sampling problems the preprocessing is by a diagonal matrix [23, 5, 24, 25, 26, 40, 1, 3, 2, 4, 11], where the entries are called “adaptive weights” or “density compensation factors”. The use of non-diagonal matrices seems to be a new idea.

4. The use of more general matrices for preprocessing is very promising, but requires additional numerical considerations. To achieve a small numerical complexity, one needs to approximate $\Sigma_{\lambda}^{-1}$ by a simpler matrix $V_{\lambda}$ and then solve the normal equations

[TABLE]

This question will be pursued in future work. **

3.5. Conditions for the approximations to coincide

While in general the reconstruction operators $Q_{1}$, $Q_{0}$ and $Q_{\lambda}$ are different, they coincide in several situations.

Lemma 3.12**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be closed subspaces of a separable Hilbert space $\mathcal{H}$ and let $(u_{j})_{j=1}^{n}$ be a frame for $\mathcal{U}$. If $\mathcal{T}\oplus\mathcal{U}^{\perp}=\mathcal{H}$ and $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ is a bounded, perfect reconstruction operator, then

[TABLE]

Consequently for $\lambda\in[0,1]$

[TABLE]

Proof.

As in the proof of Theorem 3.5 we see that $QU^{*}g=g$ for $g\in\mathcal{T}$ and $QU^{*}u^{\perp}=0$ for $u^{\perp}\in\mathcal{U}^{\perp}$ imply that $\mathcal{R}(QU^{*})\supseteq\mathcal{T}$ and $\mathcal{N}(QU^{*})\supseteq\mathcal{U}^{\perp}$. Since by assumption $\mathcal{T}\oplus\mathcal{U}^{\perp}=\mathcal{H}$, this proves (40).

Since $Q_{1}c=Q_{0}c=Q_{\lambda}c=0$ for $c\in\mathcal{R}(U^{*})^{\perp}$, this implies (41). ∎

The decomposition $\mathcal{T}\oplus\mathcal{U}^{\perp}=\mathcal{H}$ is the general assumption for consistent sampling [19, 20, 22, 17, 16, 21]. In finite dimensions the assumption $\mathcal{T}\oplus\mathcal{U}^{\perp}=\mathcal{H}$ is fulfilled only if $\dim(\mathcal{T})=\dim(\mathcal{U})$ and $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$ [9, Lemma 3.7]. In case of linearly independent sampling and reconstruction vectors, the condition $\dim(\mathcal{T})=\dim(\mathcal{U})$ requires as many sampling as reconstruction vectors. In other words, in the critical case (between overdetermined and underdetermined) all reconstruction operators coincide.

Theorem 3.13**.**

Let $\mathcal{T}$ and $\mathcal{U}$ be closed subspaces of $\mathcal{H}$ such that $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$. If $(u_{j})_{j\in\mathbb{N}}$ is a tight frame for $\mathcal{U}$, then for $\lambda\in[0,1]$

[TABLE]

Proof.

Since $Q_{0}=Q_{1}=Q_{\lambda}=0$ on $\mathcal{R}(U^{*})^{\perp}$, it is sufficient to show that $Q_{1}U^{*}=Q_{0}U^{*}=Q_{\lambda}U^{*}$. Using the frame operator $S_{\lambda}$ of the frame $(u_{\lambda,j}=M_{\lambda}^{-\frac{1}{2}}u_{j})_{j\in\mathbb{N}}$ of $\mathcal{U}$ (cf. (26)), we have $Q_{\lambda}U^{*}=P_{\mathcal{T},S_{\lambda}(\mathcal{T})^{\perp}}$ (by Theorem 3.7).

Since $(u_{j})_{j\in\mathbb{N}}$ is a tight frame for $\mathcal{U}$, its frame operator is $S_{1}=AP_{\mathcal{U}}$ for some $A>0$. Consequently,

[TABLE]

and $M_{\lambda}^{-1/2}u_{j}=(\lambda+(1-\lambda)A)^{-1/2}u_{j}$ is just a constant multiple of the original tight frame. Therefore $(M_{\lambda}^{-1/2}u_{j})_{j\in\mathbb{N}}$ is again a tight frame for every $\lambda\in[0,1]$, $S_{\lambda}(\mathcal{T})=P_{\mathcal{U}}(\mathcal{T})$ and $Q_{\lambda}U^{*}=P_{\mathcal{T},S_{\lambda}(\mathcal{T})^{\perp}}=P_{\mathcal{T},P_{\mathcal{U}}(\mathcal{T})^{\perp}}$ is independent of $\lambda$. Consequently, $Q_{0}=Q_{\lambda}=Q_{1}$.

∎

3.6. Stability with respect to a biased objects

In [2, 1, 5, 3] and also [24, 25, 26, 4, 23, 11] a notion of stability with respect to a bias in the measured object is considered (in the latter stated in terms of a frame inequality). This means that the measurements are made on the vector $f+\Delta f$ instead of the correct $f$, and $\Delta f\in\mathcal{H}$ is the bias or object uncertainty. In this case the error estimate is of the form

[TABLE]

It is important to understand the conceptual difference between (42) and (5). The error estimate (5) treats the error arising from perturbed or noisy measurements $U^{*}f+l$. Estimate (42) treats the uncertainty of the target function (object uncertainty) and assumes that the exact measurements of the biased function $f+\Delta f$ are available. Since the operator $Q_{0}$ has the smallest possible quasi-optimality constant $\mu(Q_{0})$ and since $\mu(Q_{0})=\|Q_{0}U^{*}\|_{\rm op}$, Theorem 3.5 yields the following corollary.

Corollary 3.14**.**

Let $\cos(\varphi_{\mathcal{T},\mathcal{U}})>0$ and let $Q_{0}$ be defined as in Theorem 3.3. If an operator $Q:\ell^{2}(\mathbb{N})\rightarrow\mathcal{T}$ satisfies for $f\in\mathcal{H}$ and $\Delta f\in\mathcal{H}$

[TABLE]

for some $0<\beta_{i}<\infty$, then $\beta_{i}\geqslant\mu(Q_{0})$, $i=1,2$.

Consequently, if we restrict ourselves to linear mappings, Corollary 3.14 shows that $Q_{0}$ is optimal for the problem considered in [2, 1, 5, 3, 24, 25, 26, 4, 23, 11]

4. Numerical experiments for reconstruction from Fourier measurements

In this section, we apply the various reconstruction methods to the reconstruction of a compactly supported function from non-uniform Fourier samples. This approximation problem occurs in numerous applications, for example, radial sampling of the Fourier transform is used in MRI and CT, see [31].

From the given data $\hat{f}(\omega_{j}),j=-n,\dots,n,$ of a compactly supported function, we calculate the Fourier coefficients $\hat{f}(k),k=-m,\dots,m$, of $f$ and construct a final approximation by a truncated Fourier series. This is the uniform resampling problem, see [43, 9]. If $f$ is smooth and periodic, then the Fourier series converges exponentially fast. However, if $f$ is non-periodic or discontinuous, then the Fourier series of $f$ converges slowly and also suffers from the Gibbs phenomenon. Of course, for discontinuous or non-periodic functions the trigonometric polynomials of fixed degree are a bad choice for the reconstruction space. Since the function $f$ is unknown there will always be some model mismatch in practice, independent of the particular choice of the reconstruction vectors. Our objective in this section is not to choose optimal reconstruction functions, but rather to compare how the various reconstruction operators $Q_{\lambda}$ deal with the model mismatch in noisy regimes. We will see that a smart choice of the parameter $\lambda$ yields better approximations than the standard least square approximation (3).

We remark that the Gibbs phenomenon can be avoided by choosing a more appropriate reconstruction space, e.g., algebraic polynomials [27, 7] or wavelet expansions [6, 10].

4.1. Setup

We denote by $\langle f,g\rangle_{L^{2}}=\int_{-\infty}^{\infty}\!f(x)\overline{g(x)}\,\mathrm{d}x$ the standard inner product on the Hilbert space $L^{2}(\mathbb{R})$ and the Fourier transform $\mathcal{F}$ on $L^{2}(\mathbb{R})$ with normalization

[TABLE]

Let $\mathcal{H}$ be the subspace of $L^{2}(\mathbb{R})$ of functions with support in the interval $[-\frac{1}{2},\frac{1}{2}]$, i.e.,

[TABLE]

The given data are finitely many (non-uniform) noisy Fourier measurements

[TABLE]

where $l_{j}\in\mathbb{C}$ is additive noise. The noise $l_{j}$ is assumed to be i.i.d. Gaussian with variance (average power) $\sigma_{\ell}^{2}$ and signal-to-noise ratio (SNR) $\mathrm{SNR}=\frac{\|f\|_{L^{2}}^{2}}{\sigma_{\ell}^{2}(2n+1)}$.

The sampling space consists of the exponential functions

[TABLE]

so that indeed $d_{j}=\mathcal{F}f(\omega_{j})+l_{j}=\langle f,u_{j}\rangle_{L^{2}}+l_{j}$.

The sampling frequencies $\omega_{j}\in\mathbb{R}$ are chosen

[TABLE]

with $\delta_{j}\in[-2,2]$ i.i.d. and uniformly distributed over the interval $[-2,2]$. The reconstruction space for the resampling problem is spanned by the complex exponentials

[TABLE]

with $m\leqslant n$. In the numerical simulations we approximate the exponential function

[TABLE]

from the noisy Fourier measurements (44). For the reconstruction we use the operators $Q_{\lambda}$ of Theorem 3.7. This means that the vector $\hat{c}=[\hat{c}_{-m},\dots,\hat{c}_{m}]^{T}$ containing the coefficients of the approximation

[TABLE]

of $f$ is the solution of the least squares problem

[TABLE]

For the particular bases $(u_{j}),(t_{k})$ consisting of exponentials, the cross-Gramian $U^{*}T\in\mathbb{C}^{(2n+1)\times(2m+1)}$ has the entries

[TABLE]

and the preconditioning matrix $\Sigma_{\lambda}$ is given by the entries

[TABLE]

All results in this section have been averaged over $1000$ independent realizations of the sampling frequencies and the noise.

4.2. Noisy samples

In the first experiment we study the influence of the sampling rate $\frac{2m+1}{2n+1}$ and the SNR on the recovery performance of the operators $Q_{\lambda}$. We approximate the exponential function $f(x)=\operatorname{e}^{x}\chi_{[-\frac{1}{2},\frac{1}{2}]}(x)$ from $181$ noisy Fourier samples ($n=90$) and reconstruct in a space of trigonometric polynomials of degree $m=10,20,30,40$ (with dimension $2m+1$).

Table 1 lists the operator norm $\|Q_{\lambda}\|_{\rm op}$, the quasi-optimality constant $\mu(Q_{\lambda})$, the angle $\varphi_{\mathcal{T},\mathcal{U}}$, the condition number $\kappa(\Sigma_{\lambda}^{-\frac{1}{2}}U^{*}T)$ of the matrix of the least squares problem (46) and the relative error $\frac{\|Q_{\lambda}h-f\|_{L^{2}}}{\|f\|_{L^{2}}}$ for $\text{SNR}=\infty$, $\text{SNR}=20$dB and $\text{SNR}=10$dB and $m=10$ in (a), $m=20$ in (b) $m=30$ in (c) and $m=40$ in (d). All values are listed in the form $E\pm\sigma$ where $E$ is the expected value and $\sigma$ the standard deviation of the quantity, and rounded to the third decimal place.

By (2) the (absolute) reconstruction error depends both on the quasi-optimality constant $\mu(Q_{\lambda})$ and the operator norm $\|Q_{\lambda}\|_{\rm op}$. The numerical simulations support Theorem 3.5 asserting that the quasi-optimality constant $\mu$ is minimal for $\lambda=0$. As expected in view of Lemma 3.8 the quasi-optimality constant $\mu$ is increasing with $\lambda$. The angle $\varphi_{\mathcal{T},\mathcal{U}}$ is almost zero, so the reconstruction space is “almost contained” in the sampling space. Therefore $Q_{0}U^{*}$ is nearly identical to the orthogonal projection $P_{\mathcal{T}}$ onto $\mathcal{T}$. A small angle $\varphi_{\mathcal{T},\mathcal{U}}$ is essential for stable reconstruction. Taking for example $m=n$ leads to an angle close to $\frac{\pi}{2}$ (since the sampling frequencies are contained in the interval $[-\frac{n}{2}-2,\frac{n}{2}+2]$), which necessarily leads to an unstable scenario. Taking more measurements than reconstruction vectors is a common way to stabilize the reconstruction problem [8, 7, 10, 6, 33, 27, 5, 24, 25, 26, 40, 1, 3, 2, 4, 11]. The operator norm of $Q_{\lambda}$ is decreasing in $\lambda$, and the approximation becomes less sensitive to noise, with $Q_{1}$ being the most stable reconstruction in line with [9, Theorem 6.2.] and with Theorem 3.2. The intermediate reconstruction operators offer a trade-off between sensitivity to noise and the out-of-space contributions. A suitable choice of $\lambda$ then leads to more accurate reconstructions than $Q_{0}$ and $Q_{1}$. For example for $m=10$ and $\acs{SNR}=20$dB the average relative reconstruction error is $0.088$ for $Q_{0}$, $0.098$ for $Q_{1}$, but only $0.075$ for $Q_{0.1}$.

Interestingly, for $\acs{SNR}=10$dB we obtain a higher average approximation error for dimension $m=40$ than for $m=10,20,30$. Although the increase in dimension makes the distance $\|f-P_{\mathcal{T}}f\|_{L^{2}}$ smaller, the high irregularity in the sampling frequencies seems to lead to unstable scenarios. This correlates nicely with the increase of the operator norm $\|Q_{\lambda}\|_{\rm op}$ with increasing $m$.

We next observe that the condition number $\kappa(\Sigma_{\lambda}^{-\frac{1}{2}}U^{*}T)$ is increasing with $\lambda$, as anticipated in Lemma 3.10. As discussed in Section 3.4, this opens the possibility of finding non-diagonal weight matrices.

For Figure 1(a) we have determined the parameter $\lambda_{\rm opt}$ such that the relative reconstruction error is minimal, i.e, $\lambda_{\rm opt}=\mathrm{arg~{}min}\frac{\|Q_{\lambda}d-f\|_{L^{2}}}{\|f\|_{L^{2}}}$. We then plot the correlation between $\lambda_{\rm opt}$ and the signal-to-noise ratio. The plot confirms Theorems 3.5 and 3.2: for $\mathrm{SNR}\to\infty$ the optimal reconstruction is with $Q_{0}$, and for $\mathrm{SNR}\to-\infty$, the optimal reconstruction is with $Q_{1}$.

In Figure 1(b) we depict the approximations obtained by $Q_{0}$, $Q_{1}$ and $Q_{\lambda_{\rm{opt}}}$ for a single realization of the sampling frequencies and noise with $m=20$ and SNR $=20$dB. The optimal choice of the regularization parameter $\lambda_{\rm{opt}}$ yields a significantly better approximation than the standard least square approximation (with $Q_{1}$).

In Figure 1(c) we depict the quasi-optimality constant $\mu(Q_{\lambda})$ versus the operator norm $\|Q_{\lambda}\|_{\rm op}$ for $\lambda\in[0,1]$ for $m=20$ and SNR$=20$dB. The curve consists of the points $(\mu(Q_{\lambda}),\|Q_{\lambda}\|_{\rm op})$ for $\lambda\in[0,1]$. This curve exhibits a striking change of direction at a small value of $\lambda$, as is shown by the points corresponding to $\lambda=10^{-6},0.01,0.1$. The value of $\lambda$ near the edge of the curve (around $0.01$) yields a good trade-off between quasi-optimality and operator norm.

Figure 1(d) shows the relative reconstruction error $\frac{\|Q_{\lambda}d-f\|_{L^{2}}}{\|f\|_{L^{2}}}$ as a function of $\lambda$ for fixed $\mathrm{SNR}$. This is a typical $L$-curve known from many regularization procedures of ill-posed problems. The plots supports the interpretation of $\lambda$ as a regularization parameter.

4.3. Reconstruction from measurements of a biased function

We now assume that we are given a set of Fourier measurements of a perturbation of $f\in\mathcal{H}$

[TABLE]

The sampling frequencies $\omega_{j}$ are as in (45) with $\delta_{j}\in[-2,2]$. For each set of sampling frequencies we choose $\Delta f$ as a trigonometric polynomial

[TABLE]

The coefficients $a_{j}$ in (47) are i.i.d. Gaussian distributed with variance (average power) $\sigma^{2}$. Table 2 shows the relative error $\frac{\|Q_{\lambda}\tilde{d}-f\|_{L^{2}}}{\|f\|_{L^{2}}}$ for $\text{SNR}=\infty$, $\text{SNR}=20$dB and $\text{SNR}=10$dB and $m=10$ in (a), $m=20$ in (b), $m=30$ in (c) and $m=40$ in (d). In this case the most accurate reconstruction is always given by the reconstruction with the operator $Q_{0}$, thus confirming Corollary 3.14. In addition, with increasing dimension of the reconstruction space the distance $\|f-P_{\mathcal{T}}f\|_{L^{2}}$ is decreasing with $m$, and the relative error decreases up to $m=40$. For $m=50$ the angle between the sampling and reconstruction space is $\varphi_{\mathcal{T},\mathcal{U}}=0.309\pm 0.094$, which results in a significantly higher approximation error.

APPENDIX: FRAMES IN HILBERT SPACES

We need the definition of the Moore-Penrose pseudoinverse of an operator on a Hilbert space [15, Section 2.5]. We use the notation $\mathcal{R}(A)$ for the range, and $\mathcal{N}(A)$ for the null-space of the operator $A$.

Definition 4.1**.**

Let $\mathcal{H}$ and $\mathcal{W}$ be Hilbert spaces. If $A:\mathcal{W}\rightarrow\mathcal{H}$ is a bounded operator with a closed range $\mathcal{R}(A)$, then there exists a unique bounded operator $A^{\dagger}:\mathcal{H}\rightarrow\mathcal{W}$ satisfying

[TABLE]

The operator $A^{\dagger}$ is called the Moore-Penrose pseudoinverse of $A$.

For a sequence $(u_{j})_{j\in\mathbb{N}}$ we define the synthesis operator on the subspace of finite sequences by

[TABLE]

Definition 4.2**.**

(i) If $U$ can be extended to a bounded operator $U:\ell^{2}(\mathbb{N})\rightarrow\mathcal{H}$, $(u_{j})_{j\in\mathbb{N}}$ is called a Bessel sequence.

(ii) If $U$ is bounded $U:\ell^{2}(\mathbb{N})\rightarrow\mathcal{H}$, $(u_{j})_{j\in\mathbb{N}}$ and has closed range, $(u_{j})_{j\in\mathbb{N}}$ is called a frame for the subspace $\mathcal{U}=\overline{\textnormal{span}}(u_{j})_{j\in\mathbb{N}}$.

(iii) If $(u_{j})_{j\in\mathbb{N}}$ is a Bessel sequence, then the adjoint operator of $U$ is the analysis operator

[TABLE]

and $S=UU^{*}:\mathcal{H}\to\mathcal{H}$, $Sf=\sum_{j=1}^{\infty}\langle f,u_{j}\rangle_{\mathcal{H}}u_{j}$ is the frame operator of $(u_{j})$.

(iv) The sequence $(S^{\dagger}u_{j})_{j\in\mathbb{N}}\subseteq\mathcal{U}$ is the canonical dual frame in $\mathcal{U}$ , and every $f\in\mathcal{U}$ possesses the frame expansions

[TABLE]

with unconditional convergence of both series.

Lemma 4.3**.**

*Let $\mathcal{U}$ be a closed subspace of $\mathcal{H}$ and let $(u_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{U}$.The set *

[TABLE]

forms a tight frame for $\mathcal{U}$ with frame bound equal to $1$. The synthesis operator $M$ of the sequence $(S^{\frac{\dagger}{2}}u_{j})_{j\in\mathbb{N}}$ is given by $M:=S^{\frac{\dagger}{2}}U$, and

[TABLE]

Lemma 4.4 proves the following. Suppose that we are given the inner products $(\langle f,u_{j}\rangle_{\mathcal{H}})_{j\in\mathbb{N}}$ of an element $f\in\mathcal{H}$ with a frame $(u_{j})_{j\in\mathbb{N}}$ for $\mathcal{U}$ (a closed subspace of $\mathcal{H}$). Applying the operator $(U^{*}U)^{\frac{\dagger}{2}}$ to these measurements, we obtain the inner products of $f$ with the tight frame $(S^{\frac{\dagger}{2}}u_{j})_{j\in\mathbb{N}}$ for $\mathcal{U}$.

Lemma 4.4**.**

Let $\mathcal{U}$ be a closed subspace of $\mathcal{H}$ and $(u_{j})_{j\in\mathbb{N}}$ a frame for $\mathcal{U}$ with synthesis operator $U$, analysis operator $U^{*}$, Gramian $G=U^{*}U$ and frame operator $S=UU^{*}$. Then

[TABLE]

Thus, $G^{\frac{\dagger}{2}}U^{*}$ is the analysis operator of the tight frame sequence $(S^{\frac{\dagger}{2}}u_{j})_{j\in\mathbb{N}}$.

Proof.

Obviously for $k\in\mathbb{N}$

[TABLE]

Therefore,

[TABLE]

for every polynomial $p$. We are going to prove that there exists a sequence of polynomials $(p_{k})_{k\in\mathbb{N}}$, such that for $i=1,2$

[TABLE]

simultaneously for $M_{1}:=G$ and $M_{2}:=S$.

Let $A$ and $B$ denote the lower bound and upper frame bound of the frame sequence $(u_{j})_{j\in\mathbb{N}}$. From the lower frame bound $A$ we infer that for every $f\in\mathcal{U}=\mathcal{N}(UU^{*})^{\perp}=\mathcal{N}(S)^{\perp}$

[TABLE]

Consequently the set $\sigma(S)\backslash\{0\}$ is bounded below by $A$. Here $\sigma(S)$ denotes the spectrum of the operator $S$. The upper frame bound $B$ ensures that the set $\sigma(S)$ has the upper bound $B$. This shows that [math] is an isolated point of the spectrum, and that for ${K:=\{0\}\cup[A,B]}$ the function ${h:K\rightarrow\mathbb{R}}$

[TABLE]

is continuous on $K$. Since $\sigma(S)\cup\{0\}=\sigma(G)\cup\{0\}$, $h$ is also continuous on $\sigma(G)$.

By the Weierstrass approximation theorem there exists a sequence of polynomials $(p_{m})_{m\in\mathbb{N}}$, such that

[TABLE]

uniformly on $K$. By the continuous functional calculus

[TABLE]

simultaneously for $M_{1}:=G$ and $M_{2}:=S$ and $h(M_{i})=M_{i}^{\frac{\dagger}{2}}$ for $i=1,2$. ∎

Lemma 4.5**.**

[15, Lemma 5.3.6]** Let $(f_{j})_{j\in\mathbb{N}}$ be a frame for $\mathcal{H}$ with frame operator $S$ and let $f\in\mathcal{H}$. If $f$ has a representation $f=\sum_{j\in\mathbb{N}}c_{j}f_{j}$ for some coefficients $(c_{j})_{j\in\mathbb{N}}$, then

[TABLE]