Asymptotic confidence sets for the jump curve in bivariate regression problems

TL;DR

This paper develops asymptotic confidence sets for detecting edges in bivariate images, combining kernel estimators, M-estimation, and Gaussian approximations to provide reliable inference.

Contribution

It introduces a novel method for constructing uniform and point-wise confidence sets for image edges using rotated kernel differences and advanced probabilistic tools.

Findings

Confidence sets achieve correct coverage in simulations

Method effectively detects edges in real-world images

Finite-sample performance aligns with asymptotic theory

Abstract

We construct uniform and point-wise asymptotic confidence sets for the single edge in an otherwise smooth image function which are based on rotated differences of two one-sided kernel estimators. Using methods from M-estimation, we show consistency of the estimators of location, slope and height of the edge function and develop a uniform linearization of the contrast process. The uniform confidence bands then rely on a Gaussian approximation of the score process together with anti-concentration results for suprema of Gaussian processes, while point-wise bands are based on asymptotic normality. The finite-sample performance of the point-wise proposed methods is investigated in a simulation study. An illustration to real-world image processing is also given.

Figures

Tables

Table 1. Table 1: Root of the MSE of the standard deviation estimate in scenario ϕ 1 subscript italic-ϕ 1 \phi_{1} for some points x 𝑥 x if σ ~ = 0.5 ~ 𝜎 0.5 \tilde{\sigma}=0.5 or σ ~ = 0.9 ~ 𝜎 0.9 \tilde{\sigma}=0.9 . The last row indicates the mean of the RMSE for 64 points in the corresponding setting.

${\varphi }_1$	$\tilde{\sigma }=0.5$				$\tilde{\sigma }=0.9$
$x$	$n=128$	$n=196$	$n=256$	asymp. sd	$n=128$	$n=196$	$n=256$	asymp. sd
0.040	0.104	0.110	0.112	0.888	0.232	0.183	0.201	1.599
0.142	0.101	0.080	0.071	0.611	0.209	0.154	0.119	1.100
0.347	0.144	0.128	0.122	0.913	0.272	0.208	0.200	1.644
0.449	0.107	0.084	0.075	0.617	0.183	0.163	0.136	1.111
0.653	0.186	0.160	0.136	0.935	0.325	0.222	0.244	1.683
0.858	0.160	0.132	0.123	0.725	0.275	0.206	0.204	1.305
	0.148	0.124	0.111		0.264	0.195	0.176

Table 2. Table 2: Root of the MSE of the standard deviation estimation for some points x 𝑥 x if σ ~ = 0.5 ~ 𝜎 0.5 \tilde{\sigma}=0.5 resp.  σ ~ = 0.9 ~ 𝜎 0.9 \tilde{\sigma}=0.9 and scenario ϕ 2 subscript italic-ϕ 2 \phi_{2} . The last row indicates the mean of the RMSE for 64 points in the corresponding setting.

${\varphi }_2$	$\tilde{\sigma }=0.5$				$\tilde{\sigma }=0.9$
$x$	$n=128$	$n=196$	$n=256$	asymp. sd	$n=128$	$n=196$	$n=256$	asymp. sd
0.040	0.137	0.148	0.153	1.149	0.274	0.244	0.243	2.069
0.142	0.130	0.087	0.087	0.691	0.226	0.170	0.158	1.244
0.347	0.142	0.124	0.132	0.840	0.257	0.196	0.202	1.511
0.449	0.101	0.078	0.067	0.540	0.191	0.133	0.123	0.972
0.653	0.193	0.156	0.143	0.859	0.317	0.238	0.182	1.547
0.858	0.107	0.101	0.100	0.820	0.207	0.168	0.171	1.477
	0.143	0.119	0.111		0.263	0.200	0.182

Table 3. Table 3: Choice of t n subscript 𝑡 𝑛 t_{n} in ( 14 ).

	$\tilde{\sigma }=0.5$			$\tilde{\sigma }=0.9$
	$n=128$	$n=196$	$n=256$	$n=128$	$n=196$	$n=256$
${\varphi }_1$	0.37	0.34	0.335	0.07	0.001	0
${\varphi }_2$	0.4	0.37	0.25	0.14	0.1	0.06
$1/\sqrt{\ln \left(n\right)}$	0.45	0.44	0.42	0.45	0.44	0.42

Table 4. Table 4: Average coverage and width of the point-wise confidence intervals for the jump-location in ( 9 ) for σ ~ = 0.5 ~ 𝜎 0.5 \tilde{\sigma}=0.5 and σ ~ = 0.9 ~ 𝜎 0.9 \tilde{\sigma}=0.9 over 64 design points.

$\tilde{\sigma }=0.5$					$\tilde{\sigma }=0.9$
	95% nominal coverage		99% nominal coverage		95% nominal coverage		99% nominal coverage
	coverage	width	coverage	width	coverage	width	coverage	width
$n=128$
${\varphi }_1$	0.960	0.025	0.991	0.033	0.943	0.043	0.985	0.056
${\varphi }_2$	0.958	0.025	0.991	0.033	0.946	0.042	0.983	0.056
$n=196$
${\varphi }_1$	0.958	0.016	0.992	0.021	0.944	0.028	0.990	0.037
${\varphi }_2$	0.951	0.016	0.988	0.022	0.943	0.028	0.988	0.037
$n=256$
${\varphi }_1$	0.958	0.012	0.992	0.016	0.941	0.021	0.997	0.028
${\varphi }_2$	0.949	0.013	0.988	0.017	0.949	0.021	0.993	0.029

Table 5. Table 5: Average coverage and width of the uniform confidence bands for the jump-location in ( 14 ) for σ ~ = 0.5 ~ 𝜎 0.5 \tilde{\sigma}=0.5 and σ ~ = 0.9 ~ 𝜎 0.9 \tilde{\sigma}=0.9 .

$\tilde{\sigma }=0.5$					$\tilde{\sigma }=0.9$
	95% nominal coverage		99% nominal coverage		95% nominal coverage		99% nominal coverage
	coverage	width	coverage	width	coverage	width	coverage	width
$n=128$
${\varphi }_1$	0.943	0.051	0.986	0.059	0.955	0.062	0.999	0.071
${\varphi }_2$	0.959	0.049	0.984	0.059	0.945	0.065	0.999	0.075
$n=196$
${\varphi }_1$	0.945	0.032	0.988	0.038	0.954	0.039	0.999	0.045
${\varphi }_2$	0.957	0.032	0.986	0.039	0.948	0.042	1.000	0.050
$n=256$
${\varphi }_1$	0.949	0.025	0.987	0.028	0.951	0.030	0.993	0.032
${\varphi }_2$	0.953	0.025	0.994	0.033	0.955	0.032	0.999	0.039

Table 6. Table 6: Ratio between computed bias and estimated standard deviation for different points x 𝑥 x in scenario ϕ 1 subscript italic-ϕ 1 \phi_{1} . The last line contains the average ratio over 64 design points for σ ~ = 0.5 ~ 𝜎 0.5 \tilde{\sigma}=0.5 and σ ~ = 0.9 . ~ 𝜎 0.9 \tilde{\sigma}=0.9.

${\varphi }_1$	$\tilde{\sigma }=0.5$			$\tilde{\sigma }=0.9$
$x$	$n=128$	$n=196$	$n=256$	$n=128$	$n=196$	$n=256$
0.040	0.077	0.036	0.009	0.026	0.004	0.048
0.142	0.001	0.037	0.013	0.032	0.035	0.010
0.347	0.064	0.037	0.041	0.029	0.035	0.086
0.449	0.043	0.005	0.013	0.014	0.093	0.085
0.653	0.048	0.072	0.032	0.063	0.033	0.002
0.858	0.018	0.004	0.022	0.003	0.046	0.075
	0.033	0.034	0.028	0.029	0.050	0.049

Table 7. Table 7: Ratio between empirical bias and estimated standard deviation for different points x 𝑥 x in scenario ϕ 1 subscript italic-ϕ 1 \phi_{1} . The last line contains the average ratio over 64 design points for σ ~ = 0.5 ~ 𝜎 0.5 \tilde{\sigma}=0.5 and σ ~ = 0.9 . ~ 𝜎 0.9 \tilde{\sigma}=0.9.

${\varphi }_2$	$\tilde{\sigma }=0.5$			$\tilde{\sigma }=0.9$
$x$	$n=128$	$n=196$	$n=256$	$n=128$	$n=196$	$n=256$
0.040	0.091	0.121	0.051	0.037	0.021	0.042
0.142	0.227	0.267	0.172	0.059	0.116	0.096
0.347	0.181	0.216	0.030	0.041	0.144	0.073
0.449	0.091	0.337	0.189	0.061	0.305	0.063
0.653	0.120	0.095	0.079	0.020	0.081	0.117
0.858	0.218	0.223	0.161	0.093	0.149	0.116
	0.160	0.217	0.138	0.075	0.130	0.101

Equations

$$Y_{i_1,i_2}=m_{\varphi }({\mathbf{x}}_{{\mathbf{i}}_1,{\mathbf{i}}_2})+{\mathbf{\epsilon }}_{i_1,i_2},$$

$$\begin{array}{rl}{m}_{\varphi }(x,y)& =m(x,y)+j_{\tau ,\varphi }(x,y),\\ j_{\tau ,\varphi }(x,y)& =j_{\tau }(x,y)=\tau (x)1_{[0,\varphi (x)]}(y),\end{array}$$

$$\displaystyle\boldsymbol{\mathscr{R}}_{\psi}=\left(\begin{array}[]{cc}\cos(\psi)&\sin(-\psi)\\ \sin(\psi)&\cos(\psi)\end{array}\right),$$

$$K(\mathbf{z};\psi ,h)=K(h^{-1}{\mathcal{R}}_{-\psi }\mathbf{z})/h^2,$$

$${\hat{\text{M}}}_n(\mathbf{z};\psi ,h)=n^{-2}\sum _{i_1,i_2=1}^n{Y}_{i_1,i_2}K(\mathbf{z}-{\mathbf{x}}_{{\mathbf{i}}_1,{\mathbf{i}}_2};\psi ,h).$$

$$({\hat{\varphi }}_n(x),{\hat{\psi }}_n(x))\in \underset{y\in [h,1-h],\psi \in [-\pi /2,\pi /2]}{\mathrm{argmax}}{\hat{\text{M}}}_n((x,y{)}^{\top };\psi ,h).$$

$${\hat{\tau }}_n(x)={\hat{\text{M}}}_n((x,{\hat{\varphi }}_n(x){)}^{\top };{\hat{\psi }}_n(x),h).$$

$${\hat{\text{M}}}_n(\mathbf{z};\psi ,h)=\sum _{i_1,i_2=1}^n{Y}_{i_1,i_2}{\int }_{{△}_{i_1,i_2}}K(\mathbf{z}-\mathbf{u};\psi ,h)\mathrm{d}\mathbf{u},$$

$$m_{\varphi }(x,y)=m(x,y)+\sum _{k=1}^J{j}_{{\tau }_k,{\varphi }_k}(x,y),$$

$$j_{{\tau }_k,{\varphi }_k}(x,y)={\tau }_k(x)1_{[0,{\varphi }_k(x)]}(y),$$

$$m_G\{x_{\gamma }(t)\}=m\{x_{\gamma }(t)\}+\tau (\gamma )1_G\{x_{\gamma }(t)\},$$

$$({\hat{\varphi }}_n(\gamma ),{\hat{\psi }}_n(\gamma ))\in \underset{t\in [h,1-h],\psi \in [-\pi /2,\pi /2]}{\mathrm{argmax}}\sum _{i_1,i_2=1}^n{Y}_{i_1,i_2}K(x_{\gamma }(t)-{\mathbf{x}}_{{\mathbf{i}}_1,{\mathbf{i}}_2};\psi ,h).$$

$${\int }_{[-1,1]}{K}_1(x)\mathrm{d}x=1\text{and}{K}_1^{(j)}(-1)=K_1^{(j)}(1)=0.$$

$${\int }_0^1{K}_2(x)\mathrm{d}x$$

$$\displaystyle\sup_{x\in I}\ \big{|}\hat{\phi}_{n}(x)-\phi(x)\big{|}=O_{P}\{{\ln(n)^{1/2}}/{n}\}.$$

$$\displaystyle\sup_{x\in I}\ \big{|}\hat{\psi}_{n}(x)-\psi(x)\big{|}=O_{P}\{{\ln(n)^{1/2}}/{(nh)}\},\quad$$

$${\varphi }_n(x)\in \underset{y\in [h,1-h]}{\mathrm{argmax}}\underset{\psi \in [-\pi /2,\pi /2]}{\max }\mathrm{E}\{{\hat{\text{M}}}_n((x,y{)}^{\top };\psi ,h)\}.$$

$$n\{{\hat{\varphi }}_n(x)-{\varphi }_n(x)\}⇝\mathcal{N}[0,{\sigma }^2{V}_{\varphi }^S(x)/\{V_{\varphi }^H(x){\}}^2],$$

$$\begin{array}{rl}{V}_{\varphi }^H(x)& =\tau (x){\cos }^2\{\psi (x)\}{K}_2^{(1)}(0),\\ V_{\varphi }^S(x)& ={\sin }^2\{\psi (x)\}{\int }_{[-1,1{]}^2}\{K_1^{(1)}(z_1)K_2(z_2){\}}^2\mathrm{d}{z}_1\mathrm{d}{z}_2+{\cos }^2\{\psi (x)\}{\int }_{[-1,1{]}^2}\{K_1(z_1)K_2^{(1)}(z_2){\}}^2\mathrm{d}{z}_1\mathrm{d}{z}_2.\end{array}$$

$$\left[{\hat{\varphi }}_n(x)-\frac{{\hat{\sigma }}_n\sqrt{V_{\hat{\varphi }}^S(x)}{q}_{1-\alpha /2}}{n{V}_{\hat{\varphi }}^H(x)},{\hat{\varphi }}_n(x)+\frac{{\hat{\sigma }}_n\sqrt{V_{\hat{\varphi }}^S(x)}{q}_{1-\alpha /2}}{n{V}_{\hat{\varphi }}^H(x)}\right],$$

$$V_{\hat{\varphi }}^H(x)$$

$$V_{\hat{\varphi }}^S(x)$$

$$n\{{\hat{\varphi }}_n(x)-\varphi (x)\}-b_n⇝\mathcal{N}[0,{\sigma }^2{V}_{\varphi }^S(x)/\{V_{\varphi }^H(x){\}}^2].$$

$$\begin{array}{rl}{Z}_{n,\hat{\varphi }}^{\xi }(x)& =\frac{1}{nh\{V_{\hat{\varphi }}^S(x){\}}^{1/2}}\\ & \times \sum _{i_1,i_2=1}^n{\xi }_{i_1,i_2}⟨(\nabla K)(h^{-1}{\mathcal{R}}_{-{\hat{\psi }}_n(x)}\{(x,{\hat{\varphi }}_n(x){)}^{\top }-{\mathbf{x}}_{{\mathbf{i}}_1,{\mathbf{i}}_2}\}),(\sin {\hat{\psi }}_n(x),\cos {\hat{\psi }}_n(x){)}^{\top }⟩,\end{array}$$

$$\boldsymbol{\mathscr{R}}_{-\hat{\psi}_{n}(x)}\{(x,\hat{\phi}_{n}(x))^{\top}-\mathbf{x_{i_{1},i_{2}}}\}=\left(\begin{array}[]{c}\cos\{\hat{\psi}_{n}(x)\}(x-x_{i_{1}})+\sin\{\hat{\psi}_{n}(x)\}\{\hat{\phi}_{n}(x)-x_{i_{2}}\}\\ \cos\{\hat{\psi}_{n}(x)\}\{\hat{\phi}_{n}(x)-x_{i_{2}}\}-\sin\{\hat{\psi}_{n}(x)\}(x-x_{i_{1}})\end{array}\right).$$

$$\displaystyle\mathscr{Z}^{\xi}_{n,\widehat{\phi}}=\sup_{x\in I}\big{|}Z^{\xi}_{n,\widehat{\phi}}(x)\big{|}.$$

$$\liminf_{n}\ \Pr\left[\sup_{x\in I}\Big{|}{V_{\widehat{\phi}}^{H}(x)\{\hat{\phi}_{n}(x)-\phi(x)\}}/{\sqrt{V_{\widehat{\phi}}^{S}(x)}}\Big{|}\leq{(1+t_{n})\hat{\sigma}_{n}q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}})}/{n}\right]\geq 1-\alpha.$$

$$\{[c_{\varphi ,u}^{-}(x),c_{\varphi ,u}^{+}(x)]:x\in I\},c_{\varphi ,u}^{\pm }(x)={\hat{\varphi }}_n(x)\pm (1+t_n){\hat{\sigma }}_n\sqrt{V_{\hat{\varphi }}^S(x)}{q}_{1-\alpha }({\mathcal{Z}}_{n,\hat{\varphi }}^{\xi })/\{n{V}_{\hat{\varphi }}^H(x)\}.$$

$$\{{\hat{\sigma }}_n^2\times V_{\hat{\varphi }}^S(x)/V_{\hat{\varphi }}^H(x{)}^2{\}}^{1/2}$$

$$m(x,y)=\sin (y^2)\cos \{(x-1/2{)}^2\},\tau (x)=3{\sin }^2(10x)/10+1/2,$$

$${\varphi }_1(x)=1/4+x/2,{\varphi }_2(x)=-{\left(x-1/2\right)}^2+3/5,$$

Full text

1 Introduction

††©2019. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/.

Statistical methodology, in particular nonparametric regression, plays an important role nowadays in image reconstruction and denoising. In two-dimensional image functions, interest often focuses on the location of edges, i.e., discontinuity curves of the intensity function of the image. Edge estimation has been studied in the statistics literature from a minimax point-of-view in the monograph by Korostelev and Tsybakov [11]. Qiu [24] gives an overview of more practical reconstruction methods, which are partially motivated from the literature on computer vision.

Apart from mere estimation, statistical modeling allows for the construction of confidence regions, in which the object of interest is located with high probability. In this paper, we construct asymptotic confidence sets for the single edge in an otherwise smooth image function based on the rotational difference kernel method by Qiu [22] and Müller and Song [15], which is related to the Sobel edge detector from the image processing literature [23].

Confidence sets are by now well-developed for various problems in nonparametric statistics, e.g., for nonparametric density estimation [2, 9, 4], smooth regression functions [6, 18], and deconvolution and errors-in-variables problems [3, 20, 5]. Mammen and Polonik [13] and Qiao and Polonik [21] focus on more geometrical features, and construct confidence regions for density level sets and the density ridge, respectively.

When studying nonparametric regression problems with discontinuities, the focus may either be on the detection of potential jumps, or, assuming the existence of jumps, on estimation and the construction of confidence sets. In the univariate setting with a regression curve having a single jump, the latter problem was studied under various design assumptions by Müller [14], Loader [12], Gijbels et al. [8] and Seijo and Sen [26], among others. The problem of jump detection was additionally addressed, e.g., in [16, 30, 28, 19].

In bivariate problems, Korostelev and Tsybakov [11], Müller and Song [15], Qiu and Yandell [25], Qiu [23], and Garlipp and Müller [7] focus on the estimation of a given discontinuity curve, while Wang [29], Qiu and Yandell [25], Kang and Qiu [10] and Qiu [23] also investigate edge detection. In the latter three papers, the set of jump-location curves is estimated as a point-set, where a certain criterion function exceeds a threshold value.

However, currently there seem to be no methods available to construct a confidence set for the edge in the bivariate case. Our approach to this problem uses the criterion function of the rotational difference kernel method [22, 15], for which we obtain necessary additional kernel conditions for asymptotic normality. The uniform confidence bands then rely on a Gaussian approximation of the score process together with anti-concentration results for suprema of Gaussian processes from Chernozhukov et al. [4], while point-wise bands are based on asymptotic normality. A technical difficulty in the problem are the distinct rates of the estimators of location and slope. As a byproduct of our analysis, we obtain uniform rates of convergence for the estimators of the jump-location-curve, the slope-curve as well as the height-curve which are optimal up to a logarithmic factor.

The paper is structured as follows. In Section 2 we introduce the model as well as the estimators for location and slope of the edge. The main theoretical results can be found in Section 3. Outlines of the proofs for these results are provided in Section 6. Section 4 contains a simulation study and an illustrative application of the proposed method to a real-world image, while Section 5 concludes. Detailed proofs are provided in the Online Supplement [1].

We shall use the following notation. If a sequence of random variables $X_{1},X_{2},\ldots$ converges in probability to a random variable $X$, we write $X_{n}\stackrel{{\scriptstyle\text{P}}}{{\to}}X$ or $X_{n}=X+o_{P}(1)$. If the convergence is in distribution, we write $X_{n}\rightsquigarrow X$. Let $\lambda_{d}$ denote the $d$-dimensional Lebesgue measure. For $d\in\mathbb{N},$ $\mu\in\mathbb{R}^{d},\Sigma\in\mathbb{R}^{d\times d}$ symmetric and positive definite, let $\mathcal{N}_{d}(\mu,\Sigma)$ be the normal distribution with expectation $\mu$ and covariance matrix $\Sigma.$ For a vector $\mathbf{a}\in\mathbb{R}^{d}$, we denote by ${\rm diag}(\mathbf{a})$ the $d\times d$ diagonal matrix with diagonal entries $\mathbf{a}$. Given $\alpha\in(0,1)$, we denote by $q_{\alpha}(X)$ the $\alpha$-quantile of a random variable $X$ resp.  by $q_{\alpha}(Q)$ the $\alpha$-quantile of a distribution $Q$. Let $\langle\,\cdot,\cdot\,\rangle$ denote the Euclidean scalar product on $\mathbb{R}^{d}$.

2 Model and notation

We consider a boundary fragment of a noisy gray scale image, in which real-valued random variables $Y_{i_{1},i_{2}}$ are observed according to the model defined, for all $(i_{1},i_{2})\in\{1,\ldots,n\}\times\{1,\ldots,n\}$, by

[TABLE]

where $\mathbf{x_{i_{1},i_{2}}}=(x_{i_{1}},x_{i_{2}})^{\top}$ form a deterministic, regular rectangular grid in $[0,1]^{2},$ and $m_{\phi}$ is an unknown square-integrable function on $[0,1]^{2},$ which is sufficiently smooth besides a discontinuity curve $\phi$. Specifically, we assume that $m_{\phi}$ is of the form

[TABLE]

where $m:[0,1]^{2}\to\mathbb{R}$ is the smooth part of the image, $\tau:[0,1]\to\mathbb{R}_{+}$ the jump-height curve and $\phi:[0,1]\to(0,1)$ the jump-location curve.

To estimate $\phi(x)$, define the rotation matrix

[TABLE]

where $\psi\in\mathbb{R}$. Further, for a bandwidth $h>0$ and $\mathbf{z}=(z_{1},z_{2})^{\top}\in\mathbb{R}^{2}$, consider the rotated difference kernel

[TABLE]

where $K(\mathbf{z})=K(z_{1},z_{2})=K_{1}(z_{1})K_{2}(z_{2})$ is a product kernel of univariate kernel functions $K_{1}$ and $K_{2}$, and $K_{1}$ is even while $K_{2}$ is odd and hence corresponds to the difference of two one-sided kernels.

Define for $\mathbf{z}\in[0,1]^{2}$ and $\psi\in[-\pi/2,\pi/2]$ the contrast process with Priestley–Chao-type weights as

[TABLE]

For an $x\in(0,1)$ denote by $\psi(x)=\arctan\{\phi^{\prime}(x)\}$ the slope of the tangent at $\phi(x)$. An estimator for the bivariate parameter $(\phi(x),\psi(x))$ is then given by

[TABLE]

Our estimator for the jump-height at $x$ is given by

[TABLE]

An illustration of the mechanism in (7) is given in Figure 1. The line within the kernel window indicates that the odd kernel $K_{2}$ corresponds to the difference of two one-sided kernels. In order to maximize the contrast $\hat{\text{M}}_{n}(\mathbf{z};\psi,h)$, that is the difference of two one-sided estimators, for fixed $x\in(0,1)$ the kernel window is rotated so that the red line is tangential to the edge.

Remark 1**.**

The general idea of this estimator was stated in [22], and in [7] the estimator was made precise by changing the order of scaling and rotating. As initially proposed by [22] one could also use a Gasser–Müller type version of the rotational difference kernel estimator defined by

[TABLE]

where $\triangle_{i_{1},i_{2}}=[(i_{1}-1)/n,i_{1}/n)\times[(i_{2}-1)/n,i_{2}/n)$ denotes a design square. However, due to the rectangular fixed design it is easier to analyze the contrast function in (6) instead.

Remark 2** (Several jump curves).**

Also of interest for applications is an extension to several jump-location-curves, viz.

[TABLE]

and, for all $k\in\{1,\ldots,J\}$,

[TABLE]

where $J\in\mathbb{N}$, $m:[0,1]^{2}\to\mathbb{R}$ is the smooth part of the image, $\tau_{k}:[0,1]\to\mathbb{R}_{+}$ are the jump-height curves and $\phi_{k}:[0,1]\to(0,1)$ are the jump-location-curves. In order to guarantee identification, one has to assume that the images of the jump curves are well separated, i.e., that $B_{\rho}(\phi_{i}[0,1])\cap B_{\rho}(\phi_{j}[0,1])=\emptyset$ for $i,j\in\{1,\ldots,J\}$ with $i\neq j$ and some $\rho>0$, where $B_{\rho}(A)$ denotes the $\rho$-neighborhood of a set $A\subseteq\mathbb{R}$. The theoretical concepts below should be transferable with additional notational effort (see, e.g., Remark 2.6 in [22]) and confidence bands could be constructed based on a Bonferroni-correction. Section 4 illustrates this idea by means of a simulated data example.

Remark 3** (Closed jump curves).**

The rotated difference kernel method can also be applied to closed curves as follows. Let $G\subseteq\{x\in\mathbb{R}^{2}:\ \|x\|_{2}\leq 1\}$ be a nonempty, compact, convex set with $0\in\text{int}(G)$ and assume that the image function is given by

[TABLE]

where $m:[-1,1]^{2}\to\mathbb{R}$ is the smooth part, $\tau:[0,2\pi)\to\mathbb{R}_{+}$ is the jump height and the location $x_{\gamma}$ is given by $x_{\gamma}:[0,1]\to[-1,1]^{2}:t\mapsto t(\cos(\gamma),\sin(\gamma))^{\top}$ with $\gamma\in[0,2\pi)$. Now for $\gamma\in[0,2\pi),\psi\in[-\pi/2,\pi/2]$, consider the estimator

[TABLE]

While details need to be worked out, our asymptotic analysis below should be transferable to this model.

3 Asymptotic theory

For our asymptotic analysis, we require the following set of assumptions.

Assumption 1** (Errors).**

The errors $\boldsymbol{{\varepsilon}}_{i_{1},i_{2}}$ are square-integrable, centered, independent and identically distributed random variables with common variance $\sigma^{2}.$ Moreover, ${\rm E}|\epsilon_{1,1}|^{5}<\infty.$ ∎

Assumption 2** (Smoothness).**

We have that $\phi\in C^{2}[0,1],\tau\in C^{2}[0,1]$ and $m\in C^{2}[0,1]^{2}.$ ∎

Assumption 3** (Bandwidth).**

The bandwidth $h=h_{n}$ lies in the range $[{C_{1}\ln(n)^{\eta}}/{n^{1/2}},{C_{2}}/{n^{1/3}}]$ for some finite constants $C_{1},C_{2}>0$ and some fixed $\eta>1.$ ∎

Assumption 4** (Kernel).**

The kernel functions $K_{1}$ and $K_{2}$ are three-times continuously differentiable with support in $[-1,1]$ and satisfy the following conditions.

1. (a)

$K_{1}$ is symmetric, i.e., $K_{1}(x)=K_{1}(-x)$, and $K_{1}(x)>0$ for $x\in(-1,1)$. Further, $K_{1}$ satisfies, for all $j\in\{0,1,2\}$,

[TABLE] 2. (b)

$K_{2}$ is an odd function, i.e., $K_{2}(x)=-K_{2}(-x)$, in particular $K_{2}(0)=0$, and satisfies, for all $j\in\{0,1,2\}$,

[TABLE]

∎

Assumption 5** (Kernel moment).**

$K_{2}$ satisfies $\int_{0}^{1}xK_{2}(x)\mathrm{d}x=0.$ ∎

Remark 4** (Assumptions).**

• (i)

Assumption 1 could be relaxed to ${\rm E}|\varepsilon_{1,1}|^{k}<\infty$ for some $k>4/(2-\delta)$ and $\delta\in(2/3,1],$ as can be seen from the Gaussian approximation provided in the Online Supplement; see (LABEL:eq_approx_rio) therein. However, this would introduce a new parameter $\delta,$ which in turn has to be taken into account for the range of admissible bandwidths and thus for sake of convenience we require the slightly stronger assumptions of existing fifth moments which is sufficient for the bandwidth range in Assumption 3.

• (ii)

Assumptions 2–4 are required for the asymptotic distribution of the jump-location estimate $\hat{\phi}_{n}$ in (7), whereas Assumption 5 is only needed for the faster rates of convergence of the estimates $\hat{\psi}_{n}$ and $\hat{\tau}_{n}.$ Under Assumption 2 the moment properties of $K_{1}$ resp. $K_{2}$ in Assumption 4 (resp. 5) eliminate lower order terms in Taylor expansions, similarly as for higher order kernels in standard kernel smoothing. We also point out that Assumption 3 corresponds to an undersmoothing for the continuous part of the image function. In particular, we require $nh_{n}^{2}\to\infty$ and $nh_{n}^{3}\to\text{const.},$ a standard assumption in bivariate nonparametric estimation for Lipschitz functions. Finally, note that $\ln(n)$ and $\ln(h^{-1})$ are of the same order.

• (iii)

Our new conditions in Assumption 4 are necessary for asymptotic normality of the estimates based on the rotated difference kernel method.

In the following, we fix some compact subinterval $I\subset(0,1)$. Let us start with uniform consistency of the estimators on $I$.

Theorem 1**.**

In model (1), under Assumptions 1–4 we have that

[TABLE]

If in addition Assumption 5 holds, then

[TABLE]

Remark 5** (Rates of convergence and optimality).**

Theorem 1 shows that with our kernel assumptions, i.e., Assumption 3, the estimator for the jump location achieves the optimal rate of convergence up to a logarithmic factor, see Theorem 3.3.1 in [11] for the corresponding lower bound. Note that this is faster than the rates in Remark 2.4 in [22], who lacks the kernel conditions in his asymptotic analysis.

Although there seem to be no rigorous results on lower bounds available, it is reasonable to believe that the rates for the jump-slope curve and the jump-height curve are optimal up to a logarithmic term as well, since these correspond to the uniform rate for estimating Lipschitz continuous functions.

Now we turn to asymptotic normality of $\hat{\phi}_{n}(x)$ for fixed $x$. To state a version which does not involve a bias correction, we require the maximizer of the deterministic version of the contrast function

[TABLE]

Theorem 2**.**

In model (1), under Assumptions 1–4 we have, for any $x\in I$, that as $n\to\infty$,

[TABLE]

where

[TABLE]

We even obtain joint asymptotic normality and asymptotic independence of $(\hat{\phi}_{n},\hat{\psi}_{n},\hat{\tau}_{n})$; see the proofs section, Theorem 4. In (8), $V_{\phi}^{H}(x)$ arises as a limit of the Hessian matrices, while $V_{\phi}^{S}(x)$ comes from the asymptotic variance of the score.

Remark 6** (Confidence intervals).**

In order to construct asymptotic confidence intervals, we choose a consistent estimate $\hat{\sigma}^{2}_{n}$ of the error variance $\sigma^{2}$; see, e.g., Munk et al. [17] and the simulations in Section 4. Given $\alpha\in(0,1)$, we obtain an asymptotic level $1-\alpha$ confidence interval for $\phi_{n}(x)$ by computing

[TABLE]

where $q_{\beta}=q_{\beta}(\mathcal{N}_{1}(0,1))$ is the $\beta$-quantile of $\mathcal{N}(0,1)$ and where we substitute estimators for the unknown parameters in (8),

[TABLE]

Remark 7** (Bias correction).**

The proofs show that there exists a bounded sequence $b_{n}$ such that, as $n\to\infty$,

[TABLE]

An explicit bound on $b_{n}$ based on the Lipschitz constants of $K,m,\phi$ and $\psi$ as well as on $||\,\tau\,||_{\infty}$ would be available by an explicit estimation of the second-order term and the Riemann-sum error term, see Lemma 3 and the expansion (26), and hence bias-corrected conservative confidence intervals could be constructed in a similar fashion as in Eubank and Speckman [6]. Indeed, from Korostelev and Tsybakov [11] it is known that the discretization bias for jump-curve estimation in a deterministic design is not negligible theoretically. However, our simulations show that the confidence intervals in (9) can typically be used for the actual parameter, and a bias correction is not necessary, indeed, it makes the intervals quite conservative.

Now let us turn to the construction of uniform confidence sets. For independent standard normal random variables $\xi_{1,1},\ldots,\xi_{n,n},$ independent of $Y_{i_{1},i_{2}},$ consider the process

[TABLE]

where $\nabla K(z_{1},z_{2})=(K_{1}^{(1)}(z_{1})K_{2}(z_{2}),K_{1}(z_{1})K_{2}^{(1)}(z_{2}))^{\top}$ and by definition of the rotation matrix in (5),

[TABLE]

This process corresponds to the normalized score process evaluated at the estimates $(\hat{\phi}_{n},\hat{\psi}_{n})$ with independent noise-variables $\xi_{i_{1},i_{2}}$. Furthermore, consider the maximum

[TABLE]

The quantiles of this process can be determined by bootstrap simulations. The following result is the basis for constructing uniform confidence sets.

Theorem 3**.**

Consider model (1) under Assumptions 1–4, and assume that $\hat{\sigma}_{n}$ is an estimator for $\sigma$ which satisfies $\Pr(|{\hat{\sigma}_{n}}/{\sigma}-1|\geq s_{n})=o(1)$ for some sequence $s_{n}=o\{1/\ln(n)\}$. Then for $\alpha\in(0,1)$, one has $q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}})=O_{P}\{(\ln n)^{1/2}\}$ and for any sequence $t_{n}=o(1)$ such that $t_{n}\sqrt{\ln(n)}\to\infty$, we have that

[TABLE]

Remark 8** (Asymptotic confidence bands).**

Given $\alpha\in(0,1)$ a confidence band for $\phi$ which is asymptotically conservative at level $1-\alpha$ is given by

[TABLE]

Note that this is a version of the asymptotic point-wise confidence interval in (9), corrected by the logarithmic factor $q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}})$ for uniform coverage. Further, the uniform confidence bands also directly apply to the actual parameter $\phi$, the price to pay being that they are asymptotically conservative. Similarly as in Theorem 3, one can construct uniform confidence bands for $\psi$ and $\tau$. We sketch this in Section 6.9.

Remark 9** (Choice of $t_{n}$).**

The factors $t_{n}$ can be interpreted as a bias correction, and are essential for the validity of (13), even though they do not affect the rate of convergence. We will conduct a sensitivity analysis for the choice of $t_{n}$ in Section 4. $t_{n}$ ensures that the confidence bands are asymptotically valid, i.e., the significance level $1-\alpha$ is kept asymptotically, the assumption on $t_{n}$ in Theorem 3 prevents a straightforward analysis of asymptotically correct coverage of the confidence bands as in Corollary 3.1 of [4].

4 Simulations

In this section we investigate the finite sample properties of the proposed asymptotic confidence sets for the location $\phi(x)$ of the edge as well as of the estimator

[TABLE]

for the asymptotic standard deviation of $n\hat{\phi}_{n}(x)$ in Theorem 2 using (10). Further, we also investigate the bias in the estimation of the edge when using a deterministic rectangular grid.

4.1 Simulation setup

We choose

[TABLE]

as background image and jump height, and consider the following two edge functions

[TABLE]

from which we form the regression functions $m_{\phi_{1}}$, $m_{\phi_{2}}$, according to model (2). Further, we choose $\epsilon\sim t_{10}(0,\tilde{\sigma})$, i.e., a Student $t$ distribution with location parameter zero, scale parameter $\tilde{\sigma}$ and $10$ degrees of freedom. Thus, the noise-level is given by $\sigma=\tilde{\sigma}\sqrt{10/8}$. For illustration purposes, we display observations from the two models in Figure 2 for grid size $n^{2}=128^{2}$. We use the kernels

[TABLE]

where $C_{1}$ and $C_{2}$ are normalizing constants such that $\int K_{1}=1$ and $\int_{0}^{1}K_{2}=1$, which satisfy Assumption 4.

For the asymptotic results the bandwidth $h$ need not be chosen according to unknown smoothness parameters, and hence bandwidth selection is not a serious issue from a theoretical point of view. One could choose the bandwidth according to a selection rule like cross-validation or Lepski’s rule to optimally estimate the background image $m$. Qiu [24] discusses simpler, more heuristic alternatives, one of which is to choose the window so that it contains approximately 100 design points. Although this is certainly not a universal rule which works for any $n$, we achieve reasonably good results with this approach for our grid sizes of $n^{2}\in\{128^{2},196^{2},256^{2}\}$. In repeated simulations we use $500$ repetitions.

4.2 Estimating the asymptotic standard deviation

We start by investigating the numerical performance of the estimator (15) of the asymptotic standard deviation. To this end, we need to specify $\hat{\sigma}_{n}$, for which we choose squares of differences of all neighboring observation pairs properly normalized. The theory in [17] does not immediately apply when estimating on the full image. One possibility is to restrict estimation to a smooth part of the image. We also simulated the estimator $\hat{\sigma}_{n}$ of the standard deviation $\sigma$ separately, the results (not displayed) were also satisfactory.

Next we present the results for (15). Figure 3 shows smoothed estimates for specific samples for grid sizes $n^{2}\in\{128^{2},196^{2},256^{2}\}$ for the two edge functions $\phi_{i}$. Further, in Tables 1 and 2 we plot the square roots of the Mean-Squared-Error (RMSE) of the standard deviation estimates for the three sample sizes and two edge curves at various observation points $x$ based on repetitions. For comparison purposes, the actual asymptotic standard deviation is given as well. One observes that the RMSE in most settings decreases as the number of grid points increases. Further, the magnitude of the RMSE as compared to the actual value of the asymptotic standard deviation is quite small for all cases.

4.3 Confidence intervals and confidence bands

We investigate the coverage behavior and average width of (9) as well as of (14) for the true jump-location curves $\phi_{i}$ in both settings $i\in\{1,2\}$. The results are summarized in Tables 4 and 5 for the noise-levels $\tilde{\sigma}=0.5$ and $\tilde{\sigma}=0.9$. The values in the tables of the point-wise confidence intervals correspond to the average of the respective quantity over 64 design points $x$. The quantile $q_{\beta}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}})$ for $\beta\in(0,1)$ was simulated based on a multiplier bootstrap sample of size $40{,}000$. Furthermore, as there is no explicit representation of the $t_{n}$-term we have chosen it as given in Table 3 for the different scenarios. The $t_{n}$ decrease for increasing sample size $n$ and are of the same magnitude for both scenarios, i.e., for $\phi_{1}$ and $\phi_{2}$. By way of comparison, we give the values of $1/\sqrt{\ln(n)}$ for the different sample sizes $n$ as well. Especially, in the high-noise case the magnitude of our choice for $t_{n}$ is much smaller as this benchmark.

Overall, the simulated coverage probabilities for the point-wise confidence intervals are reasonably close to their nominal values in all scenarios, and the intervals become narrower with increasing numbers of grid points. As expected from the theoretical developments, the uniform confidence bands are somewhat conservative particularly in the high noise-level case.

Figure 4 illustrates the estimated curve as well as the confidence intervals and bands for $\phi_{1}$ and $\phi_{2}$ in the low noise-level for increasing numbers of grid points for $\alpha=0.05$, that is, asymptotic $95\%$ coverage probability. Apparently, the variability of the jump-location estimator decreases and the confidence intervals or bands become narrower. Besides, the confidence bands adapt to the shape of the point-wise confidence intervals as the width-terms only differ in the choice of the quantile.

4.4 Sensitivity analysis of $t_{n}$

In order to get some insight on the role of the sequence $t_{n},$ we display in Figure 5 the empirical quantiles of

[TABLE]

together with the simulated quantile curve $\{q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}}):\alpha\in(0,1)\}$ for $\phi_{1}$ (the results for $\phi_{2}$ were similar). The simulated quantile curve is below the empirical quantile curve in particular for the low-noise level, while in the high noise-level the bootstrapped quantile curve is below the empirical quantile curve only for values smaller than a unique intersection point at approximately $0.95,$ and from then on above the empirical quantile curve. Thus, to guarantee an appropriate coverage of the confidence bands the bias correction term $t_{n}$ sequence must be of a higher magnitude in the low-noise level than in the high-noise level. Heuristically this is reasonable as the presence of the bias is much more noticeable in the low-noise case than in the high-noise case. The asymptotic choice $1/\sqrt{\ln(n)}$ would lead to valid but conservative confidence bands.

4.5 Comparing bias and standard deviation

Next, we investigate the order of the bias numerically and compare it to the standard deviation. Tables 6 and 7 contain the results for the ratio of the bias and the standard deviation for different design points $x$ in the low noise-level and the high noise-level case. The ratios are quite small, showing that the bias indeed is often of smaller magnitude than the standard deviation.

4.6 Several jump curves

We briefly investigate numerically the extension indicated in Remark 2 to two jump curves. The data are generated from

[TABLE]

and $m$ is as before, while $\sigma=0.1,$ $n=64,$ and $h=0.15.$

In each strip $\{x\}\times[0,1],$ in a first step we choose as estimates the points with maximal contrast and having at least $h$-distance, see the left picture in Figure 6. In the second step, we compute the maximizers in some $\delta$-neighborhood of the $y$-coordinate of the candidate points and construct the simultaneous confidence bands by a Bonferroni-correction. The results are displayed in the right picture of Figure 6. Both edges are within the 95% confidence bands and are of satisfactory width.

4.7 Real data illustration

Finally we apply our method to two $300\times 128$ Gray-scale real-data images, taken by camera by one of the authors. They contain the outline of a rock in front of a gray background. Once, an appropriate ISO–configuration and focus on the rock and once inappropriate ISO–configuration of the camera and no focus at all are employed. In both cases we apply our method to estimate the boundary of the rock and construct 0.95-level uniform confidence sets using $t_{n}=0$ and a naive estimator for the noise-level. Figure 7 contains the results.

The jump-location curve lies mostly inside the constructed confidence band and the width is quite satisfying, although the noise-level of the picture is rather low. This illustration might be representative for more serious applications. For instance, a polar explorer might intend to monitor the height of a glacier in order to analyze the effect of global warming. Due to external effects such as bad weather conditions the pictures of the glacier could be noisy, so that its ridge needs to be estimated. The confidence bands are useful to assess whether the height of the glacier actually decreased.

5 Discussion

In this paper we developed methods to construct asymptotic confidence sets for the jump curve in an otherwise smooth two-dimensional regression function, for which to the best of our knowledge no methods were previously available. As a further step, a combination of our results with the issue of edge detection would be desirable. A direct thresholding of the contrast function (6) without prior localization as in (17) seems not to result in confidence sets which decrease at a near optimal rate, so that further methods would be required. An extension of our results to the multivariate setting, especially to three dimensions would also be relevant. Further, images are often observed with blurring, that is, after convolution with a point-spread function. Thus, extensions to a deconvolution setting would also be of practical importance.

6 Outline of proofs

For future reference we collect additional notation that is used in the following.

6.1 Notation

We shall use the following notation. For a vector $\mathbf{z}=(z_{1},z_{2})^{\top}\in\mathbb{R}^{2}$ we denote the coordinate projection onto the $i$th coordinate as $(\mathbf{z})_{i}=z_{i}$ for $i\in\{1,2\}$. Furthermore, we write $\mathbf{z}^{\alpha}=z_{1}^{\alpha_{1}}z_{2}^{\alpha_{2}}$ for $\alpha=(\alpha_{1},\alpha_{2})^{\top}\in\mathbb{N}^{2}$ and

[TABLE]

for a function $f:\mathbb{R}^{2}\to\mathbb{R}$. In particular, $\nabla f(\mathbf{z})=(f^{(1,0)},f^{(0,1)})^{\top}(\mathbf{z})$ and

[TABLE]

We also write $\partial_{z_{i}}f(\mathbf{z})$ for $f^{\mathbf{e_{i}}^{\top}}(\mathbf{z}),$ where $\mathbf{e_{i}}$ is the $i$th canonical unit vector. If $g:X\to Y$ for $X,Y\subset\mathbb{R},$ then we let $\text{epi}(g)=\{(x,y)^{\top}\in X\times Y:g(x)\leq y\}$ be the epigraph of $g$. Let $A\triangle B$ denote the symmetric difference between two sets $A$ and $B$, i.e., for any $A,B\subset\mathbb{R}^{2}$,

[TABLE]

Write $\mathbf{z}+A=\{\mathbf{z}+\mathbf{y}:\mathbf{y}\in A\}$ for $\mathbf{z}\in\mathbb{R}^{2}$ and a set $A\subset\mathbb{R}^{2}$. For sequences $a_{n}$ and $b_{n}$ we write $a_{n}\cong b_{n}$ if there exist constants $0<C_{1}<C_{2}$ and $n_{0}\in\mathbb{N}$ such that $C_{1}\leq|a_{n}/b_{n}|\leq C_{2}$ for any $n\geq n_{0}$. Denote by $\|\cdot\|$ a norm on $\mathbb{R}^{2}$ as well as on $\mathbb{R}^{2\times 2}$, where the dimension should be clear from the context. We only assume that the matrix-norm is compatible with the vector-norm, that is $\|\mathbf{A}\mathbf{z}\|\leq\|\mathbf{A}\|\times\|\mathbf{z}\|$ for a matrix $\mathbf{A}$ and a vector $\mathbf{z}$ and that the matrix-norm is submultiplicative, i.e., $\|\mathbf{A}\mathbf{B}\|\leq\|\mathbf{A}\|\times\|\mathbf{B}\|$ for matrices $\mathbf{A},\mathbf{B}$. For a function $f:I\to\mathbb{R}^{i\times j}$ for $i,j\in\{1,2\}$ with $i\geq j$ and $I$ is a compact subinterval of $(0,1)$ we define the uniform-norm as

[TABLE]

6.2 Rescaling of the contrast function

From now on, we shall always assume that $h$ is so small ($n$ is sufficiently large) that $I\subset[h,1-h]$. Define, for $x\in I$, the set of the rescaled parameters as

[TABLE]

and for $(x,\psi,w)^{\top}\in I\times\mathbb{R}^{2}$ the rescaled contrast function

[TABLE]

where $\hat{\text{M}}_{n}$ is given in (6) and $\mathbf{p}(x)=(x,\phi(x))^{\top}$ as well as $\mathbf{e_{2}}=(0,1)^{\top}$. With this, we have for the maximizers

[TABLE]

that $\hat{w}_{n}(x)=\{\hat{\phi}_{n}(x)-\phi(x)\}/h$. The parameter set is denoted by $\Theta_{n}=\bigcup_{x\in I}\{x\}\times\tilde{\Theta}_{n,x}$. We also introduce the deterministic contrast function

[TABLE]

6.3 Uniform consistency

In this section we show uniform consistency over $I$ for the maximizers in (18).

Proposition 1**.**

Under Assumptions 1–4 we have, as $n\to\infty$,

[TABLE]

For the proof, we make use of the following adaptation of Theorem 5.7 in [27], which is proved in Section LABEL:sec:stochunifconv of the Online Supplement.

Proposition 2**.**

Assume that $\tilde{\Theta}_{n,x}\subset\mathbb{R}^{d}$ are compact sets and set $\Theta=\cup_{n}\Theta_{n},$ where $\Theta_{n}=\cup_{x\in I}\{x\}\times\tilde{\Theta}_{n,x}$. Let $\hat{f}_{n}:\mathbb{R}^{d}\times I\to\mathbb{R}$ be random functions and let $f:\mathbb{R}^{d}\times I\to\mathbb{R}$ be a deterministic function. Suppose that

[TABLE]

and that there exists a map $\theta_{0}:I\to\Theta$ such that, for any $\epsilon>0$,

[TABLE]

Then for any estimator $\hat{\theta}_{n}(x)\in\tilde{\Theta}_{n,x},$ where $x\in I,$ which satisfies

[TABLE]

one has $||\,\hat{\theta}_{n}-\theta_{0}\,||_{\infty}\stackrel{{\scriptstyle\text{P}}}{{\to}}0$ as $n\to\infty$.

In order to obtain Proposition 1, in the next two lemmas we check the requirements of Proposition 2. The next lemma determines the limit function for $\hat{\mathbb{M}}_{n}(w,\psi;x)$ and shows that they fulfill the assumption (19).

Lemma 1**.**

Under Assumptions 1–4, we have that

[TABLE]

where

[TABLE]

and, for any $\psi\in[-\pi,\pi]$, $\mathbb{H}(\psi)=\boldsymbol{\mathscr{R}}_{\psi}\{\mathbb{R}\times[0,\infty)\}$.

Proof of Lemma 1.

Split $\hat{\mathbb{M}}_{n}(w,\psi;x)$ into a smooth-image-related part, a jump-related part and an error-related part as follows

[TABLE]

Take

[TABLE]

Lemma LABEL:lemma:S_n_term in the Online Supplement with $r_{1}=k=2$, $r_{2}=r_{3}=0$ and $j=1$ states that

[TABLE]

uniformly over $\Theta_{n}.$ With the same functions $f,g_{2},g_{3}$ as above, one deduces from Lemma LABEL:lemma:J_n_term that

[TABLE]

uniformly over $\Theta_{n}.$ Finally, Lemma LABEL:lemma:E_n_term with the same $f,g_{2},g_{3}$ implies $E_{n}(w,\psi;x)=O_{p}\{\ln(n)^{1/2}(nh)^{-1}\}$ uniform over $\Theta_{n}.$ This concludes the proof of the lemma. ∎

In the next lemma, we rewrite the asymptotic form of the contrast and show that it has a unique well-separated maximum, that is the requirement (20).

Lemma 2**.**

Under Assumptions 2 and 4, one has that if $\psi(x)-\psi=\pm\pi/2$ then $\mathbb{M}\left(w,\psi;x\right)=0$ for all $w$, while otherwise

[TABLE]

where

[TABLE]

This implies that for any $\epsilon>0$,

[TABLE]

Moreover, $\mathbb{M}(0,\psi(x);x)=\tau(x)$.

Proof of Lemma 2.

If $\psi(x)-\psi=\pi/2$ then $\mathbb{H}\{\psi(x)-\psi\}=(-\infty,0]\times\mathbb{R}$. Since $\int_{\mathbb{R}}K_{2}=\int_{-1}^{1}K_{2}=0$, the statement $\mathbb{M}\left(w,\psi;x\right)=0$ follows from (21), and similarly for $\psi(x)-\psi=-\pi/2$. If $\psi(x)-\psi\not=\pm\pi/2$, we start by showing that

[TABLE]

for an illustration see Figure 8. If $w=0$ the assertion is trivial. If $w\not=0$, we need to determine $a$ such that the vector $a\,\mathbf{e_{2}}$ is on the boundary of the set $\mathbb{H}\{\psi(x)-\psi\}+w\left(\sin\psi,\cos\psi\right)^{\top}$; see Figure 8. Setting $c=w\sin\left(\psi\right)/\cos\{\psi(x)-\psi\}$ and $b=c\sin\{\psi(x)-\psi\}$ we get that (see Figure 8)

[TABLE]

From (21) we obtain that

[TABLE]

Since $|a_{x}(\psi)y+b_{x}(w,\psi)|\geq 1$ if $|w|\geq 1/{|\cos\{\psi(x)\}|}+1$ for all $y\in[-1,1]$ and $\psi\in[-\pi/2,\pi/2]$, $\mathbb{M}\left(w,\psi;x\right)$ vanishes outside a compact set of values of $w$ and $x$.

Turning to (23), if we show that it holds for individual $x$, then since the supremum in (23) can be taken over a single compact set, we have that the left-hand side of (23) is a continuous function in $x$ which is positive for any $x\in[0,1]$. Hence, the infimum over $x\in[0,1]$ is still positive.

To show that (23) holds for individual $x$, we observe that $-\bar{K}_{2}(0)=1$, $-\bar{K}_{2}(y)<1$ if $y\not=0$, and $\bar{K}_{2}(y)=0$ if $|y|\geq 1$ by Assumption 4. Since $\mathbb{M}\left(w,\psi;x\right)$ is continuous in $w$ and $\psi$, as can also be seen from (21), it is enough to show that $(0,\psi(x))$ is the unique maximizer of $\mathbb{M}(w,\psi;x)$. But this is immediate from (22) and the above properties of $\bar{K}_{2}$ and the positivity of $K_{1}$, since if $(w,\psi)\not=(0,\psi(x))$, there is at most a single value of $z_{1}$ for which $a_{x}(\psi)z_{1}+b_{x}(w,\psi)=0$ and hence $-\bar{K}_{2}\{a_{x}(\psi)z_{1}+b_{x}(w,\psi)\}=1$. ∎

Proof of Proposition 1.

The uniform consistency of the estimates $\hat{w}_{n}$ and $\hat{\psi}_{n}$ is immediate from Proposition 2, as Lemma 1 and Lemma 2 provide the necessary assumptions. ∎

6.4 Rate of convergence: Proof of Theorem 1

To prove the theorems in the main part we start with a simple linearization. By the Mean Value Theorem, we have that

[TABLE]

since $\nabla\hat{\mathbb{M}}_{n}\left(\hat{w}_{n}(x),\hat{\psi}_{n}(x);x\right)=0$. This implies

[TABLE]

where

[TABLE]

and the existence of the inverse of the Hessian matrix $\hat{\mathbf{H}}_{n}(x)$ uniformly in $x\in I$ for large $n$ and with high probability follows from Lemma 4 below.

Similarly as in (24), for the rescaled maximizers of the deterministic contrast function, i.e.,

[TABLE]

we have that

[TABLE]

where $w_{n}(x)=\{\phi_{n}(x)-\phi(x)\}/h$ and the deterministic Hessian matrix is given by

[TABLE]

By means of (25), we can decompose the representation in (24) into a stochastic part (or score part), a bias part and a remainder term as follows:

[TABLE]

where

[TABLE]

Note that by (22) we have $\tau(x)=\mathbb{M}(0,\psi(x);x)$ and $\hat{\tau}_{n}(x)=\hat{\mathbb{M}}_{n}(\hat{w}_{n}(x),\hat{\psi}_{n}(x);x)$ such that by the Mean Value Theorem as well as adding and subtracting $\mathbb{M}_{n}(0,\psi(x);x)$, one has

[TABLE]

where

[TABLE]

6.5 Asymptotic bias

Lemma 3**.**

One has $\nabla\mathbb{M}_{n}(w,\psi;x)={\rm E}\{\nabla\hat{\mathbb{M}}_{n}(w,\psi;x)\}$ for any $(x,w,\psi)\in\Theta_{n}$. Moreover, under Assumptions 2 and 4 we have that

[TABLE]

If in addition Assumption 5 is fulfilled, then

[TABLE]

The constants in the remainder terms depend only on the Lipschitz constants of $K,m,\phi$ and $\psi$ as well as on $||\,\tau\,||_{\infty}$. In particular, if Assumption 3 is satisfied, then

[TABLE]

The proof of this lemma is provided in Section LABEL:sec:asymbias of the Online Supplement. The bias consists of a smoothing bias of order $O(h^{2}),$ and a discretization bias of order $O\{(nh)^{-1}\}$, which dominates the order of the bias in case of undersmoothing, that is under Assumption 3.

6.6 Convergence of the Hessian matrix

Lemma 4**.**

Under Assumptions 1–4, we have, as $n\to\infty$,

[TABLE]

where $\mathbf{H}(x)={\rm diag}\{V_{\phi}^{H}(x),V_{\psi}^{H}(x)\}$ and $V_{\phi}^{H}(x)$ as in Theorem 2, while

[TABLE]

The proof of Lemma 4 is provided in Section LABEL:sec:convhessian of the Online Supplement. Note that the limit matrix $\mathbf{H}$ corresponds to the Hessian matrix of the asymptotic criterion function $\mathbb{M}$ at the parameters $w=0$ and $\psi=\psi(x)$. Furthermore, we only need the uniform consistency in Lemma 4 to derive the proof of Theorem 1 and 2. However, for the proof of Theorem 3 we need the uniform convergence even with an explicit rate of convergence.

Proposition 3**.**

Under Assumptions 1–4, we have that

[TABLE]

The proof of this proposition is provided in Section LABEL:sec:asympscore of the Online Supplement. $\Box$

Proof of Theorem 1.

Lemma 4 implies that $\hat{\mathbf{H}}^{-1}_{n}$ is almost surely a stochastically bounded matrix-valued sequence uniformly in $x$. Hence, in combination with Lemma 3 deduce

[TABLE]

as well as

[TABLE]

Next, Slutzky’s Lemma and Proposition 3 imply

[TABLE]

From (26) it immediately follows by combining the three latter displays that

[TABLE]

where the last equation is due to Assumption 3. Note that Assumption 5 is only needed for the bias of the slope in (28).

The uniform rate of convergence for $\hat{\tau}_{n}$ now follows with a similar argument. Indeed, due to representation (27) and Lemma 3 and Proposition 3 it suffices to show that

[TABLE]

Since we verified above that $||\,\hat{w}_{n},(\hat{\psi}_{n}-\psi)\,||_{\infty}=O_{P}\{(\ln n)^{1/2}/(nh)\}$ holds, we only need to show that

[TABLE]

where we abbreviate

[TABLE]

By the Mean Value Theorem (differentiating under the integral), Lemma 4 and since $\hat{\mathbf{H}}_{n}$ is uniformly bounded,

[TABLE]

where the last equation is justified by Assumption 3. Note that

[TABLE]

Hence, due to Lemma 3, Proposition 3 and the triangle inequality

[TABLE]

so that $\sup_{x\in I}\|f(\hat{w}_{n}(x),\hat{\psi}_{n}(x);x)\|=o_{P}\{(\ln n)^{-1/2}\}$ follows by (31) and (32) by using the triangle inequality. ∎

6.7 Asymptotic normality: Proof of Theorem 2

We first establish the asymptotic normality of the score.

Lemma 5**.**

Under Assumptions 1, 3 and 4 we have that, for any $x\in I$, as $n\to\infty$,

[TABLE]

where $V_{\phi}^{S}$ is as in Theorem 2 and

[TABLE]

Section LABEL:sec:asympscore of the Online Supplement contains the proof of this lemma. Theorem 2 now follows immediately from the following more general theorem, since $\hat{w}_{n}(x)=\{\hat{\phi}_{n}(x)-\phi(x)\}/h$ as well as $w_{n}(x)=\{\phi_{n}(x)-\phi(x)\}/h$.

Theorem 4**.**

Under Assumptions 1–4, one has, as $n\to\infty$,

[TABLE]

where $\Sigma(x)={\rm diag}[V_{\phi}^{S}(x)/\{V_{\phi}^{H}(x)\}^{2},V_{\psi}^{S}/\{V_{\psi}^{H}(x)\}^{2},V_{\tau}^{S}]$.

Proof of Theorem 4.

From (24) and (25), we obtain that

[TABLE]

Now, by the Mean Value Theorem one has that

[TABLE]

With this, (27) and the first display of this proof, we have

[TABLE]

On the one hand, the first term is asymptotically normally distributed with covariance matrix $\Sigma$ as in the assumption, since by Lemmas 4 and 5 and Slutzky’s Lemma one has, for all $x\in(0,1)$,

[TABLE]

where $(\Sigma(x))_{1:2,1:2}$ is the matrix given consisting of the first two columns or first two rows of $\Sigma(x)$. On the other hand, the second term on the right-hand side (40) is $o_{P}(1)$ by (29) and (30), whereas one can show $R_{n,2}^{\tau}(x)=o_{p}\{(nh)^{-1}\}$ similar as (30). This concludes the proof of Theorem 4. ∎

6.8 Uniform confidence bands: Proof of Theorem 3

For the proof of Theorem 3 we require rates of convergence of the normalized estimators of the Hessian matrix.

Lemma 6**.**

Under Assumptions 1–4 we have that

[TABLE]

Here, $V_{\widehat{\psi}}^{H}=\hat{\tau}_{n}(x)K_{2}^{(1)}(0)\int_{-1}^{1}y^{2}K_{1}(y)\mathrm{d}y.$

The proof is provided in Section LABEL:sec:rate_of_conhessian of the Online Supplement.

Next we extend our notation by incorporating the following definition. The Lévy-concentration function of a random variable $X$ is given, for all $\zeta\geq 0$, by

[TABLE]

We introduce the normalized score process

[TABLE]

It easily follows from Lemma LABEL:lemma:first_deriv_empcrit in the Online Supplement that

[TABLE]

such that

[TABLE]

The score process in (11) can be obtained from the latter display by replacing the actual parameters $(\phi,\psi)$ by their estimates $(\hat{\phi}_{n},\hat{\psi}_{n})$ and the noise of the observations $\boldsymbol{{\varepsilon}}_{i_{1},i_{2}}/\sigma$ by the sequence $\xi_{i_{1},i_{2}}$. Similarly to (12) we set

[TABLE]

Lemma 7**.**

Under Assumptions 1–4 the following statements are valid.

• (i)

By possibly enriching the probability space, one has

[TABLE]

• (ii)

One has $\mathscr{Z}^{\xi}_{n,\widehat{\phi}}=O_{P}\{(\ln n)^{1/2}\}$ and for any sequence $\delta_{n}=o\{(\ln n)^{-1/2}\}$, one has $\mathbb{L}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}},\delta_{n})=o(1)$.

• (iii)

Moreover, $q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}})\cong(\ln\,n)^{1/2}.$

The proof is given in Section LABEL:sec:gauss_approx_phi_psi of the Online Supplement.

Proof of Theorem 3.

Set $\hat{d}_{n}=\hat{\sigma}_{n}q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}})$. Recall that for $\mathbf{z}=(z_{1},z_{2})^{\top}\in\mathbb{R}^{2}$ we write $(\mathbf{z})_{i}=z_{i}$ for $i\in\{1,2\}$. From (26) we obtain that

[TABLE]

From the last statement of Lemma 7 and the assumption on $t_{n}$ we deduce that $t_{n}\hat{d}_{n}\to\infty$ as $n\to\infty$. Hence, by using Lemmas 3 and 6 the second term in the preceding display can be bounded by

[TABLE]

Recalling the notation in (41), the first term can be estimated by

[TABLE]

Lemma 6 implies that

[TABLE]

and together with Proposition 3 the first term in (42) can be bounded by

[TABLE]

due to Assumption 3. As for the second term, plugging in $\hat{d}_{n}=\hat{\sigma}_{n}q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}})$ gives

[TABLE]

by using the first part of Lemma 7 together with the choice of $h$, and the definition of the Lévy-concentration function. The last term in this display is $o(1)$ by using the second part of Lemma 7. Finally, observe that

[TABLE]

where $s_{n}$ is as in the assumption of the theorem and where we used the second statement of Lemma 7 together with the third statement of Lemma 7 to obtain $s_{n}q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\phi}})=o\{(\ln n)^{-1/2}\}$.

Summarizing, we obtain that

[TABLE]

which corresponds to (13). Eventually, the third statement of Lemma 7 concludes the proof. ∎

6.9 Uniform confidence bands for jump-slope and jump-height curve

For independent standard normally distributed random variables $\xi_{1,1},\ldots,\xi_{n,n},$ independent of $Y_{i_{1},i_{2}},$ consider in the spirit of (11) the processes

[TABLE]

where $V_{\psi}^{S}$ and $V_{\tau}^{S}$ are given in Lemma 5, as well as the maxima of the processes

[TABLE]

The following theorem can be used to construct uniform confidence bands for $\psi$ or $\tau$.

Theorem 5**.**

Consider model (1) under Assumptions 1–5, and assume that $\hat{\sigma}_{n}$ is an estimator for $\sigma$ which satisfies $\Pr(|{\hat{\sigma}_{n}}/{\sigma}-1|\geq s_{n})=o(1)$ for some sequence $s_{n}=o\{\ln(n)^{-1}\}$. Then for $\alpha\in(0,1)$, one has $q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\psi}})=O\{(\ln n)^{1/2}\}$ and for any sequence $t_{n}=o(1)$ such that $t_{n}\sqrt{\ln(n)}\to\infty$, one also has

[TABLE]

as well as $q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\tau}})=O\{(\ln n)^{1/2}\}$ and

[TABLE]

Sketch of proof of Theorem 5.

For the jump-slope and the jump-height, we introduce in the spirit of (41) the following processes

[TABLE]

and their suprema ${\mathscr{Z}}_{n,\psi}^{\varepsilon}=||\,Z^{\varepsilon}_{n,\psi}\,||_{\infty}$ and ${\mathscr{Z}}_{n,\tau}^{\varepsilon}=||\,Z^{\varepsilon}_{n,\tau}\,||_{\infty}$. Following the lines of proof of Lemma 7, one derives the following result.

Lemma 8**.**

Under Assumptions 1–4, and by possibly enriching the probability space, one has

[TABLE]

Moreover, $\mathscr{Z}^{\xi}_{n,\widehat{\psi}}=O_{P}\{(\ln n)^{1/2}\}$ resp. $\mathscr{Z}^{\xi}_{n,\widehat{\tau}}=O_{P}\{(\ln n)^{1/2}\}$ and for any sequence $\delta_{n}=o\{(\ln n)^{-1/2}\}$, we have that

[TABLE]

Eventually, one has $q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\psi}})\cong(\ln\,n)^{1/2}$ resp. $q_{1-\alpha}(\mathscr{Z}^{\xi}_{n,\widehat{\tau}})\cong(\ln\,n)^{1/2}.$

With this, it is straightforward to obtain the proof following the lines of the proof of Theorem 3. ∎

Acknowledgments

The authors are thankful to Axel Munk for helpful discussions in the early stage of the project, as well as to the Editor-in-Chief of the Journal of Multivariate Analysis, Christian Genest, an Associate Editor and two anonymous reviewers for helpful comments. Financial support of the Deutsche Forschungsgemeinschaft, grant Ho 3260/5-1, is gratefully acknowledged.