Box-constrained monotone $L_\infty$-approximations to Lipschitz regularizations, with applications to robust testing

TL;DR

This paper develops a new approach to robust statistical testing using box-constrained monotone $L_ abla$-approximations to Lipschitz regularizations, enabling stable and differentiable functionals for improved robustness analysis.

Contribution

It introduces a novel characterization of robust test functionals via monotone $L_ abla$-approximations, linking them to Lipschitz regularizations and envelopes, with implications for stability and asymptotic theory.

Findings

Functional admits equivalent formulations in terms of Lipschitz and monotone approximations.

Establishes stability and directional differentiability of the test functional.

Provides a basis for a Central Limit Theorem for the proposed robust test statistic.

Abstract

Tests of fit to exact models in statistical analysis often lead to rejections even when the model is a useful approximate description of the random generator of the data. Among possible relaxations of a fixed model, the one defined by contamination neighbourhoods, namely, $\mathcal{V}_\alpha(P_0)=\{(1-\alpha)P_0+\alpha Q: Q \in \mathcal{P}\}$, where $\mathcal{P}$ is the set of all probabilities in the sample space, has received much attention, from its central role in Robust Statistics. For probabilities on the real line, consistent tests of fit to $\mathcal{V}_\alpha(P_0)$ can be based on $d_K(P_0,R_\alpha(P))$, the minimal Kolmogorov distance between $P_0$ and the set of trimmings of $P$, $R_\alpha(P)=\big\{\tilde P\in\mathcal{P}:\tilde P\ll P,\,{\textstyle \frac{d\tilde P}{dP}\leq\frac{1}{1-\alpha}}\, P\text{-a.s.}\big\}$. We show that this functional admits equivalent formulations in terms of, either best approximation in uniform norm by $L$-Lipschitz functions satisfying a box constraint, or as the best monotone approximation in uniform norm to the $L$-Lipschitz regularization, which is seen to be expressable in terms of the average of the Pasch-Hausdorff envelopes. This representation for the solution of the variational problem allows to obtain results showing stability of the functional $d_K(P_0,R_\alpha(P))$, as well as directional differentiability, providing the basis for a Central Limit Theorem for that functional.

Equations

$${\mathcal{V}}_{\alpha }(P_0)=\{(1-\alpha )P_0+\alpha Q:Q\in \mathcal{P}\},$$

$$R_{\alpha}(P)=\big{\{}\tilde{P}\in\mathcal{P}:\tilde{P}\ll P,\,{\textstyle\frac{d\tilde{P}}{dP}\leq\frac{1}{1-\alpha}}\,P\text{-a.s.}\big{\}}.$$

$$P\in {\mathcal{V}}_{\alpha }(P_0)⟺P_0\in R_{\alpha }(P).$$

$$d(P_0,R_{\alpha }(P))=0.$$

$$d_K(P_0,R_{\alpha }(P))=\min \{\Vert h-\Gamma \Vert ,h\in {\mathcal{C}}_{\alpha }\},$$

$${\mathcal{C}}_{\alpha }:=\{h:[0,1]\to [0,1]\text{mbox}nondecreasing,withh(0)=0,h(1)=1,\text{mbox}and\Vert h{\Vert }_{\mathrm{Lip}}\le 1/1-\alpha \}.$$

$$\Vert f\Vert =\underset{x\in \aleph }{\sup }|f(x)|,\Vert f{\Vert }_{\mathrm{Lip}}=\underset{x,y\in \aleph }{\sup }\frac{|f(x)-f(y)|}{d(x,y)}.$$

$$R_{\alpha }(F)=\{F_h:h\in {\mathcal{C}}_{\alpha }\}.$$

$$d_K(F_0,R_{\alpha }(F)):=\underset{\tilde{F}\in R_{\alpha }(F)}{\inf }\Vert \tilde{F}-F_0\Vert =\underset{h\in {\mathcal{C}}_{\alpha }}{\inf }\Vert h\circ F-F_0\Vert .$$

$$\big{|}\|h_{1}\circ F-F_{0}\|-\|h_{2}\circ F-F_{0}\|\big{|}\leq\|h_{1}\circ F-h_{2}\circ F\|\leq\|h_{1}-h_{2}\|,$$

$$\displaystyle\big{|}d_{K}(G_{1},R_{\alpha}(F_{1}))-d_{K}(G_{1},R_{\alpha}(F_{2}))\big{|}\leq\sup_{h\in\mathcal{C}_{\alpha}}\big{|}\|G_{1}-h\circ F_{1}\|-\|G_{1}-h\circ F_{2}\|\big{|}$$

$$\displaystyle\big{|}d_{K}(G_{1},R_{\alpha}(F_{2}))-d_{K}(G_{2},R_{\alpha}(F_{2}))\big{|}\leq\sup_{h\in\mathcal{C}_{\alpha}}\big{|}\|G_{1}-h\circ F_{2}\|-\|G_{2}-h\circ F_{2}\|\big{|}\leq\|G_{1}-G_{2}\|.$$

$$\text{mbox}Thereexists\tilde{F}\in R_{\alpha }(F)\text{mbox}suchthat\Vert F_0-\tilde{F}\Vert =d_K(F_0,R_{\alpha }(F)).$$

$$F\in {\mathcal{V}}_{\alpha }(F_0)\text{mbox}ifandonlyif{d}_K(F_0,R_{\alpha }(F))=0.$$

$$d_K(G_n,R_{\alpha }(F_m))\to d_K(G,R_{\alpha }(F)),\text{mbox}asn,m\to \infty ,\text{mbox}withprobabilityone.$$

$$d_K(F_0,R_{\alpha }(F))=\underset{h\in {\mathcal{C}}_{\alpha }}{\min }\Vert h-F_0\circ F^{-1}\Vert \text{mbox}and{d}_K(F_0,R_{\alpha }(F_n))=\underset{h\in {\mathcal{C}}_{\alpha }}{\min }\Vert h-F_0\circ F_n^{-1}\Vert .$$

$$\Vert h\circ F-F_0\Vert$$

$$t_0=0,t_i=\frac{i}{n},h_i=h(F_n(x_{(i)}))=h(t_i),\text{and}{F}_{0,i}=F_0(x_{(i)}),1\le i\le n.$$

$$\|h(F_{n})-F_{0}\|=\max_{1\leq i\leq n}\max\Big{(}F_{0,i}-h_{i-1},h_{i}-F_{0,i}\Big{)}=\|h-F_{0}(F_{n}^{-1})\|,$$

$${\tilde{h}}_{\alpha }(t)=\max \left(\min \left(\frac{U(t)+L(t)}{2},0\right),\frac{-\alpha }{1-\alpha }\right).$$

$$\underset{h\in {\mathcal{C}}_{\alpha }}{\min }\Vert h-\Gamma \Vert =\Vert {\tilde{h}}_{\alpha }-G\Vert .$$

$$p(x)=({\sigma }^2-1)x^2+2\mu \sigma x+{\mu }^2-2\log ((1-\alpha )\sigma ).$$

$$d_{K}(R_{\alpha}(N(\mu,1)),N(0,1))={\textstyle\Phi\big{(}\frac{|\mu|}{2}+\frac{1}{|\mu|}\log(1-\alpha)\big{)}-\frac{1}{1-\alpha}\Phi\big{(}-\frac{|\mu|}{2}+\frac{1}{|\mu|}\log(1-\alpha)\big{)}},\quad\mu\neq 0.$$

$$d_{K}(R_{\alpha}(N(0,\sigma^{2})),N(0,1))={\textstyle\Phi\Big{(}\frac{-\sigma\frac{\Delta}{2}}{1-\sigma^{2}}\Big{)}-\frac{1}{1-\alpha}\Phi\Big{(}\frac{-\frac{\Delta}{2}}{1-\sigma^{2}}\Big{)}},\quad\mbox{if }\sigma<1.$$

$$d_{K}(R_{\alpha}(N(0,\sigma^{2})),N(0,1))={\textstyle\Phi\Big{(}\frac{\sigma\frac{\Delta}{2}}{\sigma^{2}-1}\Big{)}-\frac{\Phi\Big{(}\frac{\frac{\Delta}{2}}{\sigma^{2}-1}\Big{)}-\frac{\alpha}{2}}{1-\alpha}},\quad\mbox{if }\sigma>{\textstyle\frac{1}{1-\alpha}}.$$

$$\Vert f\wedge g{\Vert }_{\mathrm{Lip}},\Vert f\vee g{\Vert }_{\mathrm{Lip}}\le \Vert f{\Vert }_{\mathrm{Lip}}\vee \Vert g{\Vert }_{\mathrm{Lip}}.$$

$$f_{L,1}(x):=\underset{y\in \aleph }{\inf }(f(y)+Ld(x,y)),f_{L,2}(x):=\underset{y\in \aleph }{\sup }(f(y)-Ld(x,y)).$$

$$\Vert g-f\Vert \ge \Vert f_L-f\Vert =\Vert f_{L,2}-f_{L,1}\Vert$$

$$\underset{y\in \aleph }{\inf }f(y)\le f_{L,1}(x)\le f(x)+Ld(x,x)=f(x)=f(x)-Ld(x,x)\le f_{L,2}(x)\le \underset{y\in \aleph }{\sup }f(y).$$

$$|f_{L,j}(x_2)-f_{L,j}(x_1)|\le Ld(x_1,x_2)\text{mbox}forj=1,2,$$

Full text

Keywords: Contamination neighbourhoods, Kolmogorov distance, uniform norm, Lipschitz-continuous approximations, distribution functions, trimmed probabilities, Pasch-Hausdorff envelopes, Lipschitz regularization, robustness, directional differentiability.

A.M.S. classification: Primary: 49J30. Secondary: 26A16, 62G35, 41A29.

1 Introduction.

A repeated joker phrase in Statistics says that all models are wrong, but some are useful. This celebrated aphorism, attributed to the statistician G. Box, on the one hand cautions that all models are approximations, while, on the other, stresses the usefulness of good approximate models. Here, approximation should be interpreted, in words of Davies [11], as “some formal admission of the fact that the statistical models are not true representations of the data”. From this perspective, within the research objectives of Mathematical Statistics, it becomes natural the permanent interest in the design and analysis of well-behaved procedures under small variations in the model. This includes the reconsideration of excessively restrictive concepts in Statistics, such as exact fit to models (say in homogeneity, regression or time series settings). The interest is not exact equality, but only “similarity” or, alternatively, to find a “relevant” difference. Also notice that this concept is of great relevance in some applications, such as bioequivalence in Biostatistics (see, for example, [29]). Some recent references sharing this spirit are ([23], [20], [13], [14], [15], [4], [5]). That is also the perspective of our recent work [7], while the present paper addresses the mathematical bases giving support to the approach.

Let us begin some historical notes on approximate model checking. A pioneer work in that sense is [18]. There, Hodges and Lehmann pointed out that “when testing statistical hypotheses, we usually do not wish to take the action of rejection unless the hypothesis being tested is false to an extent sufficient to matter”. This fact led them to establish a distinction between statistical significance and material significance in hypotheses testing and to suggest modifications of the customary tests, in order to test for the absence of material significance. Their approach was based on assuming a distance in the parametric space and to allow some little deviation in the null hypothesis of the model.

Ten years later, in his seminal paper [19], Huber introduced the contamination neighbourhood of a probability, namely,

[TABLE]

where $\mathcal{P}$ is the set of all probability distributions in the space. Thus the probabilities in the neighbourhood are mixtures of level $\alpha$ of $P_{0}$ with other probabilities. Although it can be defined in a wholly general setting, throughout the paper $\mathcal{P}$ will be the set of probabilities on the (Borel) sets, $\beta$, of the real line $\mathbb{R}$). In this way, given an “ideal” model $P_{0}$, the vicinity includes those probabilities which are distorted versions of the model through gross errors: given a particular value $\alpha_{0}\in[0,1)$, a probability $P$ in $\mathcal{V}_{\alpha_{0}}(P_{0})$ would generate samples with an approximate $(1-\alpha_{0})\times 100$ percentage of data coming from $P_{0}$.

Contamination neighbourhoods become one of the very basis of Robust Statistics: a general attempt to provide methods with good performance when there are small departures from the assumed model. Not surprisingly, its simple interpretation in terms of mixtures, motivated their use in different settings. In particular, Rudas et al [25] introduced a new index of fit in the framework of contingency tables. Now the goal is to evaluate how well the contaminated version of the model describes the data, so statistical evidence of a “small $\alpha$” should be considered as almost agreement with the model. The reconsideration of the problem in [21], also in the multinomial setting, allowed little deviations of the model that are measured by the Kullback–Leibler divergence.

The setup considered in both [25] or [21] is constrained to the case when $P_{0}$ is a probability with a finite support. It should be noted at this point that testing fit to a neighbourhood of a fixed probability is not always a feasible task, depending on the metric or divergence which determines the neighbourhoods. Barron [8] considered the problem testing fit to approximate models and argued, while consistent tests were available for some weak metrics, it would desirable that from the statistical assessment that $P$ and $P_{0}$ are close in a certain metric one could conclude that $P$ and $P_{0}$ are close from every point of view. In plain words, he advocated for the use of strong metrics, such as the total variation metric $d_{TV}(P,P_{0})=\sup_{A\in\mathcal{A}}|P(A)-P_{0}|$, where $\mathcal{A}$ denotes the class of all measurable sets. Unfortunately, he also showed that if the probability $P_{0}$ is not discrete then there is no uniformly consistent test of fit to $P_{0}$ against alternatives at a certain distance in $d_{TV}$ and the same result remains true if the alternatives are bounded from $P_{0}$ in a distance or divergence that dominates the total variation metric. With these limitations in mind one may wonder if uniformly consistent testing to a meaningful relaxation of total variation neighbourhoods is possible beyond the discrete setting. In fact, contamination neighbourhoods are related to total variation neighbourhoods: $d_{TV}(P,Q)\leq\alpha$ if and only if there exists a probability $P_{0}$ such that $P\in\mathcal{V}_{\alpha}(P_{0})$ and $Q\in\mathcal{V}_{\alpha}(P_{0})$ (see [3]).

In [7] we showed that it is possible indeed to build a uniformly consistent test of fit to a contamination neighbourhood against increasingly closer alternatives. We addressed the problem through the dual approach of trimmed probabilities, an idea that goes back at least to [17]. A probability $\tilde{P}\in\mathbb{R}$ is said to be a trimming of level $\alpha\in[0,1)$ of $P$ whenever there exists a down-weighting function $w$ such that $0\leq w\leq 1$ and $\tilde{P}(B)=\frac{1}{1-\alpha}\int_{B}w(x)P(dx)$ for all the sets $B\in\beta$. Equivalently, it must be absolutely continuous w.r.t. $P$, with Radon-Nykodim derivative bounded by $\frac{1}{1-\alpha}$. The set of $\alpha$-trimmings of the probability distribution $P$ will be denoted by $R_{\alpha}(P)$:

[TABLE]

The key link between (1) and (2), obtained in [2], is given by

[TABLE]

This duality has been exploited for analysis of similarity between samples in a fully nonparametric context ([3]), or for the consideration of a testable almost stochastic dominance model ([4], [5]). There is a subtle, but important consequence of the duality (3). In a realistic statistical setting we do not know either the value $\alpha$ or the “contaminated” distribution $P$ but we only have an approximation $\hat{P}$ to $P$ (usually $\hat{P}$ will be the empirical measure associated to a data set), and our goal is to search for statistical evidence, based on $\hat{P}$, for or against the hypothesis $P\in\mathcal{V}_{\alpha}(P_{0})$. It turns out that sets of trimmings are often well behaved with respect to some of the most useful metrics in Statistics, while contamination neighbourhoods are not. If $d$ is a metric on $\mathcal{P}$ and $R_{\alpha}(P)$ is closed for $d$ then both conditions in (3) are equivalent to

[TABLE]

With a suitable choice of $d$ we could also ensure that $d(P_{0},R_{\alpha}(\hat{P}))$ is a consistent estimator of $d(P_{0},R_{\alpha}(P))$. The success of this strategy will strongly depend on the suitability of the metric for this task. Our choice here is the Kolmogorov distance, $d_{K}$, that for two probabilities $P,Q\in\mathcal{P}$ is defined by the $L_{\infty}$-distance between their distribution functions $F_{P}$ and $F_{Q}$. Davies [11] claims that the Kolmogorov distance induces the natural topology for statistics. Firstly, random variables are generated at the level of distribution functions …Secondly all diagnostic checks and model validation techniques operate at the level of distribution functions and not at the level of density functions … We show in this work that for $d_{K}$ the equivalent characterization (4) holds. Also, it follows from [6] that it is posible to use the empirical version of $d(P_{0},R_{\alpha}(P))$ to build a consistent estimator of $\alpha_{0}$, the minimal contamination level such that $P\in\mathcal{V}_{\alpha}(P_{0})$ (see subsection 4.2 in [7]). However, suitability of $d_{K}$ in this setting depends also on the feasibility of the generated procedures. In fact, some difficulties related this metric are well known, both for its mathematical analysis (lack of Fréchet or Hadamard differentiability) and for its computational aspect (lack of convergent algorithms).

The motivation of this work is to provide sound mathematical support to our approach in [7] focusing in tools for diagnostics, comparison and validation of an approximate statistical model. We will show (see Lemma 2.4) that the minimal Kolmogorov distance to a set of trimmings can be represented in terms of a variational problem, as follows. We set $\Gamma=F_{0}(F^{-1})$, $F_{0}$ and $F$ being the distribution functions of $P_{0}$ and $P$. Then, with great generality, the following identity holds:

[TABLE]

where

[TABLE]

Here, as will be used throughout, for any real valued mapping $f:\aleph\to\mathbb{R}$ defined on a metric space $(\aleph,d)$, with $\|f\|$ and $\|f\|_{\rm Lip}$ we will denote the $L_{\infty}$ and the Lipschitz norms:

[TABLE]

The representation in (5) translates the problem of best trimmed approximation in Kolmogorov distance into finding a useful expression for a best $L_{\infty}$-approximation to a monotone function by monotone, Lipschitz-continuous functions satisfying the boundary conditions $h(0)=0,h(1)=1$. We will show (see Theorem 2.5) that the solution to this problem can be expressed in terms of Pasch-Hausdorff envelopes (see [24]). We will also relate this process with the alternative way of obtaining Ubhaya’s monotone $L_{\infty}$-best approximation (see [27, 28]) to the Lipschitz regularization of the objective function.

There are two main implications of our analysis of the variational problem in (5) in statistical applications. First, it proves the validity of a simple, fast algorithm introduced in [7] for the computation of the empirical estimator $d_{K}(P_{0},R_{\alpha}(\hat{P}))$. Additionally, we use it to prove a result on directional differentiability of the $L_{\infty}$-distance to the regularized version (see Corollary 4.4). The relevance of this type of results on directional differentiability has been pointed out in [26], and recently highlighted in relation with statistical applications in [9]. In fact, these results provide the mathematical foundation allowing a Central Limit Theorem (see Theorem 4.1 in [7]), thus incoming statistical applications of the proposed methodology. We should note that, under the false-model paradigm, this Central Limit Theorem yielded some tools for comparing models or for determining the usefulness of particular models following lines related to [20],[10] or [12]. In particular we should highlight the applications in the False-Discovery-Rate (FDR) setting (as considered e.g. in [16] or [22]). In [7] (see Section 5 there) we discuss on the applicability of our approach to that setting.

The rest of this work is organized as follows. In Section 2 we will present some alternative characterization of the set $R_{\alpha}(P)$ as well as its main topological properties in the $L_{\infty}$ setting. We include a key result on the stability of the constrained regularizations (see Proposition 2.2) as well as the announced variational representation (Lemma 2.4) and the solution of the variational problem (Theorem 2.5). The proof of this result will follow from that of Theorem 3.3 in Section 3, which discusses best $L_{\infty}$ approximation by Lipschitz functions with box constraints. The key link here is that Pasch-Hausdorff envelopes preserve monotonicity. Under continuity (Theorem 3.4) we provide a more convenient representation of the minimal distance between a nondecreasing function and its best Lipsichtz approximation. Section 4 considers the related problem of best $L_{\infty}$ approximation by monotone functions with box constraints, generalizing Ubhaya’s results. Finally, Section 4 contains also the announced results on directional differentiability (Theorem 4.3 and Corollary 4.4).

2 The set of trimmings in the $L_{\infty}$-topological setting

Since probabilities on $(\mathbb{R},\beta)$ are determined by their distribution functions (d.f.’s in the sequel) and (1) and (2) can be equivalently stated in terms of the corresponding distribution functions, we will use the same notation $R_{\alpha}(F)$ and $\mathcal{V}_{\alpha}(F_{0})$, with the same meanings as before, but defined in terms of distribution functions. On the other hand, the Kolmogorov distance between probabilities is defined just through the $L_{\infty}$-distance between the corresponding d.f.’s, but we will often keep the notation $d_{K}$ for this distance.

The set $R_{\alpha}(F)$ can be also characterized, as shown in [1] (see also Proposition 2.2 in [2] for a more general result), in terms of the set of $\alpha$-trimmed versions of the uniform probability $U(0,1)$. Notice that this set is just $\mathcal{C}_{\alpha}$, as defined in (6). The parameterization, obtained through the composition of the functions $h$ and $F$: $F_{h}=h\circ F$ gives

[TABLE]

We note that, as a consequence, the “trimmed Kolmogorov distance” from $F$ to $F_{0}$ is

[TABLE]

The set $R_{\alpha}(F)$ is convex and also well behaved w.r.t. weak convergence of probabilities and widely employed probability metrics (see Section 2 in [2]). We show next that this also holds for $d_{K}$.

Proposition 2.1

For $\alpha\in(0,1)$ and distribution functions $F$, $F_{0},F_{1},F_{2},G_{1}$ and $G_{2}$, we have:

• (a)

$R_{\alpha}(F)$* is compact w.r.t. $d_{K}$.*

• (b)

$d_{K}(F_{0},R_{\alpha}(F))=\min_{\tilde{F}\in R_{\alpha}(F)}\|\tilde{F}-F_{0}\|=\min_{h\in\mathcal{C}_{\alpha}}\|h\circ F-F_{0}\|$.

• (c)

$|d_{K}(G_{1},R_{\alpha}(F_{1}))-d_{K}(G_{2},R_{\alpha}(F_{2}))|\leq\|G_{1}-G_{2}\|+{\textstyle\frac{1}{1-\alpha}}\|F_{1}-F_{2}\|.$**

Proof. By the Ascoli-Arzelà Theorem, $\mathcal{C}_{\alpha}$ is a compact subset of the space of continuous functions on $[0,1]$ endowed with the uniform norm. Hence, from any sequence of elements in $R_{\alpha}(F)$, say $\{h_{n}\circ F\}$ (recall (7)), we can extract a uniformly convergent subsequence $h_{n_{j}}\to h_{0}\in\mathcal{C}_{\alpha}$. But then, obviously, $h_{n_{j}}\circ F\to h_{0}\circ F$ in $d_{K}$, which proves (a). Since, on the other hand,

[TABLE]

we see that the map $h\mapsto\|h\circ F-F_{0}\|$ is continuous and, consequently, it attains its minimum in $R_{\alpha}(F)$, as claimed in (b). Finally, to check (c) we note that

[TABLE]

and

[TABLE]

Now, (2) and (9) yield (c).  $\Box$

Proposition 2.1 guarantees the existence of optimal $L_{\infty}$-approximations to every distribution function $F_{0}$ by $\alpha$-trimmed versions of $F$:

[TABLE]

It also shows, through (3), that for $\alpha\in[0,1)$

[TABLE]

Moreover, by convexity of $R_{\alpha}(F)$, the set of optimally trimmed versions of $F$ associated to problem (10) is also convex. However, guarantying uniqueness of the minimizer (as it holds w.r.t. $L_{2}$- Wasserstein metric by Corollary 2.10 in [2]) is not possible here.

An additional consequence of Proposition 2.1 is the continuity of $d_{K}(F_{0},R_{\alpha}(F))$ in $F_{0}$ and $F$. We quote this and some additional facts in our next result.

Proposition 2.2

For $\alpha\in[0,1)$, if $\{F_{n}\}$ and $F$ are d.f.’s such that $d_{K}(F_{n},F)\to 0,$ then:

• a)

for every $\tilde{F}\in R_{\alpha}(F),$ there exist $\tilde{F}_{n}\in R_{\alpha}(F_{n}),n\in\mathbb{N}$ such that $d_{K}(\tilde{F}_{n},\tilde{F})\to 0.$

• b)

if $\tilde{F}_{n}\in R_{\alpha}(F_{n}),n\geq 1$, then there exists some $d_{K}$-convergent subsequence $\{\tilde{F}_{n_{k}}\}$. If $\tilde{F}$ is the limit of such a subsequence, necessarily $\tilde{F}\in R_{\alpha}(F)$.

• c)

if, additionally, $\{G_{n}\}$ and $G$ are d.f.’s such that $d_{K}(G_{n},G)\to 0,$ then $d_{K}(G_{n},$ $R_{\alpha}(F_{m}))\to d_{K}(G,R_{\alpha}(F))$ as $n,m\to\infty.$

Proof. To prove a), since $\tilde{F}=h\circ F$, with $h\in\mathcal{C}_{\alpha}$, it suffices to consider $\tilde{F}_{n}:=h\circ F_{n}\in R_{\alpha}(F_{n})$ and recall that $h$ is Lipschitz. For b), we write $\tilde{F}_{n}=h_{n}\circ F_{n}$ and argue as in the proof of Proposition 2.1 to get a $d_{K}$-convergent subsequence $h_{n_{k}}\to h\in\mathcal{C}_{\alpha}$ from which we easily get $d_{K}(h_{n_{k}}\circ F_{n_{k}},h\circ F)\to 0.$ Finally c) is a direct consequence of Proposition 2.1 (c).  $\Box$

By Polya’s uniform convergence theorem, if $F$ and $G$ are continuous and $\{F_{n}\},\{G_{n}\}$ are sequences of d.f.’s which, respectively, weakly converge to $F,G$, then they also converge in the $d_{K}$-sense, therefore $d_{K}(G_{n},R_{\alpha}(F_{m}))\to d_{K}(G,R_{\alpha}(F))$ holds. Also, a direct application of the Glivenko-Cantelli theorem and item c) above guarantee the following strong consistency result.

Proposition 2.3

Let $\alpha\in[0,1)$ and $\{F_{n}\}$ be the sequence of empirical d.f.’s based on a sequence $\{X_{n}\}$ of independent random variables with distribution function $F$. If $\{G_{n}\}$ is any sequence of distribution functions $d_{K}$-approximating the d.f. $G$ (i.e. $d_{K}(G_{n},G)\to 0$), then:

[TABLE]

Given a d.f. $F$, we write $F^{-1}$ for the associated quantile function (or left continuous inverse function), namely, $F^{-1}(t):=\inf\{x|\ t\leq F(x)\}$. We recall that if $U$ is a uniformly distributed $U(0,1)$ random variable, $F^{-1}(U)$ has d.f. $F$. Similarly, if $X$ has a continuous d.f. $F$, the composed function $F_{0}\circ F^{-1}$ is the quantile function associated to the r.v. $Y=F_{0}(X)$ . As we show next, under some regularity assumptions $d_{K}(F_{0},R_{\alpha}(F))$ can be expressed in terms of the function $F_{0}\circ F^{-1}$. We will see later the usefulness of this fact both for the asymptotic analysis and the practical computation of $d_{K}(F_{0},R_{\alpha}(F_{n}))$ when $F_{n}$ is an empirical d.f. based on a data sample $x_{1},\dots,x_{n}$. Recall that then $F_{n}(x):=\frac{1}{n}\sum_{i=1}^{n}I_{(-\infty,x]}(x_{i})$.

Lemma 2.4

Let $\alpha\in[0,1)$. If $F,F_{0}$ are continuous d.f.’s and $F$ is additionally strictly increasing then

[TABLE]

Proof. For the first identity observe that

[TABLE]

On the other hand, if $x_{(i)},i=1,\dots,n,$ denote the ordered sample associated to $x_{1},\dots,x_{n}$ (the same set of values but ordered in nondecreasing sense) and

[TABLE]

Taking into account that $h(F_{n})$ and $F_{0}(F_{n}^{-1})$ are piecewise constant while $F_{0}$ and $h$ are non decreasing and continuous, we obtain

[TABLE]

and the other identity follows from Proposition 2.1, part (b).  $\Box$

Our final result in this section provides a simple representation of $\min_{h\in\mathcal{C}_{\alpha}}\|h-F_{0}\circ F^{-1}\|$ (hence, of $d_{K}(F_{0},R_{\alpha}(F))$). In this statement we assume that $\Gamma$ is a nondecreasing function taking values in $[0,1]$ (which is always the case if $\Gamma=F_{0}\circ F^{-1}$). Note that taking right and left limits at 0 and 1, respectively, we can assume that $F_{0}\circ F^{-1}$ is a nondecreasing (and left continuous) function from $[0,1]$ to $[0,1]$.

Theorem 2.5

Let $\alpha\in[0,1)$. Assume $\Gamma:[0,1]\to[0,1]$ is a nondecreasing function. Define $G(t)=\Gamma(t)-\frac{t}{1-\alpha}$, $U(t)=\sup_{t\leq s\leq 1}G(s)$, $L(t)=\inf_{0\leq s\leq t}G(s)$ and

[TABLE]

Then,

[TABLE]

The proof of this result will be developed in Section 3. In fact Theorem 3.3 is just a rephrasing of this result. A look at that Theorem shows that $h_{\alpha}=\tilde{h}_{\alpha}+\frac{\cdot}{1-\alpha}$ is an element of $\mathcal{C}_{\alpha}$ such that $\|h_{\alpha}-\Gamma\|=\min_{h\in\mathcal{C}_{\alpha}}\|h-\Gamma\|$, that is, $h_{\alpha}$ is an optimal trimming function in the sense described above. We recall that we do not claim uniqueness of this minimizer, but this particular choice allows to compute $d_{K}(F_{0},R_{\alpha}(F_{n}))$ for sample d.f.’s. Moreover, Theorem 2.5 even provides a simple way for the computation of $d_{K}(F_{0},R_{\alpha}(F))$ for theoretical distributions. Let us see an illustration of this use.

Example 2.1** (Trimmed Kolmogorov distances in the Gaussian model.)**

Consider the case $F_{0}=\Phi$, $F=\Phi((\cdot-\mu)/\sigma)$, where $\Phi$ denotes the standard normal d.f., $\mu\in\mathbb{R}$ and $\sigma>0$. Here we have $H^{-1}(t):=F_{0}\circ F^{-1}(t)=\Phi(\mu+\sigma\Phi^{-1}(t))$. We note that $w(t):=(H^{-1})^{\prime}(t)\leq 1/(1-\alpha)$ if and only if $p(\Phi^{-1}(t))\geq 0$, where

[TABLE]

To avoid cumbersome computations we focus on the cases $\sigma=1$, $\mu\neq 0$ and $\mu=0$, $\sigma\neq 1$.

If $\sigma=1$ and $\mu>0$ then $p$ is linear with positive slope and we see that $w(t)\leq 1/(1-\alpha)$ if and only if t\geq t_{0}=\Phi\big{(}-\frac{\mu}{2}+\frac{1}{\mu}\log(1-\alpha)\big{)}. This means that $G(s)=H^{-1}(s)-s/(1-\alpha)$ is increasing in $[0,t_{0}]$ and decreasing in $[t_{0},1]$. Since, $H^{-1}(0)=G(0)=0$, we have that, $\tilde{h}_{\alpha}(t)=0$ for $t\in[0,t_{1}]$, where $t_{1}\in(t_{0},1)$ is (the unique) solution to $G(t_{1})=0$, and $\tilde{h}_{\alpha}(t)=G(t)$ for $t\in[t_{1},1]$. We conclude that $d_{K}(R_{\alpha}(N(\mu,1)),N(0,1))=G(t_{0})$. The case $\mu<0$ can be handled similarly to obtain

[TABLE]

We focus now on the case $\mu=0$. If $\sigma^{2}<1$, $p$ is a parabola with negative leading coefficient and discriminant $\Delta^{2}=8(\sigma^{2}-1)\log(\sigma(1-\alpha))>0$. Hence, $p(x)$ is positive for $x\in(x_{a},x_{b})$ with $x_{a}=-\frac{\Delta}{2(1-\sigma^{2})}$, $x_{b}=\frac{\Delta}{2(1-\sigma^{2})}$. Equivalently, $w(t)\leq 1/(1-\alpha)$ if and only if $t_{a}:=\Phi(x_{a})\leq t\leq t_{b}:=\Phi(x_{b})$. This means that $G$ is increasing in $[0,t_{a})$, decreasing in $[t_{a},t_{b}]$, increasing in $(t_{b},1]$, $G(0)=0$ and $G(1)=-\alpha/(1-\alpha)$. Arguing as above, we have $\tilde{h}_{\alpha}(t)=\min(G(t),0)$ for $0\leq t\leq\frac{1}{2}$, $\tilde{h}_{\alpha}(t)=\max(G(t),-\frac{\alpha}{1-\alpha})$ for $\frac{1}{2}\leq t\leq 1$, $\tilde{h}_{\alpha}(t_{a})=0$ and $\tilde{h}_{\alpha}(t_{b})=\frac{-\alpha}{1-\alpha}$. We conclude that $d_{K}(R_{\alpha}(N(\mu,\sigma^{2})),N(0,1))=G(t_{a})-\tilde{h}_{\alpha}(t_{a})=\tilde{h}_{\alpha}(t_{b})-G(t_{b})$. Hence,

[TABLE]

If $1\leq\sigma\leq 1/(1-\alpha)$ then we have that $w(t)\leq 1/(1-\alpha)$ for all $t$ and $h_{0}=H^{-1}\in\mathcal{C}_{\alpha}$. In particular, $d_{K}(R_{\alpha}(N(0,\sigma^{2})),N(0,1))=0$.

Finally, we consider the case $\sigma>1/(1-\alpha)$. In this case $p$ is positive for $x\notin[x_{a},x_{b}]$ with $x_{a}=-\frac{\Delta}{2(\sigma^{2}-1)}$, $x_{b}=\frac{\Delta}{2(\sigma^{2}-1)}$. This means that $(H^{-1})^{\prime}(t)>\frac{1}{1-\alpha}$ for $t\in(t_{a},t_{b})$ with $t_{a}=\Phi(x_{a}),t_{b}=\Phi(x_{b})$. Therefore, $G$ is decreasing in $[0,t_{a})$, increasing in $[t_{a},t_{b}]$, decreasing in $(t_{b},1]$, $G(0)=0$ and $G(1)=-\alpha/(1-\alpha)$. Hence, $\tilde{h}_{\alpha}(t)=\max(G(t),\frac{G(t)+G(t_{b})}{2})$, $0\leq t\leq t_{a}$, $\tilde{h}_{\alpha}(t)=\frac{G(t_{a})+G(t_{b})}{2}$, $t_{a}\leq t\leq t_{b}$, $\tilde{h}_{\alpha}(t)=\min(G(t),\frac{G(t_{a})+G(t)}{2})$, $t_{b}\leq t\leq 1$. In particular, $d_{K}(R_{\alpha}(N(0,\sigma^{2})),N(0,1))=\tilde{h}_{\alpha}(t_{a})-G(t_{a})=G(t_{b})-\tilde{h}_{\alpha}(t_{b})=\frac{1}{2}(G(t_{b})-G(t_{a}))$, that is,

[TABLE]

$\Box$**

3 Best $L_{\infty}$-approximations by Lipschitz-continuous functions with box constraints

In this section we refresh the notation. The role of $1/(1-\alpha)$ will be played now by a generic Lipschitz constant $L$; our $\Gamma$ will be substituted by a bounded function $f:\aleph\to\mathbb{R}$, where $(\aleph,d)$ is (at least at the beginning) a general metric space, while we maintain $[0,1]$ as the range of values. We will also use the notation $x\lor y$ (resp. $x\land y$) for the maximum (resp. minimum) of both numbers (or functions). Regarding the Lipschitz norm, recall the trivial inequalities

[TABLE]

The first lemma collects some basic properties on the role of the Pasch-Hausdorff envelopes of a function to obtain a Lipschitz-continuous best $L_{\infty}$-approximation with constrained Lipschitz constant. For the sake of completeness, we will also include a simple proof.

Lemma 3.1

For a function $f:\aleph\to[0,1]$, given a constant $L\geq 0,$ let us consider

[TABLE]

• (i)

This defines functions $f_{L,1},f_{L,2}:\aleph\to\mathbb{R}$ such that $0\leq f_{L,1}\leq f_{L,2}\leq 1.$

• (ii)

$f_{L,1}$* is the pointwise largest function $g:\aleph\to\mathbb{R}$ satisfying $g\leq f$ and $\|g\|_{\rm Lip}\leq L$. Likewise $f_{L,2}$ is the pointwise smallest function $g:\aleph\to\mathbb{R}$ satisfying $g\geq f$ and $\|g\|_{\rm Lip}\leq L$.*

• (iii)

The average $f_{L}:=(f_{L,1}+f_{L,2})/2$ satisfies $\|f_{L}\|_{\rm Lip}\leq L$ and

[TABLE]

for any function $g:\aleph\to\mathbb{R}$ such that $\|g\|_{\rm Lip}\leq L.$

Proof. Part (i) follows directly from the definitions of $f_{L,1}$ and $f_{L,2}$, because, for every $x\in\aleph$:

[TABLE]

To address part (ii) observe that, for arbitrary $x_{1},x_{2},y\in\aleph$, the triangle inequality for the distance implies $|Ld(x_{1},y)-Ld(x_{2},y)|\leq Ld(x_{1},x_{2}),$ leading to the inequalities

[TABLE]

thus to $\|f_{L,j}\|_{\rm Lip}\leq L,j=1,2.$ Now, if $g:\aleph\to\mathbb{R}$ satisfies $g\leq f$ and $\|g\|_{\rm Lip}\leq L,$ then for $x,y\in\aleph$: $g(x)\leq g(y)+Ld(x,y)$ with equality if $x=y$. Hence

[TABLE]

Analogously, it follows from $g\geq f$ and $\|g\|_{\rm Lip}\leq L$ that $g\geq f_{L,2},$ proving (ii).

As to part (iii), let $\epsilon:=\|g-f\|.$ Then $\|g\pm\epsilon\|_{\rm Lip}=\|g\|_{\rm Lip}$ and $g-\epsilon\leq f\leq g+\epsilon.$ Consequently, by part (ii),

[TABLE]

This implies that

[TABLE]

whence

[TABLE]

Since $\|f_{L}\|_{\rm Lip}\leq\|f_{L,1}\|_{\rm Lip}/2+\|f_{L,2}\|_{\rm Lip}/2\leq L,$ taking $g=f_{L}$ gives the announced equality $\|f_{L}-f\|=\|f_{L,2}-f_{L,1}\|/2.$  $\Box$

When $\aleph$ is a real interval and $f$ is non-decreasing, the functions $f_{L,1}$ and $f_{L,2}$ in Lemma 3.1 share also that property and can be alternatively expressed in terms of the the Ubhaya’s monotone envelopes of the function $f(x)-Lx$. This is the content of the following lemma.

Lemma 3.2

Let $\aleph$ be a real interval, equipped with the usual distance $d(x,y)=|x-y|.$ If $f:\aleph\to[0,1]$ is non-decreasing, then the functions $f_{L,1},f_{L,2}$ in Lemma 3.1 are non-decreasing too, and for arbitrary $x\in\aleph$ and $j=1,2,$

[TABLE]

where $\gamma_{L,j},j=1,2$ are the non-increasing functions

[TABLE]

In particular,

[TABLE]

Proof. The representations of $f_{L,1}$ and $f_{L,2}$ in terms of $\gamma_{L,1}$ and $\gamma_{L,2}$ follow from the fact that for arbitrary $x,y\in\aleph,$

[TABLE]

[TABLE]

where the inequalities follow from $f$ being non-decreasing. Note that both functions $\gamma_{L,1}$ and $\gamma_{L,2}$ are non-increasing, but adding the term $Lx$ to them leads to non-decreasing functions: For $x_{1},x_{2}\in\aleph$ with $x_{1}<x_{2},$ isotonicity of $f$ implies that

[TABLE]

and

[TABLE]

because $f_{L,1}\leq f\leq f_{L,2}.$  $\Box$

Finally, let us include in the problem the boundary restrictions.

Theorem 3.3

Let $f:[0,1]\to[0,1]$ be non-decreasing. For $L\geq 1$ consider the function

[TABLE]

where $\gamma_{L}:=(\gamma_{L,1}+\gamma_{L,2})/2,$ and $f_{L},\gamma_{L,1},\gamma_{L,2}$ are defined as in Lemmas 3.1 and 3.2. Then $\tilde{f}_{L}:[0,1]\to\mathbb{R}$ is non-decreasing and verifies $\tilde{f}_{L}(0)=0$ and $\tilde{f}_{L}(1)=1$ and $\|\tilde{f}_{L}\|_{\rm Lip}\leq L,$ and for arbitrary functions $g:[0,1]\to\mathbb{R}$ with $g(0)=0$ and $g(1)=1$ and $\|g\|_{\rm Lip}\leq L,$

[TABLE]

Proof. Let us begin noting that both expressions for $\tilde{f}_{L}$ are trivially equivalent from the relations between $\gamma_{L,j}$ and $f_{L,j}.$

That $\tilde{f}_{L}$ verifies the required properties easily follows from the preceding lemmas (recall also inequalities (14)). Let then $g:[0,1]\to\mathbb{R}$ with $\|g\|_{\rm Lip}\leq L$. Also by the precedent lemmas,

[TABLE]

Under the additional constraint that $g(0)=0$, for arbitrary $x\in[0,1],$

[TABLE]

whence

[TABLE]

Analogously, the additional constraint $g(1)=1$ implies that

[TABLE]

whence

[TABLE]

These considerations show that for any function $g:[0,1]\to\mathbb{R}$ verifying the conditions $g(0)=0,$ $g(1)=1$ and $\|g\|_{\rm Lip}\leq L,$

[TABLE]

The function $\tilde{f}_{L}$ satisfies the previous constraints on $g$, too, so

[TABLE]

It remains to prove the reverse inequality. For $x\in[0,1]$, we have to distinguish three cases: If $1-L+Lx\leq f_{L}(x)\leq Lx,$ then $\tilde{f}_{L}(x)=f_{L}(x),$ so $|\tilde{f}_{L}(x)-f(x)|\leq\|f_{L}-f\|.$ If $f_{L}(x)>Lx,$ then $\tilde{f}_{L}(x)=Lx,$ and

[TABLE]

Similarly, if $f_{L}(x)<1-L+Lx,$ then $\tilde{f}_{L}(x)=Lx,$ and

[TABLE]

$\Box$

In the case, considered in Theorem 3.3, of a non-decreasing function $f$, since the functions $f_{L,j}$ are absolutely continuous and the relations $\gamma_{L,j}=f_{L,j}-Lx$ hold, all the functions $f_{L},\gamma_{L},\gamma_{L,j}$ are absolutely continuous so $\{\gamma_{L}\leq 1-L\},\{\gamma_{L}\geq 0\},\{\gamma_{L}\in[1-L,0]\}$ are compact sets and continuous functions attain their maximum values on these sets. This allows to get alternative expressions for (16) as given in the following theorem. We note that here and throughout we use the convention that the $\max$ over an empty set equals $-\infty$.

Theorem 3.4

Let $f:[0,1]\to[0,1]$ be non-decreasing and continuous and assume the notation in Theorem 3.3. Then the following alternative expressions for (16) hold:

[TABLE]

Here, we used the notation $\mathcal{T}_{1}=\{x\in[0,1]:\gamma_{L}(x)\geq 0\},$ $\mathcal{T}_{2}=\{x\in[0,1]:\gamma_{L}(x)\leq 1-L\},$ $\mathcal{T}_{3}=\{(y,x):0\leq y\leq x\leq 1,1-L\leq\frac{1}{2}(f(y)+f(x)-L(y+x))\leq 0\}.$

Once we know Theorem 3.3, a proof of this result would take advantage of the fact that the right-hand side in (17) is upper bounded by the same expression with the unrestricted maxima, which, by (15) is just the right-hand side in (16) when $f$ is continuous. However, with some additional effort we can obtain a more general result that does not requires the monotonicity assumption on the objective function and opens a way to address the directional differentiability of the functional $f\to\|f-\tilde{f}_{L}\|$. Both goals will be carried through the following section.

4 Best $L_{\infty}$-approximations by monotone functions with box constraints

The following theorem gives appropriate characterizations of the best approximation of a bounded function (in uniform norm) by monotone functions with a box constraint. Without this constraint, best approximation by monotone functions in the $L_{\infty}$-norm has been considered in [27, 28], with results that cover the case $A=-\infty$, $B=\infty$ in Theorem 4.1 below. Notice that this theorem, based on Ubhaya’s envelopes, would also provide an (arguably more involved) alternative proof for Theorem 3.3. Notice that the function $G$ plays the role of the transformed function, $f(x)-Lx$ (the difference of two nondecreasing functions) in the previous section, while the scope here is general.

Theorem 4.1

Assume $G:[0,1]\to\mathbb{R}$ is a bounded function and $-\infty\leq A\leq B\leq\infty$. Define $U(x)=\sup_{x\leq y\leq 1}G(y)$, $L(x)=\inf_{0\leq y\leq x}G(y)$, $\bar{G}(x)=(L(x)+U(x))/2$ and

[TABLE]

Then $U,L,\bar{G}$ and $\bar{G}_{A,B}$ are nonincreasing, $L(x)\leq G(x)\leq U(x)$ and for every nonincreasing $h:[0,1]\to[A,B]$ we have

[TABLE]

Furthermore, if $G$ is continuous then $U,L,\bar{G}$ and $\bar{G}_{A,B}$ are also continuous and

[TABLE]

where $\mathcal{T}_{1}=\{x\in[0,1]:\,\bar{G}(x)\geq B\}$, $\mathcal{T}_{2}=\{x\in[0,1]:\,\bar{G}(x)\leq A\}$ and $\mathcal{T}_{3}=\{(y,x):\,0\leq y\leq x\leq 1,A\leq\frac{1}{2}(G(y)+G(x))\leq B\}$.

Proof. The bounds $L(x)\leq G(x)\leq U(x)$ are obvious, and also the fact that $U$ and $L$ are nonincreasing (hence, also $\bar{G}$ and $\bar{G}_{A,B}$).

$\bullet$   Next, consider some nonincreasing $h:[0,1]\to[A,B]$ and $x\in[0,1]$. Since $L(x)\leq G(x)\leq U(x)$, we have that $G(x)=\bar{G}(x)$ whenever $U(x)=L(x)$. Hence, if $U(x)=L(x)\in[A,B]$ we have $\bar{G}_{A,B}(x)=G(x)$ and, consequently,

[TABLE]

$\bullet$   Obviously, $\bar{G}_{A,B}(x)=B$ if $U(x)=L(x)>B$ and we still have that

[TABLE]

and similarly for the case $U(x)=L(x)<A$.

$\bullet$   It remains to deal with the case $U(x)>L(x)$. For every $\varepsilon>0$ there exist $x_{a}\in[0,x]$, $x_{b}\in[x,1]$ such that $G(x_{a})<L(x)+\varepsilon$ and $G(x_{b})>U(x)-\varepsilon$. If $\bar{G}(x)>B$ then $\bar{G}_{A,B}(x)=B$. Using again that $L(x)\leq G(x)\leq U(x)$ we see that $|\bar{G}_{A,B}(x)-G(x)|\leq U(x)-B<G(x_{b})-B+\varepsilon\leq|G(x_{b})-h(x_{b})|+\varepsilon$ for small enough $\varepsilon$, showing that $|\bar{G}_{A,B}(x)-G(x)|\leq\|h-G\|$.

Similarly, if $\bar{G}(x)<A$ we conclude that $|\bar{G}_{A,B}(x)-G(x)|\leq\|h-G\|$.

Finally, assume that $U(x)>L(x)$ and $\bar{G}(x)\in[A,B]$. Since $h$ is nonincreasing we have that $h(x_{a})\geq h(x_{b})$ and, consequently,

[TABLE]

for $\varepsilon$ small enough. This completes the proof of (19).

To check continuity of $U$ note that for $0\leq y<x\leq 1$ $U(y)=\max(U(x),\max_{y\leq z\leq x}G(z))$. Now, given $\varepsilon>0$ we can fix $\delta>0$ such that $|G(x)-G(y)|\leq\varepsilon$ whenever $|y-x|\leq\delta$. But then $|U(y)-U(x)|\leq\varepsilon$ if $|y-x|\leq\delta$, proving continuity of $U$. $L$ can be handled similarly. As a consequence we see that $\bar{G}$ and $\bar{G}_{A,B}$ are also continuous.

Now, to prove the first equality in the statement we take $x\in[0,1]$ and consider first the case $x\in\mathcal{T}_{1}$. Note that, necessarily, $U(x)\geq B$, $U(x)-B\geq B-L(x)$ and $\bar{G}_{A,B}(x)=B$.

$\bullet$   If $G(x)\geq B$ then $|G(t)-\bar{G}_{A,B}(x)|=G(x)-B$.

$\bullet$   Assume, on the contrary, that $G(x)<B$. Set $x_{+}=\inf\{y\leq x:\,G(y)=U(x)\}$. By continuity, $G(x_{+})=U(x)=U(x_{+})$.

Now, if $\bar{G}(x_{+})\geq B$ then $G(x_{+})-B=U(x)-B\geq B-L(x)\geq B-G(x)=|G(x)-\bar{G}_{A,B}(x)|$. If, on the contrary, $\bar{G}(x_{+})<B$, then there exists $x^{\prime}\in[x,x_{+}]$ such that $\bar{G}(x^{\prime})\in(A,B)$. But we must have $U(x^{\prime})=U(x)=U(x_{+})$ and $L(x^{\prime})<L(x)$ and, consequently, we have that

[TABLE]

Summarizing, we see that

[TABLE]

Similarly,

[TABLE]

and, obviously, if $\bar{G}(x)\in[A,B]$ then $\bar{G}_{A,B}(x)=\bar{G}(x)$ and $|G(x)-\bar{G}_{A,B}(x)|\leq\frac{1}{2}(U(x)-L(x))$, which implies that

[TABLE]

Now combining (21), (22) and (23) we see that

[TABLE]

Assume now that $x_{0}$ is such that $\bar{G}(x_{0})\geq B$. Then $\bar{G}_{A,B}(x_{0})=B$ and $G(x_{0})-B\leq|G(x_{0})-\bar{G}_{A,B}(x_{0})|$. This implies $\max_{\bar{G}(x)\geq B}(G(x)-B)\leq\|G-\bar{G}_{A,B}\|$.

Similarly, $\max_{\bar{G}(t)\leq A}(A-G(x))\leq\|G-\bar{G}_{A,B}\|$.

Finally, suppose $x_{0}$ is such that $\bar{G}(x_{0})\in[A,B]$ and

[TABLE]

$\bullet$   If $U(x_{0})=L(x_{0})$ then

[TABLE]

$\bullet$   If $U(x_{0})>L(x_{0})$ then we set $x_{+}=\inf\{y\in[x_{0},1]:\,G(y)=U(x_{0})\}$. Then $U(y)=U(x_{0})$ for $y\in[x_{0},x_{+}]$ and

[TABLE]

Set $x_{+}=\sup\{y\in[0,x_{0}]:\,G(y)=L(x_{0})\}$. We have $L(y)=L(x_{0})=G(x_{-})$ for $y\in[x_{-},x_{0}]$. We claim that

[TABLE]

To check (24) note that, if $\bar{G}(x_{0})>A$ and (24) fails then we could find $y\in[x_{0},x_{+}]$ with $L(y)<L(x_{0})$, $\bar{G}(y)\in(A,B]$ and $U(y)-L(y)>U(x_{0})-L(x_{0})$, while if $\bar{G}(x_{0})=A$ and (24) fails then $G(y)<L(x_{0})$ for some $y\in(x_{0},x_{+})$, $\bar{G}(y)<A$ and $A-L(y)>A-L(x_{0})=\frac{1}{2}(U(x_{0}-L(x_{0})$, against the assumption on $x_{0}$.

Hence, from (24) we conclude that $\bar{G}(x_{+})=\bar{G}(x_{0})\in[A,B]$ and $|G(x_{+})-\bar{G}_{A,B}(x_{+})|=\frac{1}{2}(U(x_{0})-L(x_{0}))$, showing that $\frac{1}{2}(U(x_{0})-L(x_{0}))\leq\|G-\bar{G}_{A,B}\|$. Combining the last estimates we see that the first equality in (20) holds.

For the second identity we note that arguing as above we see that $U(x_{0})-L(x_{0})=G(x)-G(y)$ for some $(y,x)\in\mathcal{T}_{3}$ if $\bar{G}(x_{0})\in[A,B]$. Assume, on the other hand, that $(y_{0},x_{0})\in\mathcal{T}_{3}$ satisfies

[TABLE]

$\bullet$   We consider first the case $\frac{1}{2}(G(y_{0})+G(x_{0}))\in(A,B)$.

We claim that $U(x_{0})=G(x_{0})$ since, otherwise, there exists $x^{\prime}>x_{0}$ such that $\frac{1}{2}(G(y_{0})+G(x^{\prime}))\in(A,B)$ and $G(x^{\prime})>G(x_{0})$ and this would imply $G(x^{\prime})-G(y_{0})>G(x_{0})-G(y_{0})$, against the assumption.

Similarly, we see that $G(y_{0})=L(x_{0})$.

Furthermore, $L(x)=L(y_{0})$ for $x\in[y_{0},x_{0}]$. If $G(x_{0})<U(x_{0})$ then there exists $x^{\prime}>x_{0}$ such that $\frac{1}{2}(G(y_{0})+G(x^{\prime}))\in(A,B)$ and $G(x^{\prime})>G(x_{0})$, but then $G(x^{\prime})-G(y_{0})>G(x_{0})-G(y_{0})$, contradicting maximality of $(y_{0},x_{0})$. Similarly we see that $G(y_{0})=L(y_{0})$ and also that $L(x)=L(y_{0})$ for $x\in[y_{0},x_{0}]$. Hence, $G(x_{0})-G(y_{0})=U(x_{0})-L(x_{0})$ and $\bar{G}(x_{0})\in(A,B)$.

$\bullet$   In the case $\frac{1}{2}(G(y_{0})+G(x_{0}))=B$ we have that necessarily $G(x_{0})\geq B$ and, arguing as above, we see that $G(y_{0})=L(y)$ for all $y\in[y_{0},x_{0}]$. This implies that $\bar{G}(x_{0})\geq B$ and $\frac{1}{2}(G(x_{0})-G(y_{0}))=G(x_{0})-B$.

$\bullet$   Arguing similarly for the case $\frac{1}{2}(G(y_{0})+G(x_{0}))=A$ we conclude that the second equality in (20) holds. $\Box$

Remark 4.2

The sets of optimizers within $\mathcal{T}_{1},\mathcal{T}_{2}$ and $\mathcal{T}_{3}$ in Lemma 4.1 play an important role in the next results. For convenience, we denote ${T}_{1}=\{x_{0}\in\mathcal{T}_{1}:\,G(x_{0})-B=\|G-\bar{G}_{A,B}\|\}$, ${T}_{2}=\{x_{0}\in\mathcal{T}_{2}:\,A-G(x_{0})=\|G-\bar{G}_{A,B}\|\}$ and ${T}_{3}=\{(y_{0},x_{0})\in\mathcal{T}_{3}:\,\frac{1}{2}(G(x_{0})-G(y_{0}))=\|G-\bar{G}_{A,B}\|\}$. A look at the proof of Lemma 4.1 shows that if $x_{0}\in T_{1}$ then $G$ has a local maximum at $x_{0}$ and a local minimum if $x_{0}\in T_{2}$. Also, if $(y_{0},x_{0})\in T_{3}$ then $G$ has a local maximum at $x_{0}$ and a local minimum at $y_{0}$.

Our next result addresses the directional differentiability of the functional $G\to\|G-\bar{G}_{A,B}\|$ that appeared in the last theorem. This kind of result typically allows to obtain efficiency and asymptotic distributional behaviour of functionals in the statistical setting (see e.g. [9]). In fact it allows to prove the Central Limit Theorem for the statistical functional $d_{K}(F_{0},R_{\alpha}(F_{n}))$ (see Theorem 4.1 in [7]).

Theorem 4.3

Assume $G,J:[0,1]\to\mathbb{R}$ are continuous functions and $r_{n}>0$ is a sequence of real numbers such that $r_{n}\to\infty$. Define $G_{n}=G+\frac{J}{r_{n}}$ and consider $\bar{G},\bar{G}_{A,B}$ as in Theorem 4.1 and $\bar{G}_{A,B,n}$ built in the same way as $G_{A,B}$ but from $G_{n}$. Assume further that $T_{1},T_{2}$ and $T_{3}$ are as in Remark 4.2 and that there is no $x\in{T}_{1}$ with $\bar{G}(x)=B$, no $x\in{T}_{2}$ with $\bar{G}(x)=A$ and no $(y,x)\in{T}_{3}$ with $\frac{1}{2}(G(x)+G(y))\in\{A,B\}$. Then

[TABLE]

Proof. We use the notation $U,L$ from Theorem 4.1 and write $U_{n},L_{n},\bar{G}_{n},T_{n,i}$ for the corresponding objects coming from $G_{n}$. Observe that $\|U_{n}-U\|\leq\|J\|/r_{n}\to 0$ and, similarly, $\|\bar{G}_{n}-\bar{G}\|\to 0$. Assume that $x\in{T}_{1}$. By assumption and the last convergence we have that $\bar{G}_{n}(x)>B$ for large enough $n$ and, therefore, $\|G_{n}-\bar{G}_{A,B,n}\|\geq(G_{n}(t)-B)$. But this implies

[TABLE]

Arguing similarly for ${T}_{2}$ and ${T}_{3}$ we conclude that

[TABLE]

For the upper bound assume $x_{n}\in{T}_{n,1}$ (that is, $x_{n}\in\mathcal{T}_{n,1}$ such that $G_{n}(x_{n})-B=\|G_{n}-\bar{G}_{A,B,n}\|)$. By compactness, taking subsequences if necessary, we can assume that $x_{n}\to x_{0}$ for some $x_{0}\in[0,1]$ with $\bar{G}(x_{0})\geq B$ and $G(x_{0})-B=\|G-\bar{G}_{A,B}\|$. But this means that $x_{0}\in{T}_{1}$. Hence, by assumption $G(x_{0})>B$ and, consequently, $G(x_{n})>B$ for large enough $n$. In this case $\|G-\bar{G}_{A,B}\|\geq(G(x_{n})-B)$, which implies that

[TABLE]

With the same argument applied to ${T}_{2}$ and ${T}_{3}$ we conclude that

[TABLE]

and complete the proof. $\Box$

Specializing the last results for $G(x)=f(x)-Lx$, where $f$ is nondecreasing, $L\geq 1$ a constant, and $A=1-L,B=0$, we can obtain a first result on the directional differentiability of the functional $f\to\|f-\tilde{f}_{L}\|$ considered in Section 3. Note that now, recovering the notation in that section, the relevant sets are $\mathcal{T}_{1},\mathcal{T}_{2}$ and $\mathcal{T}_{3}$ as defined in Theorem 3.4, and ${T}_{1}=\{x_{0}\in\mathcal{T}_{1}:\,f(x_{0})-Lx_{0}=\|f-\tilde{f}_{L}\|\}$, ${T}_{2}=\{x_{0}\in\mathcal{T}_{2}:\,1-L+Lx_{0}-f(x_{0})=\|f-\tilde{f}_{L}\|\}$ and ${T}_{3}=\{(y_{0},x_{0})\in\mathcal{T}_{3}:\,\frac{1}{2}(f(x_{0})-f(y_{0})-L(x_{0}-y_{0}))=\|f-\tilde{f}_{L}\|\}$. Theorem 4.3 translates then to the following immediate corollary.

Corollary 4.4** (Directional differentiability.)**

Let $f,f_{n}:[0,1]\to\mathbb{R}$ be nondecreasing functions, $r_{n}>0$ a sequence of real numbers such that $r_{n}\to\infty$ and $r_{n}(f_{n}-f)\to J$ pointwise, where $J:[0,1]\to\mathbb{R}$ is a continuous function. Assume further that $f$ is continuous, that $T_{1},T_{2}$ and $T_{3}$ are as above and that there is no $x\in{T}_{1}$ with $\gamma_{L}(x)=0$, no $x\in{T}_{2}$ with $\gamma_{L}(x)=1-L$ and no $(y,x)\in{T}_{3}$ with $\frac{1}{2}(f(x)+f(y)-L(x+y))\in\{1-L,0\}$. Let $\tilde{f}_{n,L},\tilde{f}_{L}$ respectively denote the best $L_{\infty}$-approximations to $f_{n}$ and $f$ by Lipschitz-continuous functions $h:[0,1]\to\mathbb{R}$ with $\|h\|_{\rm Lip}\leq L$ and verifying $h(0)=0,h(1)=1$, as in Theorem 3.3. Then

[TABLE]