Outer limits of subdifferentials for min-max type functions

TL;DR

This paper extends the outer subdifferential concept to min-max functions and clarifies the relationship between the outer subdifferential of a support function and the end set of its subdifferential, addressing open questions.

Contribution

It generalizes the outer subdifferential for min-max functions and resolves an open problem about the connection between support functions and subdifferential end sets.

Findings

Extended outer subdifferential to pointwise minima of Lipschitz functions

Provided a relation between the outer subdifferential of support and subdifferential end set

Addressed an open question in subdifferential theory

Abstract

We generalise the outer subdifferential constructon suggested by C\'anovas, Henrion, L\'opez and Parra for max type functions to pointwise minima of regular Lipschitz functions. We also answer an open question about the relation between the outer subdifferential of the support of a regular function and the end set of its subdifferential posed by Li, Meng and Yang.

Equations

$$S(\bar{x})=\{x\in X|f(x)\le f(\bar{x})\},$$

$$L\mathrm{dist}\left(x,S(\bar{x})\right)\le \max \{0,f\left(x\right)-f\left(\bar{x}\right)\}$$

$$\mathrm{Er}f\left(\bar{x}\right):=\underset{\begin{array}{c}x\to \bar{x}\\ f\left(x\right)>f\left(\bar{x}\right)\end{array}}{\mathrm{liminf}}\frac{f\left(x\right)-f\left(\bar{x}\right)}{\mathrm{dist}\left(x,S(\bar{x})\right)}.$$

$$\mathrm{Er}f\left(\bar{x}\right)\ge \mathrm{dist}\left(0,\underset{\genfrac{}{}{0ex}{}{x\to \bar{x}}{f\left(x\right)\downarrow f\left(\bar{x}\right)}}{\mathrm{Limsup}}\partial f(x)\right),$$

$$f^{\prime }(x;p)=\underset{\genfrac{}{}{0ex}{}{t\downarrow 0}{p^{\prime }\to p}}{\lim }\frac{f(x+t{p}^{\prime })-f(x)}{t}$$

$$f^{\prime }(x;\lambda p)=\lambda f^{\prime }(x;p)\forall p\in {\mathbb{R}}^n,\lambda >0.$$

$$f(\bar{x}+s)=f(\bar{x})+f^{\prime }(\bar{x};s)+o(s),\frac{o(s)}{\Vert s\Vert }\underset{s\to 0}{⟶}0.$$

$$f^{\prime }(\bar{x};p_k)=\frac{f(\bar{x}+t_k{p}_k)-f(\bar{x})}{t_k}+{\delta }_k,|{\delta }_k|<1/k.$$

$$\underset{k\to \infty }{\lim }\frac{f(\bar{x}+t_k{p}_k)-f(\bar{x})}{t_k}=f^{\prime }(\bar{x};p).$$

$$\underset{k\to \infty }{\lim }{f}^{\prime }(\bar{x};p_k)=\underset{k\to \infty }{\lim }\frac{f(\bar{x}+t_k{p}_k)-f(\bar{x})}{t_k}+\underset{k\to \infty }{\lim }{\delta }_k=f^{\prime }(\bar{x};p),$$

$$\frac{|f(\bar{x}+s_k)-f(\bar{x})-f^{\prime }(\bar{x};s_k)|}{\Vert s_k\Vert }>c\forall k\in \mathbb{N}.$$

$$0$$

$$=\left|f^{\prime }(\bar{x};p)-\underset{k\to \infty }{\lim }\frac{f^{\prime }(\bar{x};s_k)}{\Vert s_k\Vert }\right|\text{(using eqrefeq:HadDDHomogeneous)}$$

$$=\left|\underset{k\to \infty }{\lim }\frac{f(\bar{x}+\Vert s_k\Vert p_k)-f(\bar{x})}{\Vert s_k\Vert }-\underset{k\to \infty }{\lim }\frac{f^{\prime }(\bar{x};s_k)}{\Vert s_k\Vert }\right|$$

$$=\underset{k\to \infty }{\lim }\frac{|f(\bar{x}+s_k)-f(\bar{x})-f^{\prime }(\bar{x};s_k)|}{\Vert s_k\Vert }\ge c,$$

$$\partial f(\bar{x})=\left\{v\in {\mathbb{R}}^n|\underset{x\to \bar{x},x\ne \bar{x}}{\mathrm{liminf}}\frac{f(x)-f(\bar{x})-⟨v,x-\bar{x}⟩}{\Vert x-\bar{x}\Vert }\ge 0\right\}.$$

$$\partial f(\bar{x})=\left\{v\in {\mathbb{R}}^n|f^{\prime }(\bar{x};p)\ge ⟨v,p⟩\forall p\in {\mathbb{R}}^n\right\}.$$

$$f^{\prime }(\bar{x};p)=\underset{v\in \partial f(\bar{x})}{\max }⟨v,p⟩\forall p\in {\mathbb{R}}^n;$$

$$\partial f(x)$$

$$=\{v\in {\mathbb{R}}^n|f^{\prime }(x;p)\ge ⟨v,p⟩\forall p\in \mathcal{S}\}.$$

$$h(x)=\underset{v\in C}{\max }⟨x,v⟩,$$

$$\partial h(x)=\underset{v\in C}{\mathrm{Argmax}}⟨x,v⟩.$$

$$f(x)=\underset{i\in I}{\min }{f}_i(x)\forall x\in X,$$

$$f_i^{\prime }(\bar{x};p)=\underset{v\in \partial f_i(\bar{x})}{\max }⟨v,p⟩\forall p\in {\mathbb{R}}^n,$$

$$f_i(\bar{x}+s)=f_i(\bar{x})+f_i^{\prime }(\bar{x};s)+o_i(s),\frac{o_i(s)}{\Vert s\Vert }\underset{s\to 0}{⟶}0.$$

$$I(x)=\{i\in I|f(x)=f_i(x)\},$$

$$I(x,p)=\left\{i_{0}\in I(x)\,\Bigl{|}\,\max_{v\in\partial f_{i_{0}}(x)}\langle v,p\rangle=\min_{i\in I(x)}\max_{v\in\partial f_{i}(x)}\langle v,p\rangle\right\}.$$

$$f^{\prime }(\bar{x};p)=\underset{i\in I(\bar{x})}{\min }{f}_i^{\prime }(\bar{x};p)\forall p\in \mathcal{S}.$$

$$\underset{\genfrac{}{}{0ex}{}{t\downarrow 0}{p^{\prime }\to p}}{\lim }\frac{f(\bar{x}+t{p}^{\prime })-f(\bar{x})}{t}$$

$$=\underset{\genfrac{}{}{0ex}{}{t\downarrow 0}{p^{\prime }\to p}}{\lim }\frac{{\min }_{i\in I(\bar{x})}[f_i(\bar{x}+t{p}^{\prime })-f_i(\bar{x})]}{t}$$

$$=\underset{i\in I(\bar{x})}{\min }\underset{\genfrac{}{}{0ex}{}{t\downarrow 0}{p^{\prime }\to p}}{\lim }\frac{f_i(\bar{x}+t{p}^{\prime })-f_i(\bar{x})}{t}=\underset{i\in I(\bar{x})}{\min }{f}_i^{\prime }(\bar{x},p).$$

$$f(x)=\underset{i\in I}{\min }{f}_i(x)\forall x\in X,$$

$$\partial f(\bar{x})=\bigcap _{i\in I(\bar{x})}\partial f_i(\bar{x}).$$

Full text

1 Introduction

Our motivation for the study of outer limits of subdifferentials is the problem of constructive evaluation of error bounds. The error bound modulus measures whether a given function is steep enough locally outside of its level set. This idea stems from the works of Hoffman [9] and Łojasiewicz [16]. Error bounds are crucial for a range of stability questions, for the existence of exact penalty functions, and for the convergence of numerical methods. The literature on error bounds is vast, and we refer the reader to the following selection of recent works and classic review papers for more details [3, 14, 13, 7, 6, 18, 22, 1, 2]. In this work we focus on the constructive evaluation of error bound modulus for structured continuous functions.

Let $f:X\to\mathbb{R}$ be a continuous function defined on an open set $X\subseteq\mathbb{R}^{n}$. We define the sublevel set

[TABLE]

where $\bar{x}\in X$. We say that $f$ has a local (linear) error bound at $\bar{x}$ if there exists a constant $L>0$ such that

[TABLE]

for all points $x$ in a sufficiently small neighbourhood of $\bar{x}$. Here $\operatorname*{dist}(x,A)=\inf_{v\in A}\|x-a\|$ is the distance from $x$ to $A$. Taking the supremum over all constants $L$ that satisfy (1) over all neighbourhoods of $\bar{x}$ we arrive at an exact quantity called the error bound modulus of $f$ at $\bar{x}$, which can be explicitly expressed as

[TABLE]

It is possible to obtain sharp estimates of the error bound modulus $\operatorname*{Er}f\left(\bar{x}\right)$ for sufficiently structured functions by means of subdifferential calculus. For continuous functions that we are considering in this paper the error bound modulus is bounded from below by the distance from zero to the outer limits of Fréchet subdifferentials (see [6]),

[TABLE]

with equality holding when $f$ is sufficiently regular (for instance convex), see [6, Theorem 5 and Proposition 10]. In [15] this equality is proved for a lower $C^{1}$ function and an additional upper estimate of the error bound of a regular locally Lipschitz function is given via the distance to the outer limits of the Fréchet subdifferentials of the subdifferential support function, and such limits are in turn expressed using the notion of the end of a closed convex set introduced in [10]. We emphasise here that the inequality (2) can be used to establish the inequality $\operatorname*{Er}f\left(\bar{x}\right)>0$, hence, the computation of the outer limits of subdifferentials in the right-hand side of (2) is of significant interest even when the equality does not hold. Note that the outer limits of subdifferentials are called outer (limiting) subdifferentials (see [1]).

In this paper we generalise some of the constructive results of [4] to the case of min-max type functions, providing an exact description for the outer limits of subdifferentials in the case of polyhedral functions and sharp bounds for a more general case (see Theorems 1 and 3). We also strengthen Theorem 3.2 of [4] in Corollary 3 by dropping the affine independence assumption (although the latter result can probably be obtained from the findings of [15]). Finally, we answer in the affirmative the open question of [15] for the case of functions with sublinear Hadamard directional derivative (see Corollary 2 and Remark 2).

As we work in a finite-dimensional real space, throughout the paper we use the standard scalar product $\langle x,y\rangle=x^{T}y$, and denote the Euclidean norm by $\|x\|$. We also denote the closed unit ball and the unit sphere by $\mathcal{B}$ and $\mathcal{S}$ respectively.

2 Preliminaries

Recall that a function $f:X\to\mathbb{R}$, where $X$ is an open subset of $\mathbb{R}^{n}$, is Hadamard directionally differentiable at $x\in X$ for $p\in\mathbb{R}^{n}$ if the limit

[TABLE]

exists and is finite. The quantity $f^{\prime}(x;p)$ is called the (Hadamard) directional derivative of $f$ at $x$ in the direction $p$. It follows from the definition that the directional derivative is a positively homogeneous function of degree one, i.e. if $f$ is Hadamard directionally differentiable at $x\in X$, then

[TABLE]

Hadamard directionally differentiable functions enjoy certain continuity properties that we summarise in the next proposition. These properties are well-known (e.g. see [5]), but we provide a proof here for convenience. Throughout the paper we assume that $X$ is an open subset of $\mathbb{R}^{n}$.

Proposition 1**.**

Let $f:X\to\mathbb{R}$ be Hadamard directionally differentiable at $\bar{x}\in X$. Then the directional derivative $f^{\prime}(\bar{x};\cdot)$ is a continuous function; moreover,

[TABLE]

Proof.

We first show that the Hadamard directional derivative is continuous. Choose an arbitrary $p\in\mathbb{R}^{n}$ and a sequence $\{p_{k}\}$, $p_{k}\to p$. By the definition of the directional derivative for every $k\in\mathbb{N}$ there exist $t_{k}$ such that $0<t_{k}<1/k$ and

[TABLE]

Since $t_{k}\downarrow 0$, $p_{k}\to p$, and $f$ is Hadamard directionally differentiable at $\bar{x}$, we have

[TABLE]

Now passing to the limit on both sides of (5) and using (6), we obtain

[TABLE]

and so the directional derivative is continuous.

It remains to show the relation (4). Assume the contrary. Then there is $\bar{x}\in X$, a constant $c>0$ and a sequence $\{s_{k}\}$, $s_{k}\to 0$ such that

[TABLE]

Without loss of generality we can assume that $s_{k}/\|s_{k}\|=:p_{k}\to p\in\mathcal{S}$, then from the continuity of $f^{\prime}(\bar{x};\cdot)$ we get

[TABLE]

which is impossible by our assumption that $c>0$. ∎

In this work our focus is on the functions with sublinear Hadamard directional derivatives and finite minima of such functions. The former are called subdifferentiable functions in [5], however this notation is not universally accepted (e.g. in [12] subdifferentiable functions are the ones with nonempty Fréchet subdifferential). To avoid possible confusion with definitions, throughout the paper we sacrifice brevity for clarity and use the full description. Regular Lipschitz functions have sublinear Hadamard directional derivatives, see [19, Theorem 9.16], therefore all results obtained here for functions with sublinear Hadamard directional derivatives also apply to regular Lipschitz functions.

Recall that the Fréchet subdifferential of a function $f:X\rightarrow\mathbb{R}$ at $\bar{x}\in X$ is the set

[TABLE]

For Hadamard directionally differentiable function $f:X\to\mathbb{R}$ one has (see [12, Proposition 1.17])

[TABLE]

When the directional derivative is sublinear, the Fréchet subdifferential of $f$ at $\bar{x}\in X$ coincides with the subdifferential of the directional derivative at [math], so we have

[TABLE]

moreover, for a convex function $f:\mathbb{R}^{n}\to\mathbb{R}$ the Fréchet subdifferential coincides with the classic Moreau-Rockafellar subdifferential,

[TABLE]

We will be using the following result explicitly (see [8, Chap. VI, Example 3.1]).

Proposition 2**.**

Let $h:\mathbb{R}^{n}\to\mathbb{R}$ be a sublinear function,

[TABLE]

where $C\subset\mathbb{R}^{n}$ is a compact convex set. Then

[TABLE]

We will also utilise the following optimality condition (see Corollary 1.12.3 in [12]).

Proposition 3**.**

Let $f_{1}:X\to\mathbb{R}$ and $f_{2}:X\to\mathbb{R}$ and assume that $f_{1}$ is Fréchet differentiable at $x$. If $f_{1}+f_{2}$ attains a local minimum at $x$, then $-\nabla f_{1}(x)\subset\partial f_{2}(x)$.

Let $f:X\to\mathbb{R}$ be a pointwise minimum of a finite set of functions with sublinear Hadamard directional derivatives. We have explicitly

[TABLE]

where $f_{i}:X\to\mathbb{R}$ are Hadamard directionally differentiable at $\bar{x}\in X$ with sublinear directional derivatives, so that (see (8))

[TABLE]

and $I$ is a finite index set. Observe that the relation (4) is valid for each individual function $f_{i}$, $i\in I$, so that we have

[TABLE]

Throughout the paper we use the following two active index sets.

[TABLE]

[TABLE]

We will need the following well known relation (see [5]).

Proposition 4**.**

Let $f:X\to\mathbb{R}$ be a pointwise minimum of a finite number of functions with sublinear Hadamard directional derivatives, as in (11). Then

[TABLE]

Proof.

The proof follows from the definition of directional derivative. Indeed, for all $x$ in a sufficiently small neighbourhood of $\bar{x}$ we have $I(x)\subset I(\bar{x})$. Therefore

[TABLE]

∎

The next relation is well known (see [20] for the discussion of more general calculus rules for Fréchet subdifferentials) and follows directly from the definition of the Fréchet subdifferential and Proposition 4. We provide a proof here for the sake of completeness.

Proposition 5**.**

Let $f:X\to\mathbb{R}$ be a finite minimum of Hadamard directionally differentiable functions with sublinear derivatives at $\bar{x}\in X$. Then the Fréchet subdifferential of $f$ at $\bar{x}\in X$ is the intersection of the Fréchet subdifferentials of the active functions. In other words, given $f:X\to\mathbb{R}$ such that

[TABLE]

where $I$ is a finite index set, and $f_{i}:X\to\mathbb{R}$ are Hadamard directionally differentiable with sublinear directional derivatives at $\bar{x}\in X$, one has

[TABLE]

Proof.

First of all, from Proposition 4 we have

[TABLE]

Using (7) and (15) we have

[TABLE]

∎

Proposition 6**.**

Let $g:X\to\mathbb{R}$ be a pointwise maximum of a finite number of $C^{1}(X)$ functions, i.e. $g(x)=\max_{j\in J}g_{j}(x)$, $g_{j}\in C^{1}(X)$, and $|J|<\infty$. Then $g$ has a nonempty Fréchet subdifferential that can be expressed explicitly as

[TABLE]

where $J(x)$ is the active index set. Moreover, the function $g$ is Hadamard directionally differentiable with

[TABLE]

Proof.

This result is well known, and its proof can be easily deduced from the fact that the Hadamard directional derivative is the support function of the convex hull in (16). ∎

Proposition 7**.**

Let $f=\min_{i}f_{i}$, where $f_{i}$ are Hadamard directionally differentiable with sublinear directional derivatives at $\bar{x}\in X$. Then for every $p\in\mathcal{S}$ there exists $\varepsilon=\varepsilon(\bar{x},p)>0$ such that

[TABLE]

Proof.

Suppose that the claim is not true. Then there exist sequences $\{\varepsilon_{k}\}$, $\{t_{k}\}$ and $\{u_{k}\}$ such that $\varepsilon_{k}\downarrow 0$, $t_{k}\in(0,\varepsilon_{k}]$, $u_{k}\in\mathcal{B}$ and for

[TABLE]

we have $I(x_{k})\setminus I(\bar{x},p)\neq\emptyset$. Without loss of generality assume that there is an $i_{0}\in I$ such that $i_{0}\in I(x_{k})\setminus I(\bar{x},p)$. Observe that

[TABLE]

hence, by the continuity of $f$, $i_{0}\in I(\bar{x})$; moreover, observing that $p+\varepsilon_{k}u_{k}\to p$, we have

[TABLE]

We then have from Proposition 4

[TABLE]

hence, $i_{0}\in I(\bar{x},p)$, which contradicts our assumption. ∎

3 Limiting subdifferential for pointwise minima

Our results rely on the following technical lemma, whose proof is inspired by the proofs of fuzzy mean value theorems for Fréchet subdifferential (see [11, 17]). To show the existence of a nearby point with a desired subgradient, an auxiliary function is constructed which attains a local minimum at such point.

Lemma 1**.**

Let $f:X\to\mathbb{R}$ be a pointwise minimum of finitely many functions with sublinear Hadamard directional derivatives at $\bar{x}\in X$, as in (11). Then for every $p\in\mathcal{S}$ and

[TABLE]

there exist sequences $\{x_{k}\}$and $\{y_{k}\}$ such that

[TABLE]

Proof.

Fix $p\in\mathcal{S}$ and $y$ such that

[TABLE]

Observe that by the relation (12) and by the positive homogeneity of the directional derivative (3) we have for all $i\in I(\bar{x},p)$

[TABLE]

For any $\lambda>0$ define the function $\varphi_{\lambda}:X\to\mathbb{R}$ as follows

[TABLE]

We will show that for sufficiently small $\lambda$ a minimum of the function $\varphi_{\lambda}$ on the ball $\bar{x}+\lambda p+\lambda\varepsilon\mathcal{B}$ is attained at an interior point (here $\varepsilon\in(0,\min(\varepsilon(\bar{x},p),1))$, where $\varepsilon(\bar{x},p)$ comes from Proposition 7). Note here that since $X$ is an open set, there exists $r\in(0,1)$ such that $\bar{x}+r\mathcal{B}\subset X$. If $\lambda$ is smaller than $r/2$, then $\bar{x}+\lambda p+\lambda\varepsilon\mathcal{B}\subset\bar{x}+r\mathcal{B}\subset X$. We will assume that our $\lambda$ is always chosen small enough to satisfy this condition, and also that $\lambda<\varepsilon(\bar{x},p)$ (see Proposition 7).

Observe that the function $\varphi_{\lambda}$ is continuous, and hence it attains its minimum on the ball $\bar{x}+\lambda p+\lambda\varepsilon\mathcal{B}$. Assume that contrary to what we want to prove, there exist $\{\lambda_{k}\}$ and $\{u_{k}\}$ such that $\lambda_{k}\downarrow 0$, $u_{k}\in\mathcal{S}$ and

[TABLE]

We therefore have

[TABLE]

or explicitly

[TABLE]

By our choice of $\varepsilon$ and $\lambda$, Proposition 7 yields that

[TABLE]

Without loss of generality, due to the finiteness of $I(\bar{x},p)$, we can assume that the index set is constant, i.e.

[TABLE]

which together with (3) yields

[TABLE]

Notice that by our choice of $y$ we have from the relations (12) and (18) for all $i\in\tilde{I}\subset I(\bar{x})$

[TABLE]

and

[TABLE]

Noticing that $\|u_{k}\|=1$, substituting (21) and (22) into (3), dividing the whole expression by $\lambda_{k}$, we obtain for every $i\in\tilde{I}$

[TABLE]

where $o_{i}(\cdot)$’s are as in (13). It is not difficult to see that the right hand side goes to zero as $k\to\infty$, and hence $\varepsilon^{2}=0$, which contradicts our choice of a fixed positive $\varepsilon$.

We have shown that our assumption is wrong, and given a fixed $\varepsilon>0$ for sufficiently small $\lambda(\varepsilon)$ the function $\varphi_{\lambda}$ has a local minimum in the interior of the ball $\bar{x}+\lambda p+\varepsilon\lambda\mathcal{B}$ for all $\lambda\in(0,\lambda(\varepsilon))$; in other words, it attains an unconstrained local minimum at this point.

Let $\{\varepsilon_{k}\}$ be such that $\varepsilon_{k}\downarrow 0$, and choose

[TABLE]

For each $k\in\mathbb{N}$ there exists a point $u_{k}$ in the interior of $\mathcal{B}$ such that $\bar{x}+\lambda_{k}p+\varepsilon_{k}\lambda_{k}u_{k}$ is a minimum of the function $\varphi_{\lambda}$ on the ball $\bar{x}+\lambda_{k}p+\varepsilon_{k}\lambda_{k}\mathcal{B}$. From the optimality condition in Proposition 3 we have

[TABLE]

Observe that $y_{k}\to y$ and for $x_{k}:=\bar{x}+\lambda_{k}p+\lambda_{k}\varepsilon_{k}u_{k}$ we have $x_{k}\to\bar{x}$ and

[TABLE]

so we are done. ∎

We use Lemma 1 to obtain an inclusion relation for the outer limits of subdifferentials.

Theorem 1**.**

Let $f:X\to\mathbb{R}$ be as in (11). Then

[TABLE]

where $I(x,p)$ is the index set as defined in (14).

Proof.

For $\bar{x}\in X$ choose any direction $p\in\mathcal{S}$ such that $f^{\prime}(\bar{x};p)>0$. We will show that for

[TABLE]

we have

[TABLE]

By Lemma 1 there exist sequences $\{x_{k}\}$ and $\{y_{k}\}$ such that $y_{k}\in\partial f(x_{k})$, $y_{k}\to y$, $x_{k}\to\bar{x}$ and

[TABLE]

It remains to show that for sufficiently large $k$ we have $f(x_{k})>f(\bar{x})$. Assume this is not so. Then without loss of generality, $f(x_{k})\leq f(\bar{x})$ for all $k\in\mathbb{N}$, and

[TABLE]

which contradicts our choice of $p$. ∎

We have the following corollary that follows directly from Theorem 1 and the fact that the limit on the right-hand side of (23) is a closed set.

Corollary 1**.**

Let $f:X\to\mathbb{R}$ be as in (11). Then

[TABLE]

where $I(x,p)$ is the index set as defined earlier.

We would like to point out that the result of the computation of the expression on the left hand side of (23) and (24) depends on the position of zero with respect to the subdifferential. The following example taken from [21, Example 3.2] illustrates this observation.

Example 1**.**

Consider the two functions

[TABLE]

Notice that $f=\min\{f_{1},f_{2}\}$ is nonnegative everywhere, and $f(x,y)=0$ iff $(x,y)=0_{2}$ (see the plots in Fig. 1).

It is not difficult to observe that the subdifferentials of $f_{1}$ and $f_{2}$ at zero are unit disks centred at $(\frac{1}{2},0)$ and $(-\frac{1}{2},0)$ respectively (see Example 3.2 in [20] for detailed explanation). The left-hand side in (23) for the function $f$ at $0_{2}$ is the union of two (open) semi-circles, see Fig. 2. Observe that the closure of this set coincides with the outer limit on the right hand side of (23), so in fact (24) is an exact characterisation in this case.

We next modify this example by translating the subdifferentials and obtaining a different set on the left hand side of (23).

Consider the modified functions

[TABLE]

The minimum function $\tilde{f}(x,y)=\min\{\tilde{f}_{1}(x,y),\tilde{f}_{2}(x,y)\}$ is no longer nonnegative (see Fig. 3.

Similar to the previous example, the subdifferentials of $f_{1}$ and $f_{2}$ at zero are unit disks centred at $(\frac{3}{2},0)$ and $(\frac{1}{2},0)$ respectively. The left-hand side in (23) is the union of one semi-circle and two smaller segments of the other circle, see Fig. 4.

We have the following useful special case of Theorem 1.

Corollary 2**.**

Let $f:X\to\mathbb{R}$ be Hadamard directionally differentiable at $\bar{x}\in X$, assume that the directional derivative $f^{\prime}(\bar{x};\cdot)$ is a sublinear function, then

[TABLE]

Remark 2**.**

Observe that in the notation of [15] the closure of the union on the left hand side of (25) coincides with the outer limit of the Fréchet subdifferentials of the support of $\partial f(\bar{x})$. Hence we answer affirmatively the open question of [15] on whether such outer limit is a subset of the right hand side of (25).

In Corollary 3 we strengthen Theorem 3.2 of [4], dropping the affine independence assumption. We first recall the notation from [4] and the related geometric constructions.

Let $g:X\to\mathbb{R}$ be a pointwise maximum of smooth functions, i.e.

[TABLE]

where $J$ is a finite index set. As in [4] define the collection $\mathcal{D}(\bar{x})$ of index subsets $D\subset J(\bar{x})$ such that the following system is consistent with respect to $d$

[TABLE]

Corollary 3**.**

Let $g(x)$ to be the pointwise maximum of smooth functions as in (26). Then

[TABLE]

in other words, in [4, Theorem 3.2] the subsets $\mathcal{D}_{AI}(\bar{x})$ can be replaced by $\mathcal{D}(\bar{x})$.

Moreover when all $\{g_{j}\}_{j\in J}$ are affine we have an identity (instead of an inclusion) in (28).

Proof.

We begin by showing the following identity:

[TABLE]

Observe that explicitly the equality below holds for all $p$ (see Proposition 6)

[TABLE]

hence, for each direction $p$ that features in the union on the right hand side of (29) the relevant $\operatorname*{Arg\,max}$ gives the support face of the subdifferential. Explicitly, fix $p\in\mathcal{S}$ and let

[TABLE]

then

[TABLE]

We now get back to the definition of our index subsets $\mathcal{D}$. The system (27) is consistent for some nonempty $D\subset J(\bar{x})$ and $d\in\mathbb{R}^{n}\setminus\{0\}$ if and only if for $p=d/\|d\|$

[TABLE]

and

[TABLE]

hence we get (29). The last inclusion of (28) follows from Corollary 2.

Finally, to show that in the affine case an equality holds in (29), observe that there is a sufficiently small neighbourhood $N(\bar{x})$ of $\bar{x}$ on which the affine function $g$ coincides with the sum $g(\bar{x})+\sigma_{\partial}g(\bar{x})(x-\bar{x})$, where $\sigma_{\partial}g(\bar{x})(\cdot)$ is the support of the subdifferential, and hence for any $x$ in this neighbourhood we have $\partial g(x)=\partial\sigma_{\partial}g(\bar{x})(x-\bar{x})$. Since the number of different subdifferentials of points in this neighbourhood is finite, the right hand side is in fact the union

[TABLE]

where the last equality follows from Proposition 2. Note here that $g(x)>g(\bar{x})$ if and only if $\partial\sigma_{\partial}g(\bar{x})(x-\bar{x})>0$ or equivalently $g^{\prime}(\bar{x},x-\bar{x})>0$. ∎

In the expression (29), the $\operatorname*{Arg\,max}$ construction gives the support faces of the subdifferential, while the positivity constraint on the directional derivative means that zero lies in the same (strict) half-space of some hyperplane exposing the face as the rest of the subdifferential. We consider a very simple example that demonstrates how this construction works.

Example 2**.**

Let $g=\min_{1\in\{1,\dots,6\}}g_{i}$, where $g_{i}:\mathbb{R}^{2}\to\mathbb{R}$ are linear functions,

[TABLE]

[TABLE]

Observe that each of these functions is active at zero, so we have $J(0)=\{1,2,\dots,6\}$. Furthermore, we have the gradients

[TABLE]

[TABLE]

These gradients are shown in the left hand side image of Fig. 5.

It is not difficult to see that

[TABLE]

the relevant set

[TABLE]

is shown in Fig. 2 with thick solid lines.

If we replace the function $g_{4}$ with $g^{\prime}_{4}=2x_{1}-x_{2}$, and construct the relevant arrangement of the gradients $a_{1},a_{2},a_{3},a^{\prime}_{4},a_{5},a_{6}$, we would lose the index subsets $\{4\}$ and $\{4,5\}$, since now zero is on the line connecting $a_{4}^{\prime}$ with $a_{5}$, and there is no $d$ that would make (27) consistent for $D=\{4\}$ and $D=\{4,5\}$ (see the image on the right-hand side of Fig. 2).

The next example is taken from [15]. It shows that even for smooth functions it is not always possible to replace the inclusion in (28) with an equality.

Example 3**.**

Let $g:\mathbb{R}^{2}\to\mathbb{R}$ be defined as the piecewise maximum of two smooth functions,

[TABLE]

where

[TABLE]

The graph of $g$ is shown in Fig. 6.

It is clear from the illustration that the directional derivative at the point $x=0_{2}$, where the level set $\{x\,|\,g(x)=0\}$ touches the linear ‘slice’ of the graph, the Hadamard directional derivative is a piecewise linear function, however the outer limits of subdifferentials capture the liming gradients that come from the gradients of the curved parts of the graph, and this results in the underestimation of the outer limit. We have (see the explanation in [15]),

[TABLE]

however checking the consistency of the system (27) for different index subsets, $\{1\},\{2\},\{1,2\}$, it is easy to see that this system has a solution $d$ only for $\{2\}$, hence, $\mathcal{D}(0)=\{2\}$, and the left-hand side of (28) gives

[TABLE]

*We have hence ‘lost’ the second gradient. *

4 Exact representations for piecewise affine functions

We are now ready to generalise Theorem 3.1 from [4]. We first prove that for positively homogeneous functions the inclusion (23) can be replaced by an equality.

Lemma 2**.**

Let $h:\mathbb{R}^{n}\to\mathbb{R}$ be a pointwise minimum of a finite number of sublinear functions, i.e.

[TABLE]

where $C_{i}$ is a compact convex set for each $i\in I$. Then

[TABLE]

Proof.

Observe that the inclusion “$\subseteq$” in (30) follows directly from Theorem 1 substituting $\bar{x}=0$, observing that $h^{\prime}(0;p)=h(p)$, $h(0)=0$ (so $h(p)>0$ iff $h^{\prime}(0;p)>0$) and that the right hand side is a closed set. It remains to show the reverse inclusion. Choose any

[TABLE]

There exist sequences $\{x_{k}\}$ and $\{y_{k}\}$ such that $x_{k}\to 0$, $y_{k}\to y$ and $y_{k}\in\partial h(x_{k})$. We have by Proposition 5

[TABLE]

furthermore, Proposition 2 yields

[TABLE]

and hence

[TABLE]

Observe that since $x_{k}\neq 0$ and $h_{i}$’s are positively homogeneous, we have

[TABLE]

Together with (31) these observations yield

[TABLE]

and hence the limit of the sequence $\{y_{k}\}$ must indeed belong to the left hand side of (30). ∎

We are now ready to obtain a generalisation of Theorem 3.1 in [4].

Theorem 3**.**

Let $f:X\to\mathbb{R}$ be as in (11), and in addition assume that for every $i\in I$ the function $f_{i}$ is piecewise affine, i.e.

[TABLE]

where $J_{i}$’s are finite index sets for each $i\in I$. Then

[TABLE]

Proof.

Observe that when the sets $C_{i}$ in Lemma 2 are polyhedral, there is no need for the closure operation in (30), since there are finitely many different faces of each subdifferential, and we therefore have a finite union of closed convex sets which is always closed.

To finish the proof it remains to note that convex polyhedral functions are locally positively homogeneous and coincide with a translation of their first order approximations in a sufficiently small neighbourhood of each point. Since outer limits of subdifferentials are local notions, it is clear that the application of Lemma 2 to the directional derivatives of the active functions yields the required result. ∎

The next example is a practical demonstration of the construction given in Theorem 3.

Example 4**.**

Let $f=\min\{f_{1},f_{2}\}$, where $f_{1},f_{2}:\mathbb{R}^{2}\to\mathbb{R}$ are piecewise linear max-functions,

[TABLE]

The graphs of $f_{1}$, $f_{2}$ and $f$ are shown in Fig 7.

The subdifferentials of $f_{1}$ and $f_{2}$ at zero are easy to compute:

[TABLE]

The subdifferentials are shown in Fig. 8,

where $\partial f_{1}(0)=\mathrm{conv}\{a_{1},a_{2},a_{3},a_{4}\}$ and $\partial f_{2}(0)=\mathrm{conv}\{b_{1},b_{2},b_{3},b_{4}\}$.

To carry out the calculation of the left-hand side expression in (32) it is enough to consider every vertex and face of the subdifferential and to verify that there is an exposing ‘minimal’ hyperplane (line in our case) such that zero belongs to the relevant strictly negative subspace defined by this hyperplane. The relevant vector $p$ is the normal to such hyperplane. The resulting outer limit of subdifferentials is the set

[TABLE]

We leave finer details to the reader. It is also not difficult to see from the graph that the two line segments correspond to the ‘convex’ part of the graph, and the standalone point is the gradient of the linear part at the front of the plot in Fig. 7 (e) which is connected to the rest of the plot in a ‘concave’ fashion.

We highlight here that the index-based representation that is valid for the convex case and that was used in Example 2 can not be generalised directly to the case of a min-max type function. This happens because the intersection of two polytopes of dimension 2 or higher can not always be represented via the convex hull of a subset of vertices of these polytopes, and hence the intersection in the left-hand side of (32) may not be representable as a convex hull of a selection of $a_{ij}$’s.

Note that in the case of a min-max type function the piecewise affine assumption is essential for the equality in (32) to hold. Consider the following semialgebraic example (unfortunately we could not recollect where the idea of this example came from).

Example 5**.**

Let $f=\min\{f_{1},f_{2}\}$, where

[TABLE]

At the point $(x,y)=0_{2}=(0,0)$ we have

[TABLE]

therefore,

[TABLE]

and so the expression on the left hand side of (23) produces the empty set.

We now compute the outer limit directly. We have for the Fréchet subdifferential

[TABLE]

Substituting the values and the gradients we have

[TABLE]

It is not difficult to observe that

[TABLE]

On the other hand, observe that we can construct sequences of points converging to zero along the curves in the regions that correspond to $f_{2}>f_{1}>0$ and $0<f_{2}<f_{1}$ respectively. This works for points on the two curves

[TABLE]

shown in dashed and dotted lines in the last plot of Fig. 10.

5 Acknowledgements

The authors are grateful to Prof. Alex Kruger for insightful comments and corrections and also to Drs Kaiwen Meng and Minghua Li for their comments and explanations that significantly improved both our understanding of the subject and the clarity of the exposition. We are also immensely grateful to the three anonymous Referees for their insightful comments, corrections and suggestions that significantly improved the quality of this paper. The research was partially supported by the Australian Research Council grant DE150100240.