About intrinsic transversality of pairs of sets

TL;DR

This paper extends the concept of intrinsic transversality for pairs of sets to infinite-dimensional spaces, providing new characterizations and focusing on convergence of alternating projections in optimization.

Contribution

It introduces infinite-dimensional extensions of intrinsic transversality and establishes new characterizations involving limiting objects, especially for convex sets.

Findings

Characterizations of intrinsic transversality in infinite dimensions

New limiting objects for pairs of sets

Enhanced understanding of convergence in convex cases

Abstract

The article continues the study of the 'regular' arrangement of a collection of sets near a point in their intersection. Such regular intersection or, in other words, transversality properties are crucial for the validity of qualification conditions in optimisation as well as subdifferential, normal cone and coderivative calculus, and convergence analysis of computational algorithms. One of the main motivations for the development of the transversality theory of collections of sets comes from the convergence analysis of alternating projections for solving feasibility problems. This article targets infinite dimensional extensions of the intrinsic transversality property introduced recently by Drusvyatskiy, Ioffe and Lewis as a sufficient condition for local linear convergence of alternating projections. Several characterisations of this property are established involving new limiting objects defined for pairs of sets. Special attention is given to the convex case.

Figures

Equations

$$N_A(\bar{x})$$

$$x_{2k+1}\in P_B(x_{2k}),x_{2k+2}\in P_A(x_{2k+1})(k=0,1,\dots ),$$

$$x_{k+1}\in P_A{P}_B(x_k)(k=0,1,\dots ).$$

$$\Vert x_k-\hat{x}\Vert \le \alpha c^k(k=1,2,\dots ),$$

$$\alpha d\left(x,A\cap B\right)\le \max \left\{d(x,A),d(x,B)\right\}\text{mbox}forallx.$$

$$\text{transversality}⟹\text{intrinsic transversality}⟹\text{subtransversality},$$

$$N_A(\bar{x}):=\left\{x^{\ast }\in X^{\ast }\mid \underset{a\to \bar{x},a\in A\setminus \{\bar{x}\}}{\mathrm{limsup}}\frac{⟨x^{\ast },a-\bar{x}⟩}{\Vert a-\bar{x}\Vert }\le 0\right\}.$$

$$\partial f(\bar{x}):=\left\{x^{\ast }\in X^{\ast }\mid \underset{x\to \bar{x},x\ne \bar{x}}{\mathrm{liminf}}\frac{f(x)-f(\bar{x})-⟨x^{\ast },x-\bar{x}⟩}{\Vert x-\bar{x}\Vert }\ge 0\right\}.$$

$${\overline{N}}_A(\bar{x}):={\mathrm{Limsup}}_{a\to \bar{x},a\in A}{N}_A(a):=\left\{x^{\ast }=\underset{k\to \infty }{\lim }{x}_k^{\ast }\mid x_k^{\ast }\in N_A(a_k),a_k\in A,a_k\to \bar{x}\right\}.$$

$$N_A^p(\bar{x}):=\mathrm{cone}\left(P_A^{-1}(\bar{x})-\bar{x}\right).$$

$$P_A(x):=\left\{a\in A\mid \Vert x-a\Vert =d_A(x)\right\},x\in X.$$

$$N_A(\bar{x}):=\left\{x^{\ast }\in X^{\ast }\mid ⟨x^{\ast },a-\bar{x}⟩\le 0\text{mbox}foralla\in A\right\}.$$

$$\alpha d\left(x,A\cap B\right)\le \max \left\{d(x,A),d(x,B)\right\}\text{mbox}forallx\in {\mathbb{B}}_{\delta }(\bar{x}).$$

$$\begin{gathered} \alpha d\left(x,(A-x_1)\cap (B-x_2)\right)\le \max \left\{d(x,A-x_1),d(x,B-x_2)\right\} \\ \text{mbox}forallx\in {\mathbb{B}}_{\delta }(\bar{x}),x_1,x_2\in \delta \mathbb{B}. \end{gathered}$$

$$\mathrm{tr}[A,B](\bar{x})\le \mathrm{str}[A,B](\bar{x}).$$

$$\Vert x_1^{\ast }\Vert +\Vert x_2^{\ast }\Vert =1,⟨x_1^{\ast },x^{\prime }-x_1^{\prime }⟩=\Vert x_1^{\ast }\Vert \Vert x^{\prime }-x_1^{\prime }\Vert ,⟨x_2^{\ast },x^{\prime }-x_2^{\prime }⟩=\Vert x_2^{\ast }\Vert \Vert x^{\prime }-x_2^{\prime }\Vert ,$$

$$d(x_1^{\ast },N_A(a^{\prime }))<\delta ,d(x_2^{\ast },N_B(b^{\prime }))<\delta .$$

$$\Vert x_1^{\ast }\Vert +\Vert x_2^{\ast }\Vert =1,⟨x_1^{\ast },x-a⟩=\Vert x_1^{\ast }\Vert \Vert x-a\Vert ,⟨x_2^{\ast },x-b⟩=\Vert x_2^{\ast }\Vert \Vert x-b\Vert ,$$

$$d(x_1^{\ast },N_A(a))<\delta ,d(x_2^{\ast },N_B(b))<\delta .$$

$$\Vert x_1^{\ast }\Vert +\Vert x_2^{\ast }\Vert =1,⟨x_1^{\ast },x-x_1^{\prime }⟩=\Vert x_1^{\ast }\Vert \Vert x-x_1^{\prime }\Vert ,⟨x_2^{\ast },x-x_2^{\prime }⟩=\Vert x_2^{\ast }\Vert \Vert x-x_2^{\prime }\Vert .$$

$${\mathrm{itr}}_w[A,B](\bar{x}):=$$

$$\underset{\begin{array}{c}{x}_1^{\prime }\to a,x_2^{\prime }\to b,a^{\prime }\to a,b^{\prime }\to b\\ a^{\prime }\in A,b^{\prime }\in B,\Vert x-x_1^{\prime }\Vert =\Vert x-x_2^{\prime }\Vert \\ d(x_1^{\ast },N_A(a^{\prime }))<\rho ,d(x_2^{\ast },N_B(b^{\prime }))<\rho ,\Vert x_1^{\ast }\Vert +\Vert x_2^{\ast }\Vert =1\\ ⟨x_1^{\ast },x-x_1^{\prime }⟩=\Vert x_1^{\ast }\Vert \Vert x-x_1^{\prime }\Vert ,⟨x_2^{\ast },x-x_2^{\prime }⟩=\Vert x_2^{\ast }\Vert \Vert x-x_2^{\prime }\Vert \end{array}}{\mathrm{liminf}}\Vert x_1^{\ast }+x_2^{\ast }\Vert ,$$

$${\mathrm{str}}_c[A,B](\bar{x}):=$$

$$\mathrm{itr}[A,B](\bar{x}):=\underset{\begin{array}{c}a\to \bar{x},b\to \bar{x},x\to \bar{x}\\ a\in A\setminus B,b\in B\setminus A,x\ne a,x\ne b\\ x_1^{\ast }\in N_A(a)\setminus \{0\},x_2^{\ast }\in N_B(b)\setminus \{0\},\Vert x_1^{\ast }\Vert +\Vert x_2^{\ast }\Vert =1\\ \frac{\Vert x-a\Vert }{\Vert x-b\Vert }\to 1,\frac{⟨x_1^{\ast },x-a⟩}{\Vert x_1^{\ast }\Vert \Vert x-a\Vert }\to 1,\frac{⟨x_2^{\ast },x-b⟩}{\Vert x_2^{\ast }\Vert \Vert x-b\Vert }\to 1\end{array}}{\mathrm{liminf}}\Vert x_1^{\ast }+x_2^{\ast }\Vert ,$$

$${\mathrm{itr}}_w[A,B](\bar{x})=\underset{\begin{array}{c}a\to \bar{x},b\to \bar{x},x\to \bar{x}\\ a\in A\setminus B,b\in B\setminus A,\Vert x-a\Vert =\Vert x-b\Vert \\ d(x_1^{\ast },{\overline{N}}_A(a))\to 0,d(x_2^{\ast },{\overline{N}}_B(b))\to 0,\Vert x_1^{\ast }\Vert +\Vert x_2^{\ast }\Vert =1\\ ⟨x_1^{\ast },x-a⟩=\Vert x_1^{\ast }\Vert \Vert x-a\Vert ,⟨x_2^{\ast },x-b⟩=\Vert x_2^{\ast }\Vert \Vert x-b\Vert \end{array}}{\mathrm{liminf}}\Vert x_1^{\ast }+x_2^{\ast }\Vert ,$$

$$\Vert x_1^{\ast }\Vert +\Vert x_2^{\ast }\Vert =1,\frac{⟨x_1^{\ast },x-a⟩}{\Vert x_1^{\ast }\Vert \Vert x-a\Vert }>1-\delta ,\frac{⟨x_2^{\ast },x-b⟩}{\Vert x_2^{\ast }\Vert \Vert x-b\Vert }>1-\delta .$$

$$x_{1k}^{\ast }\in N_A(a_k),x_{2k}^{\ast }\in N_B(b_k)(k=1,2,\dots ),$$

$$\frac{\Vert x_k-a_k\Vert }{\Vert x_k-b_k\Vert }\to 1,\frac{⟨x_{1k}^{\ast },x_k-a_k⟩}{\Vert x_{1k}^{\ast }\Vert \Vert x_k-a_k\Vert }\to 1,\frac{⟨x_{2k}^{\ast },x_k-b_k⟩}{\Vert x_{2k}^{\ast }\Vert \Vert x_k-b_k\Vert }\to 1,$$

$$d(x_{1k}^{\ast },N_A(a_k))\to 0,d(x_{2k}^{\ast },N_B(b_k))\to 0,$$

$$⟨x_{1k}^{\ast },x_k-a_k⟩=\Vert x_{1k}^{\ast }\Vert \Vert x_k-a_k\Vert ,⟨x_{2k}^{\ast },x_k-b_k⟩=\Vert x_{2k}^{\ast }\Vert \Vert x_k-b_k\Vert (k=1,2,\dots ).$$

$$x_k^{\ast }\in N_A(a_k)(k=1,2,\dots ),\frac{⟨x_k^{\ast },x_k-a_k⟩}{\Vert x_k^{\ast }\Vert \Vert x_k-a_k\Vert }\to 1,$$

$${\overline{N}}_A(\bar{x};(x_k))\subset {\overline{N}}_A(\bar{x}).$$

$$\mathrm{itr}[A,B](\bar{x})=\underset{\begin{array}{c}(x_1^{\ast },x_2^{\ast })\in {\overline{N}}_{A,B}(\bar{x})\\ \Vert x_1^{\ast }\Vert +\Vert x_2^{\ast }\Vert =1\end{array}}{\min }\Vert x_1^{\ast }+x_2^{\ast }\Vert ,$$

Full text

††dedicatory: Dedicated to the memory of Professor Jonathan Michael Borwein

1 Introduction

This article continues the study of the ‘regular’ arrangement of a collection of sets near a point in their intersection. Such regular intersection or, in other words, transversality properties are crucial for the validity of qualification conditions in optimization as well as subdifferential, normal cone and coderivative calculus, and convergence analysis of computational algorithms. This explains the growing interest of researchers to investigating this class of properties and obtaining primal and dual necessary and/or sufficient conditions in various settings (convex or nonconvex, finite or infinite dimensional, finite or infinite collections); cf. Bauschke and Borwein BauBor93 ; BauBor96 , Ngai and Théra NgaThe01 , Ng and Yang NgYan04 , Bakan et al. BakDeuLi05 , Kruger et al. Kru05 ; Kru06 ; Kru09 ; KruLop12.1 ; KruTha13 ; KruTha14 ; KruTha15 ; KruTha16 ; KruLukTha ; KruLukTha2 , Chong Li et al. LiNg05 ; LiNgPon07 ; LiNg14 , Ng and Zhang NgZan07 , Lewis et al. LewMal08 ; LewLukMal09 , Zheng et al. ZheNg08 ; ZheWeiYao10 , Bauschke et all BauLukPhaWan13.1 ; BauLukPhaWan13.2 , Hesse and Luke HesLuk13 , Drusvyatskiy et al. DruIofLew15 , Noll and Rondepierre NolRon16 . Note also the very well known connections (in fact, equivalences) between transversality properties of collections of sets and the corresponding regularity properties of set-valued mappings. For example, the properties of transversality and subtransversality of pairs of sets correspond in a sense to metric regularity and metric subregularity of set-valued mappings, respectively (cf. Iof00_ ; Iof16 ; Kru05 ; Kru06 ; Kru09 ; KruTha15 ; KruLukTha ; KruLukTha2 ).

Due to the wide variety of applications coming from different areas, some transversality properties together with the corresponding necessary and/or sufficient conditions have been rediscovered many times in different contexts and often under different names. The intrinsic transversality property studied in the current article was originally introduced in 2015 by Drusvyatskiy et al. DruIofLew15 as an important sufficient condition for local linear convergence of alternating projections for solving finite dimensional nonconvex feasibility problems. The new term has not been immediately accepted: in KruTha16 the property is referred to as DIL-restricted regularity by the first letters of the names of the three authors. Another (unnamed) transversality property has appeared in (KruLukTha, , Theorem 4(ii)), also in the finite dimensional setting, and has been used alongside intrinsic transversality (see (KruLukTha, , Theorem 4(iii))) as a dual space sufficient condition for a much better known property called subtransversality. A more general and refined infinite dimensional version of the property from (KruLukTha, , Theorem 4(ii)) has been formulated in KruLukTha2 and proved to imply subtransversality in Asplund spaces. Its thorough analysis is continued in the current article with several new limiting and other characterizations produced, and special attention given to the convex case. It has come as a surprise that, when reduced to finite dimensional Euclidean spaces, this property is equivalent (see Theorem 4.2 below) to intrinsic transversality as defined by Drusvyatskiy et al. DruIofLew15 . Although the definition is different from the one in DruIofLew15 , here and in KruLukTha2 the name ‘intrinsic transversality’ is adopted for this property in both finite and infinite dimensions.

The origins of the concept of regular arrangement of sets in space can be traced back to that of transversality in differential geometry (see, for instance, GuiPol74 ; Hir76 ). Given smooth manifolds $A$ and $B$ in a finite dimensional normed linear space with a point $\bar{x}\in A\cap B$, their transversality can be characterized in dual terms:

[TABLE]

where $N_{A}(\bar{x})$ and $N_{B}(\bar{x})$ are the normal spaces (i.e., orthogonal complements to the tangent spaces) to $A$ and $B$, respectively, at the point $\bar{x}$.

Since the pioneering work by Bauschke and Borwein BauBor93 in 1993, a strong motivation for the development of the transversality theory of collections of sets has been coming from the convergence analysis of alternating (or cyclic) projections for solving feasibility problems. Given two sets $A$ and $B$, the feasibility problem consists in finding a point in their intersection $A\cap B$. This is a very general model which includes, in particular, solving systems of all sorts of equations and inequalities (algebraic, differential, etc.).

Assuming for simplicity that $A$ and $B$ are closed sets in finite dimensions, alternating projections are determined by a sequence $(x_{k})$ alternating between the sets:

[TABLE]

with some initial point $x_{0}$; see Fig. 1.

Here $P_{A}$ and $P_{B}$ stand for the projection operators (see (7)) on the respective sets, corresponding to the Euclidean norm. Equivalently, one can talk about a sequence $(x_{k})$ defined using the composition of projection operators:

[TABLE]

This simple algorithm has a long history. It is often referred to as von Neumann method, although some traces of this method can be found in the 19th century’s publications (see the comments in NolRon16 ).

Up until very recently, the method of alternating projections has been mostly studied in the convex setting. If the sets $A$ and $B$ are convex, the projections are unique, and if $A\cap B\neq\emptyset$, the sequence always converges to a point in $A\cap B$; see Bregman Bre65 and Gurin et al GurPolRai67 , Bauschke and Borwein BauBor93 . However, as one can see from comparing the two illustration in Fig. 1, the type of convergence can be strongly different. Fig. 1(a) represents the case of linear convergence characterized by the inequalities

[TABLE]

where $\hat{x}\in A\cap B$ is the limit of the sequence, $\alpha>0$ and $c\in]0,1[$ is the rate of convergence. In the case represented in Fig. 1(b), the above linear estimates do not hold, and the convergence obviously slows down. It is easy to realize that the type of convergence and its rate are determined by the way the sets intersect. For the linear convergence of alternating projections, the sets must intersect in a certain regular way.

A systematic analysis of the convergence of alternating projections in the convex setting was done by Bauschke and Borwein BauBor93 ; BauBor96 . In particular, they demonstrated (see (BauBor93, , Corollary 3.14)) that alternating projections converge linearly with rate $\sqrt{1-\alpha^{2}}$, provided that the pair $\{A,B\}$ of sets with $A\cap B\neq\emptyset$ is linearly regular with rate $\alpha\in]0,1[$:

[TABLE]

Clearly, this is the case in the example in Fig. 1(a), while the pair of convex sets in Fig. 1(b) is not linearly regular. It has been shown very recently by Luke et al. LukThaTeb that linear regularity of the pair of convex sets with nonempty intersection is not only sufficient for the linear convergence of alternating projections, but is also necessary. This last result together with the theory developed by Bauschke and Borwein in the 1990s make the picture in the convex setting complete and positions the linear regularity property (2) as the core regularity property for a pair of convex sets with nonempty intersection.

The picture becomes much more complicated if the convexity assumption is dropped. First, one can obviously talk only about local convergence and local (near a point in the intersection) regularity/transversality properties. The local version of the linear regularity property (2) — called in this article subtransversality (see Definition 1(i)) — remains a necessary condition for certain types of local linear convergence of alternating projections; cf. LukThaTeb . This property has been thoroughly studied in KruLukTha2 . On the other hand, a simple example in Fig. 2(b) shows that it is not sufficient to guarantee (any) convergence of alternating projections.

The pair of sets in this example satisfies even the global linear regularity condition (2). However, the set $A$ is not convex, and the sequence determined by the alternating projections becomes stationary and is not going to converge to any point in $A\cap B$. At the same time, many important applications naturally lead to feasibility problems for collections of nonconvex sets, and alternating projections often demonstrate reasonably good convergence.

Lewis and Malick LewMal08 and Lewis et al. LewLukMal09 demonstrated recently in the Euclidean space setting that the uniform version of the subtransversality property — called in this article transversality (see Definition 1(ii)) — guarantees local linear convergence of alternating projections for, respectively, a pair of smooth manifolds or a pair of arbitrary closed sets one of which is super-regular at the reference point. Next, Drusvyatskiy et al. DruIofLew15 showed that the super-regularity assumption can be dropped at the expense of reduced (but still linear) convergence rate. The transversality property holds, for instance, in the example in Fig. 1(a). Thanks to LewMal08 ; LewLukMal09 ; DruIofLew15 , the transversality has become a candidate for the position of the core regularity property for a pair of nonconvex sets with nonempty intersection from the point of view of local convergence of alternating projections.

However, the picture in the nonconvex setting is far from being complete. The transversality is not necessary for the local convergence of alternating projections even in the convex case. For example, it always fails when the affine span of the union of the sets is not equal to the whole space, while alternating projections can still converge linearly as is the case when the sets are convex with nonempty intersection of their relative interiors. Another example is given in Fig. 2(a). Comparing this example with the one in Fig. 1(a) illustrates the difference between the transversality and subtransversality properties. In these two examples, this difference does not affect the convergence of alternating projections. The role of the transversality property in the convergence analysis of alternating projections in the nonconvex setting has been further studied in Noll and Rondepierre NolRon16 , and Kruger et al. KruTha16 ; KruLukTha .

A quest has started for the weakest regularity property lying between transversality and subtransversality and still being sufficient for the local linear convergence of alternating projections in the nonconvex setting. We mention here the articles by Bauschke et al. BauLukPhaWan13.1 ; BauLukPhaWan13.2 utilizing restricted normal cones, Drusvyatskiy et al. DruIofLew15 introducing and successfully employing intrinsic transversality, Noll and Rondepierre NolRon16 introducing a concept of separable intersection, with 0-separability being a weaker property than intrinsic transversality and still implying the local linear convergence of alternating projections under the additional assumption that one of the sets is 0-Hölder regular at the reference point with respect to the other. In particular, Drusvyatskiy et al. have shown in (DruIofLew15, , Proposition 3.2 and Theorems 6.1 and 6.2) that, for a pair of closed sets with nonempty intersection, it holds

[TABLE]

and intrinsic transversality ensures local linear convergence of alternating projections. This makes intrinsic transversality the main candidate for the role of the core regularity property from the point of view of local convergence of alternating projections. It is also the main object of interest in the current article.

All the considerations above are for the case when the pair of sets under consideration has nonempty intersection. At the same time, the alternating projections used in the current article for illustrating the transversality theory of collections of sets can be applied in situations when the intersection is empty, and one can still talk about their ‘convergence’ to some collection of points solving an inconsistent feasibility problem. This motivates expanding the transversality theory to the case of inconsistent feasibility. The first attempt of this kind has been made recently by Luke et al. LukThaTam .

Intrinsic transversality is defined in DruIofLew15 in the Euclidean space setting using Fréchet normal cones. Unlike intrinsic transversality, the conventional properties of transversality and subtransversality are defined in the setting of an arbitrary normed linear space in purely primal space terms (see Definition 1). However, in applications it is often more convenient to work with dual space conditions in terms of normal cones. In the case of transversality, its equivalent Fréchet normal cone characterizations in Asplund spaces (or even in general Banach spaces if the sets are convex) are well known. These representations and not the original primal space definitions were mainly used in LewMal08 ; LewLukMal09 when establishing local linear convergence of alternating projections for pairs of nonconvex sets in a finite dimensional space. For subtransversality, no normal cone conditions have been known up until recently. The first condition of this type was announced without proof in the Euclidean space setting in (KruLukTha, , Theorem 4(ii)). A more general and slightly improved Asplund space version of this result has been proved in KruLukTha2 . Unlike the case of transversality, the mentioned normal cone conditions characterizing subtransversality are only sufficient. The subtransversality property, as is well known, lacks stability. This fact makes obtaining general necessary and sufficient normal cone characterizations of this property highly unlikely. The two sets of sufficient normal cone conditions of the subtransversality property, established in KruLukTha2 , are themselves important transversality/regularity properties of pairs of sets in general normed linear spaces (see Definition 2 below) lying between transversality and subtransversality. In a finite dimensional Euclidean space, the strongest of the two properties is equivalent (see Theorem 4.2 below) to intrinsic transversality as defined by Drusvyatskiy et al. DruIofLew15 . Borrowing partially the terminology from DruIofLew15 , the two properties are called here weak intrinsic transversality and intrinsic transversality.

In the current article which continues KruLukTha2 , the two versions of intrinsic transversality are studied in detail mainly in the finite dimensional setting. First, spaces with arbitrary norms are considered, and then the results are further specified for Euclidian spaces.

In Section 2 we recall the definitions of transversality and subtransversality properties of pairs of sets in general normal linear spaces and provide their dual space normal cone necessary and sufficient or just sufficient characterizations in Asplund spaces. If the sets are convex, the characterizations are formulated in general Banach spaces. A slightly simpler version of the Asplund space dual sufficient condition of subtransversality from KruLukTha2 is given. Then the definitions of intrinsic transversality and weak intrinsic transversality from KruLukTha2 are reproduced and their relationships with the conventional subtransversality property are formulated.

Sections 3 and 4 are restricted to the finite dimensional situation. In Section 3 two new limiting objects are introduced for pairs of sets: the cone of pairs of relative limiting normals and the cone of pairs of restricted relative limiting normals. They allow one to formulate simple limiting criteria of intrinsic transversality and in the convex case also subtransversality. The criteria are further simplified if the Euclidian norm is used. In particular, it is shown that, if the sets are convex, the properties of intrinsic transversality, weak intrinsic transversality and subtransversality are equivalent. In Section 4 several more criteria of intrinsic transversality are presented. In particular, it is shown that the normed linear space definition of intrinsic transversality adopted in this article, in the Euclidean space setting reduces to the original definition of this property due to Drusvyatskiy et al. DruIofLew15 . This justifies the name ‘intrinsic transversality’ used throughout this article.

The concluding Section 5 contains a collection of questions related to the content of the article, to which the author does not know the answers.

Notation and preliminaries.

Given a normed linear space $X$, its topological dual is denoted by $X^{*}$, while $\langle\cdot,\cdot\rangle$ denotes the bilinear form defining the pairing between the spaces. $\mathbb{B}$ and $\mathbb{B}^{*}$ stand for the closed unit balls in $X$ and $X^{*}$, respectively, while $\mathbb{B}_{\delta}(x)$ denotes the open ball with centre at $x$ and radius $\delta>0$. Given a set $A$ in a normed linear space, its interior and boundary are denoted by ${\rm int}\,A$ and ${\rm bd}\,A$, respectively, while ${\rm cone}\,A$ denotes the cone generated by $A$: ${\rm cone}\,A:=\{ta\mid a\in A,t\geq 0\}$. $d_{A}(x)$ stands for the distance from a point $x$ to a set $A$. Given an $\alpha\in\mathbb{R}_{\infty}:=\mathbb{R}\cup\{+\infty\}$, $\alpha_{+}$ denotes its positive part: $\alpha_{+}:=\max\{\alpha,0\}$. We regularly use the convention that the supremum of the empty subset of $\mathbb{R}_{+}$ equals 0. Regarding the infimum of the empty subset of $\mathbb{R}_{+}$, we occasionally use different conventions which are always explicitly specified in the text: depending on the context, the infimum can be assumed equal either 1 or 2.

Dual characterizations of transversality and subtransversality properties involve dual space objects – normal cones. For the detailed discussion of the objects introduced below, the readers are referred to the standard references RocWet98 ; Mor06.1 ; Kru03 . The terminology and notation adopted here mostly follow that inKru03 .

Given a subset $A$ of a normed linear space $X$ and a point $\bar{x}\in A$, the Fréchet normal cone to $A$ at $\bar{x}$ is defined as follows:

[TABLE]

It is a nonempty norm closed convex cone, often trivial ($N_{A}(\bar{x})=\{0\}$). Similarly, given a function $f:X\to\mathbb{R}_{\infty}:=\mathbb{R}\cup\{+\infty\}$ and a point $\bar{x}\in{\rm dom}\,f$, the Fréchet subdifferential of $f$ at $\bar{x}$ is defined as

[TABLE]

It is a norm closed convex set, often empty.

If $\dim X<\infty$, the limiting normal cone to $A$ at $\bar{x}$ can be useful:

[TABLE]

If $X$ is a Euclidian space and $A$ is closed, the Fréchet normal cones in definition (5) can be replaced by the proximal ones:

[TABLE]

Here $P_{A}$ is the projection mapping:

[TABLE]

If $A$ is closed and convex, then $P_{A}$ is a singleton. It is easy to verify that $N_{A}^{p}(\bar{x})\subset N_{A}(\bar{x})$, and $\overline{N}_{A}(\bar{x})\neq\{0\}$ if and only if $\bar{x}\in{\rm bd}\,A$. Unlike (3) and (6), the cone (5) can be nonconvex.

If $A$ is a convex set, then all three cones (3), (5) and (6) coincide and reduce to the normal cone in the sense of convex analysis:

[TABLE]

Recall that a Banach space is Asplund if every continuous convex function on an open convex set is Fréchet differentiable on some its dense subset Phe93 , or equivalently, if the dual of each its separable subspace is separable. We refer the reader to Phe93 ; Mor06.1 ; BorZhu05 for discussions about and characterizations of Asplund spaces. All reflexive, in particular, all finite dimensional Banach spaces are Asplund.

2 Transversality, subtransversality and intrinsic transversality

For brevity, in this article we consider the case of two nonempty sets $A$ and $B$. The extension of the definitions and characterizations of the properties to the case of any finite collection of $n$ sets ($n>1$) is straightforward (cf. Kru05 ; Kru06 ; Kru09 ; KruTha13 ; KruTha15 ). The sets are assumed to have a common point $\bar{x}\in A\cap B$. The notation $\{A,B\}$ is used when referring to the pair of two sets $A$ and $B$ as a single object.

Transversality and subtransversality.

We first briefly recall two standard regularity properties of a pair of sets in a normed linear space, namely transversality and subtransversality (also known under other names).

Definition 1

Suppose $X$ is a normed linear space, $A,B\subset X$, and $\bar{x}\in A\cap B$.

1. (i)

$\{A,B\}$ is subtransversal at $\bar{x}$ if there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that

[TABLE] 2. (ii)

$\{A,B\}$ is transversal at $\bar{x}$ if there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that

[TABLE]

The exact upper bound of all $\alpha\in]0,1[$ such that condition (8) or condition (9) is satisfied for some $\delta>0$ is denoted by ${\rm str}[A,B](\bar{x})$ or ${\rm tr}[A,B](\bar{x})$, respectively, with the convention that the supremum of the empty set equals 0.

The requirement that $\alpha<1$ in both parts of Definition 1 imposes no restrictions on the property. It is only needed in the case $\bar{x}\in{\rm int}\,(A\cap B)$ (when conditions (8) and (9) are satisfied for some $\delta>0$ with any $\alpha>0$) to ensure that ${\rm str}[A,B](\bar{x})$ and ${\rm tr}[A,B](\bar{x})$ are always less than or equal to 1 and simplify the subsequent quantitative estimates. It is easy to check that when $\bar{x}\in{\rm bd}\,(A\cap B)$, each of the conditions (8) and (9) implies $\alpha\leq 1$. We are going to use similar requirements in other definitions throughout the article.

The subtransversality (transversality) of $\{A,B\}$ is equivalent to the condition ${\rm str}[A,B](\bar{x})>0$ (${\rm tr}[A,B](\bar{x})>0$), and ${\rm str}[A,B](\bar{x})$ (${\rm tr}[A,B](\bar{x})$) provides a quantitative characterization of this property.

The metric property in part (i) of Definition 1 is a very well known regularity property that has been around for more than 30 years under various names ((local) linear regularity, metric regularity, linear coherence, metric inequality, and subtransversality); cf. BakDeuLi05 ; BauBor93 ; Dol82 ; RocWet98 ; BauBor96 ; Iof89 ; Iof00_ ; Iof16 ; KlaLi99 ; HesLuk13 ; LiNgPon07 ; NgaThe01 ; Pen13 ; ZheNg08 ; ZheWeiYao10 ; DruIofLew15 . It has been used as the key assumption when establishing linear convergence of sequences generated by alternating projection algorithms and a qualification condition for subdifferential and normal cone calculus formulae. If the sets are convex, it is equivalent to the linear regularity property (2).

The property in part (ii) of Definition 1 was referred to in Kru05 ; Kru06 ; Kru09 as strong metric inequality. If $A$ and $B$ are closed convex sets and ${\rm int}\,A\neq\emptyset$, it is equivalent to the conventional qualification condition: ${\rm int}\,A\cap B\neq\emptyset$ (cf. (Kru05, , Proposition 14)).

There are other equivalent primal space definitions for each of the properties in Definition 1; cf. Kru05 ; Kru06 ; Kru09 ; KruTha13 ; KruTha15 .

From comparing the properties in Definitions 1, one can see that the transversality of a pair of sets corresponds to the subtransversality of all their small translations holding uniformly (cf. (DruIofLew15, , p. 1638)). The next inequality is straightforward:

[TABLE]

We refer the reader to KruTha15 ; KruLukTha2 for more examples illustrating the relationship between the properties in Definition 1.

Remark 1

1. The maximum of the distances in Definition 1 and some other representations in the sequel corresponds to the maximum norm in $\mathbb{R}^{2}$ employed in all these definitions and assertions. It can be replaced everywhere by the sum norm (pretty common in this type of definitions in the literature) or any other equivalent norm. All quantitative characterizations of the properties will remain valid (as long as the same norm is used everywhere), although the exact values of ${\rm str}[A,B](\bar{x})$ and ${\rm tr}[A,B](\bar{x})$ do depend on the chosen norm and some estimates can change.

2. In some situations it can be convenient to use the reciprocals $({\rm str}[A,B](\bar{x}))^{-1}$ and $({\rm tr}[A,B](\bar{x}))^{-1}$ instead of ${\rm str}[A,B](\bar{x})$ and ${\rm tr}[A,B](\bar{x})$, respectively, when characterizing the corresponding properties. Instead of checking whether the constant is nonzero when verifying the property, one would need to check wether its reciprocal is finite.  $\triangle$

Transversality properties of pairs of sets are strongly connected with the corresponding regularity properties of set-valued mappings. The properties in parts (i) and (ii) of Definition 1 correspond, respectively, to metric subregularity and metric regularity of set-valued mappings (cf. Kru05 ; Kru06 ; Kru09 ; KruTha15 ; KruLukTha ; KruLukTha2 ; Iof00_ ; Iof16 ), which partially explains the terminology adopted in the current article. These regularity properties of set-valued mappings lie at the core of the contemporary variational analysis. They have their roots in classical analysis and are crucial for the study of stability of solutions to (generalized) equations and various aspects of subdifferential calculus and optimization theory. For the state of the art of the regularity theory of set-valued mappings and its numerous applications we refer the reader to the book by Dontchev and Rockafellar DonRoc14 and the comprehensive survey by Ioffe Iof16 ; Iof16.2 .

Dual characterizations.

The dual criterion for the transversality property in Definition 1(ii) in Asplund spaces is well known; see Kru05 ; Kru06 ; Kru09 ; KruTha13 ; KruTha15 .

Theorem 2.1

Suppose $X$ is Asplund, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Then $\{A,B\}$ is transversal at $\bar{x}$ if and only if there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that $\|x^{*}_{1}+x^{*}_{2}\|>\alpha$ for all $a\in A\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in B\cap\mathbb{B}_{\delta}(\bar{x})$, and all $x_{1}^{*}\in N_{A}(a)$ and $x_{2}^{*}\in N_{B}(b)$ satisfying $\|x^{*}_{1}\|+\|x^{*}_{2}\|=1$. Moreover, the exact upper bound of all such $\alpha$ equals ${\rm tr}[A,B](\bar{x})$.

In finite dimensions, the above criterion admits convenient equivalent reformulations in terms of limiting normals.

Corollary 1

Suppose $\dim X<\infty$, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. The following conditions are equivalent:

1. (i)

$\{A,B\}$* is transversal at $\bar{x}$;* 2. (ii)

there exists a number $\alpha\in]0,1[$ such that ${\|x^{*}_{1}+x^{*}_{2}\|>\alpha}$ for all $x^{*}_{1}\in\overline{N}_{A}(\bar{x})$ and $x^{*}_{2}\in\overline{N}_{B}(\bar{x})$ satisfying $\|x^{*}_{1}\|+\|x^{*}_{2}\|=1$; 3. (iii)

$\overline{N}_{A}(\bar{x})\cap\left(-\overline{N}_{B}(\bar{x})\right)=\{0\}$.

Moreover, the exact upper bound of all $\alpha$ in (ii) equals ${\rm tr}[A,B](\bar{x})$.

The property in part (iii) of Corollary 1 is a well known qualification condition/nonseparabilty property that has been around for about 30 years under various names (basic qualification condition, normal qualification condition, transversality, transversal intersection, regular intersection, linearly regular intersection, and alliedness property); cf. Mor88 ; Mor06.1 ; ClaLedSteWol98 ; Pen13 ; LewMal08 ; LewLukMal09 ; Iof16 . When $A$ and $B$ are smooth manifolds, it coincides with (1).

The next two theorems established recently in KruLukTha2 deal with the subtransversality property in Definition 1(i). They provide, respectively, a dual sufficient condition for this property in Asplund spaces and a necessary and sufficient dual criterion for convex sets in general Banach spaces. Not surprisingly, the second statement is simpler.

Theorem 2.2

Suppose $X$ is Asplund, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Then $\{A,B\}$ is subtransversal at $\bar{x}$ if there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that, for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$ and $x\in\mathbb{B}_{\delta}(\bar{x})$ with $\left\|x-a\right\|=\left\|x-b\right\|$, there exists an $\varepsilon>0$ such that $\|x^{*}_{1}+x^{*}_{2}\|>\alpha$ for all $a^{\prime}\in A\cap\mathbb{B}_{\varepsilon}(a)$, $b^{\prime}\in B\cap\mathbb{B}_{\varepsilon}(b)$, $x_{1}^{\prime}\in\mathbb{B}_{\varepsilon}(a)$, $x_{2}^{\prime}\in\mathbb{B}_{\varepsilon}(b)$, $x^{\prime}\in\mathbb{B}_{\varepsilon}(x)$ with $\left\|x^{\prime}-x_{1}^{\prime}\right\|=\left\|x^{\prime}-x_{2}^{\prime}\right\|$, and $x_{1}^{*},x_{2}^{*}\in X^{*}$ satisfying

[TABLE]

Moreover, ${\rm str}[A,B](\bar{x})\geq\alpha$.

Theorem 2.3

Suppose $X$ is a Banach space, $A,B\subset X$ are closed and convex, and $\bar{x}\in A\cap B$. Then $\{A,B\}$ is subtransversal at $\bar{x}$ if and only if there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that $\|x^{*}_{1}+x^{*}_{2}\|>\alpha$ for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$, $x\in\mathbb{B}_{\delta}(\bar{x})$ with $\left\|x-a\right\|=\left\|x-b\right\|$, and $x_{1}^{*},x_{2}^{*}\in X^{*}$ satisfying

[TABLE]

Moreover, the exact upper bound of all such $\alpha$ equals ${\rm str}[A,B](\bar{x})$.

Below we reformulate Theorem 2.2 in a slightly simpler way (one parameter less).

Theorem 2.4

Suppose $X$ is Asplund, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Then $\{A,B\}$ is subtransversal at $\bar{x}$ if there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that, for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$ and $x\in\mathbb{B}_{\delta}(\bar{x})$ with $\left\|x-a\right\|=\left\|x-b\right\|$, there exists an $\varepsilon>0$ such that $\|x^{*}_{1}+x^{*}_{2}\|>\alpha$ for all $a^{\prime}\in A\cap\mathbb{B}_{\varepsilon}(a)$, $b^{\prime}\in B\cap\mathbb{B}_{\varepsilon}(b)$, $x_{1}^{\prime}\in\mathbb{B}_{\varepsilon}(a)$, $x_{2}^{\prime}\in\mathbb{B}_{\varepsilon}(b)$ with $\left\|x-x_{1}^{\prime}\right\|=\left\|x-x_{2}^{\prime}\right\|$, and $x_{1}^{*},x_{2}^{*}\in X^{*}$ satisfying (11) and

[TABLE]

Moreover, ${\rm str}[A,B](\bar{x})\geq\alpha$.

The conditions in Theorem 2.2 obviously imply those in Theorem 2.4. In fact, the opposite implication is also true, and Theorem 2.4 is a consequence of Theorem 2.2.

Proof (Theorem 2.4 from Theorem 2.2)

Suppose the conditions of Theorem 2.4 are satisfied with some numbers $\alpha\in]0,1[$ and $\delta>0$. Take any $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$ and $x\in\mathbb{B}_{\delta}(\bar{x})$ with $\left\|x-a\right\|=\left\|x-b\right\|$, and choose an $\varepsilon>0$ in accordance with the conditions of Theorem 2.4. Next set $\varepsilon^{\prime}:=\varepsilon/2$ and take any $a^{\prime}\in A\cap\mathbb{B}_{\varepsilon^{\prime}}(a)$, $b^{\prime}\in B\cap\mathbb{B}_{\varepsilon^{\prime}}(b)$, $x_{1}^{\prime}\in\mathbb{B}_{\varepsilon^{\prime}}(a)$, $x_{2}^{\prime}\in\mathbb{B}_{\varepsilon^{\prime}}(b)$, $x^{\prime}\in\mathbb{B}_{\varepsilon^{\prime}}(x)$ with $\left\|x^{\prime}-x_{1}^{\prime}\right\|=\left\|x^{\prime}-x_{2}^{\prime}\right\|$, and $x_{1}^{*},x_{2}^{*}\in X^{*}$ satisfying (10) and (11). Then $x_{1}^{\prime\prime}:=x_{1}^{\prime}+x-x^{\prime}\in\mathbb{B}_{\varepsilon}(a)$, $x_{2}^{\prime\prime}:=x_{2}^{\prime}+x-x^{\prime}\in\mathbb{B}_{\varepsilon}(b)$, $\left\|x-x_{1}^{\prime\prime}\right\|=\left\|x-x_{2}^{\prime\prime}\right\|$, and conditions (13) are satisfied with $x_{1}^{\prime\prime}$ and $x_{2}^{\prime\prime}$ in place of $x_{1}^{\prime}$ and $x_{2}^{\prime}$, respectively. Hence, $\|x^{*}_{1}+x^{*}_{2}\|>\alpha$, i.e., the conditions of Theorem 2.2 are satisfied with the same numbers $\alpha\in]0,1[$ and $\delta>0$, and $\{A,B\}$ is subtransversal at $\bar{x}$ with ${\rm str}[A,B](\bar{x})\geq\alpha$. ∎

Remark 2

1. It is sufficient to check the conditions of Theorems 2.1–2.4 only for $x^{*}_{1}\neq 0$ and $x^{*}_{2}\neq 0$. Indeed, if one of the vectors $x^{*}_{1}$ and $x^{*}_{2}$ equals 0, then by the normalization condition $\|x^{*}_{1}\|+\|x^{*}_{2}\|=1$, the norm of the other one equals 1, and consequently $\|x^{*}_{1}+x^{*}_{2}\|=1$, i.e., such pairs $x^{*}_{1},x^{*}_{2}$ do not impose any restrictions on $\alpha$.

2. Similarly to the classical condition (1), the (sub)transversality characterizations in Theorems 2.1–2.4 require that among all admissible (i.e., satisfying all the conditions of the theorems) pairs of nonzero elements $x^{*}_{1}$ and $x^{*}_{2}$ there is no one with $x^{*}_{1}$ and $x^{*}_{2}$ oppositely directed.

3. The sum $\|x^{*}_{1}\|+\|x^{*}_{2}\|$ in Theorems 2.1–2.4 corresponds to the sum norm on $\mathbb{R}^{2}$, which is dual to the maximum norm on $\mathbb{R}^{2}$ used in Definition 1. It can be replaced by $\max\{\|x^{*}_{1}\|,\|x^{*}_{2}\|\}$ (cf. (Pen13, , (6.11))) or any other norm on $\mathbb{R}^{2}$.  $\triangle$

The proof of Theorems 2.2 and 2.3 given in KruLukTha2 follows the sequence proposed in Kru15 when deducing metric subregularity criteria for set-valued mappings and consists of a series of propositions providing lower primal and dual estimates for the constant ${\rm str}[A,B](\bar{x})$ and, thus, sufficient conditions for the subtransversality of the pair $\{A,B\}$ at $\bar{x}$ which can be of independent interest. In what follows, we will use notations ${\rm itr}_{w}[A,B](\bar{x})$ and ${\rm str}_{c}[A,B](\bar{x})$ for the supremum of all $\alpha$ in Theorems 2.4 and 2.3, respectively, with the convention that the supremum over the empty set equals 0. It is easy to check the following explicit representations of the two constants:

[TABLE]

with the convention that the infimum over the empty set equals 1.

Intrinsic transversality.

The two-limit definition (14) as well as the corresponding dual space sufficient characterization of subtransversality in Theorem 2.4 look complicated and difficult to verify. The following one-limit modification of (14) in terms of Fréchet normals can be useful:

[TABLE]

with the convention that the infimum over the empty set equals 1.

The relationships between the constants ${\rm str}[A,B](\bar{x})$, ${\rm str}_{c}[A,B](\bar{x})$, ${\rm itr}[A,B](\bar{x})$ and ${\rm itr}_{w}[A,B](\bar{x})$ are collected in the next proposition.

Proposition 1

Suppose $X$ is a Banach space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$.

1. (i)

$0\leq{\rm itr}[A,B](\bar{x})\leq{\rm itr}_{w}[A,B](\bar{x})\leq{\rm str}_{c}[A,B](\bar{x})\leq 1$; 2. (ii)

if $X$ is Asplund, then ${\rm str}[A,B](\bar{x})\geq{\rm itr}_{w}[A,B](\bar{x})$; 3. (iii)

if $\dim X<\infty$, then

[TABLE]

with the convention that the infimum over the empty set equals 1; 4. (iv)

if $A$ and $B$ are convex, then ${\rm str}[A,B](\bar{x})={\rm str}_{c}[A,B](\bar{x})$; 5. (v)

if $\dim X<\infty$, and $A$ and $B$ are convex, then ${\rm itr}_{w}[A,B](\bar{x})={\rm str}_{c}[A,B](\bar{x})={\rm str}[A,B](\bar{x})$.

Proof

Part (i) follows immediately from the definitions. Parts (ii) and (iv) are consequences of Theorems 2.4 and 2.3, respectively. Parts (iii) and (v) have been established in KruLukTha2 . ∎

The property introduced in Theorem 2.2 (or equivalently, Theorem 2.4) as a sufficient dual space characterization of subtransversality and corresponding to the condition ${\rm itr}_{w}[A,B](\bar{x})>0$ as well as the stronger property corresponding to the condition ${\rm itr}[A,B](\bar{x})>0$ are themselves important transversality properties of the pair $\{A,B\}$ at $\bar{x}$. Borrowing partially the terminology from DruIofLew15 , these properties are referred to in KruLukTha2 as weak intrinsic transversality and intrinsic transversality, respectively.

Definition 2

Suppose $X$ is a normed linear space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$.

1. (i)

$\{A,B\}$ is weakly intrinsically transversal at $\bar{x}$ if ${\rm itr}_{w}[A,B](\bar{x})>0$, i.e., there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that, for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$ and $x\in\mathbb{B}_{\delta}(\bar{x})$ with $\left\|x-a\right\|=\left\|x-b\right\|$, one has $\|x^{*}_{1}+x^{*}_{2}\|>\alpha$ for some $\varepsilon>0$ and all $a^{\prime}\in A\cap\mathbb{B}_{\varepsilon}(a)$, $b^{\prime}\in B\cap\mathbb{B}_{\varepsilon}(b)$, $x_{1}^{\prime}\in\mathbb{B}_{\varepsilon}(a)$, $x_{2}^{\prime}\in\mathbb{B}_{\varepsilon}(b)$ with $\left\|x-x_{1}^{\prime}\right\|=\left\|x-x_{2}^{\prime}\right\|$, and $x_{1}^{*},x_{2}^{*}\in X^{*}$ satisfying conditions (11) and (13); 2. (ii)

$\{A,B\}$ is intrinsically transversal at $\bar{x}$ if ${\rm itr}[A,B](\bar{x})>0$, i.e., there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that $\|x^{*}_{1}+x^{*}_{2}\|>\alpha$ for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$, $x\in\mathbb{B}_{\delta}(\bar{x})$ with $x\neq a$, $x\neq b$, $1-\delta<\frac{\left\|x-a\right\|}{\left\|x-b\right\|}<1+\delta$, and $x_{1}^{*}\in N_{A}(a)\setminus\{0\}$, $x_{2}^{*}\in N_{B}(b)\setminus\{0\}$ satisfying

[TABLE]

Remark 3

1. The properties introduced in Definition 2 are less restrictive than the dual criterion of transversality in Theorem 2.1.

2. Unlike the transversality and subtransversality properties defined originally by the primal space Definition 1 with the dual space characterizations (not always equivalent!) given by Theorems 2.1–2.4, the intrinsic transversality and weak intrinsic transversality properties are defined in Definition 2 directly in dual space terms and do not have in general equivalent primal space representations.  $\triangle$

In view of Definition 2, Theorem 2.4 says that in Asplund spaces weak intrinsic transversality (and consequently intrinsic transversality) implies subtransversality. Thanks to Proposition 1(i) and (iii), and Remark 3, we have the following relationships between the transversality properties in Asplund spaces.

Corollary 2

Suppose $X$ is Asplund, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Consider the following conditions:

1. (i)

$\{A,B\}$* is transversal at $\bar{x}$;* 2. (ii)

$\{A,B\}$* is intrinsically transversal at $\bar{x}$;* 3. (iii)

$\{A,B\}$* is weakly intrinsically transversal at $\bar{x}$;* 4. (iv)

$\{A,B\}$* is subtransversal at $\bar{x}$.*

Then (i) $\Rightarrow$ (ii) $\Rightarrow$ (iii) $\Rightarrow$ (iv). If $\dim X<\infty$, and $A$ and $B$ are convex, then (iii) $\Leftrightarrow$ (iv).

3 Intrinsic transversality and relative limiting normals

From now on we assume that $\dim X<\infty$.

Intrinsic transversality in finite dimensions.

Definition 2 introduces certain limiting processes (cf. definitions (14), (15) and (16) and representation (17)) and can lead naturally to employing certain limiting normals to the sets under consideration. Observe that not all limiting normals are relevant for characterizing the intrinsic transversality and weak intrinsic transversality properties of a pair of sets. Only those normals to each of the sets can be of interest which take into account the relative location of the other set. It makes sense considering pairs of normals approximately ‘directed’ towards the same point. This observation motivates considering pairs of relative limiting normals.

Definition 3

Suppose $A,B\subset X$ and $\bar{x}\in A\cap B$.

1. (i)

A pair $(x^{*}_{1},x^{*}_{2})\in X^{*}\times X^{*}$ is called a pair of relative limiting normals to $\{A,B\}$ at $\bar{x}$ if there exist sequences $(a_{k})\subset A\setminus B$, $(b_{k})\subset B\setminus A$, $(x_{k})\subset X$ and $(x_{1k}^{*}),(x_{2k}^{*})\subset X^{*}$ such that $x_{k}\neq a_{k}$, $x_{k}\neq b_{k}$ $(k=1,2,\ldots)$, $a_{k}\to\bar{x}$, $b_{k}\to\bar{x}$, $x_{k}\to\bar{x}$, $x_{1k}^{*}\to x_{1}^{*}$, $x_{2k}^{*}\to x_{2}^{*}$, and

[TABLE]

with the convention that $\frac{0}{0}=1$. The collections of all pairs of relative limiting normals to $\{A,B\}$ at $\bar{x}$ will be denoted by $\overline{N}_{A,B}(\bar{x})$. 2. (ii)

A pair $(x^{*}_{1},x^{*}_{2})\in X^{*}\times X^{*}$ is called a pair of restricted relative limiting normals to $\{A,B\}$ at $\bar{x}$ if there exist sequences $(a_{k})\subset A\setminus B$, $(b_{k})\subset B\setminus A$, $(x_{k})\subset X$ and $(x_{1k}^{*}),(x_{2k}^{*})\subset X^{*}$ such that $\left\|x_{k}-a_{k}\right\|=\left\|x_{k}-b_{k}\right\|$ $(k=1,2,\ldots)$, $a_{k}\to\bar{x}$, $b_{k}\to\bar{x}$, $x_{k}\to\bar{x}$, $x_{1k}^{*}\to x_{1}^{*}$, $x_{2k}^{*}\to x_{2}^{*}$, and

[TABLE]

The collections of all pairs of restricted relative limiting normals to $\{A,B\}$ at $\bar{x}$ will be denoted by $\overline{N}{}^{c}_{A,B}(\bar{x})$.

Thus, $\overline{N}_{A,B}(\bar{x})$ and $\overline{N}{}^{c}_{A,B}(\bar{x})$ are formed by limits of certain sequences of pairs of Fréchet normals to each of the sets ‘directed’ approximately towards the same point.

Remark 4

1. In Definition 3, one can always assume that $\left\|x^{*}_{1k}\right\|=\left\|x^{*}_{1}\right\|$, $\left\|x^{*}_{2k}\right\|=\left\|x^{*}_{2}\right\|$, $(k=1,2,\ldots)$. Indeed, if e.g. $x^{*}_{1}=0$, one can take $x^{*}_{1k}:=0$ $(k=1,2,\ldots)$; if $x^{*}_{1}\neq 0$, then, without loss of generality, $x^{*}_{1k}\neq 0$ $(k=1,2,\ldots)$, and one can substitute $x^{*}_{1k}$ with $(x^{*}_{1k})^{\prime}:=\frac{\left\|x^{*}_{1}\right\|}{\left\|x^{*}_{1k}\right\|}{x^{*}_{1k}}$. The same argument applies to $x^{*}_{2}$ and $(x^{*}_{2k})$.

2. Given a subset $A\subset X$, a point $\bar{x}\in A$, and a sequence $(x_{k})\subset X$ converging to $\bar{x}$, it could make sense considering the set $\overline{N}_{A}(\bar{x};(x_{k}))$ of limiting normals to $A$ at $\bar{x}$ relative to $(x_{k})$ defined as the set of vectors $x^{*}\in X^{*}$ such that there exist sequences $(a_{k})\subset A$ and $(x_{k}^{*})\subset X^{*}$ such that $a_{k}\neq x_{k}$ $(k=1,2,\ldots)$, $a_{k}\to\bar{x}$, $x_{k}^{*}\to x^{*}$ as $k\to\infty$ and

[TABLE]

with the convention that $\frac{0}{0}=1$.

This definition is an important ingredient of Definition 3(i) above. If $x^{*}\in\overline{N}_{A}(\bar{x};(x_{k}))$ and $(a_{k})\subset A$ is a sequence corresponding to $x^{*}$ in accordance with this definition, then one has $\left\langle x^{*},x\right\rangle=\left\|x^{*}\right\|$ for any limiting point $x$ of the sequence $\left(\frac{x_{k}-a_{k}}{\left\|x_{k}-a_{k}\right\|}\right)$. Obviously, $\overline{N}_{A}(\bar{x};(x_{k}))$ is a cone in $X^{*}$, and

[TABLE]

Since $\dim X<\infty$, it is easy to check that the cone $\overline{N}_{A}(\bar{x};(x_{k}))$ is closed. $\overline{N}_{A}(\bar{x};(x_{k}))$ can be empty. Indeed, if e.g., $A=\{\bar{x}\}$ and $x_{k}=\bar{x}$ $(k=1,2,\ldots)$, then there is no sequence $(a_{k})\subset A$ with $a_{k}\neq x_{k}$, and consequently $\overline{N}_{A}(\bar{x};(x_{k}))=\emptyset$.  $\triangle$

Proposition 2

Suppose $A,B\subset X$ and $\bar{x}\in A\cap B$.

1. (i)

Each of the sets $\overline{N}_{A,B}(\bar{x})$ and $\overline{N}{}^{c}_{A,B}(\bar{x})$ is a closed cone in $X^{*}\times X^{*}$, possibly empty. Moreover, if the set contains a pair $(x^{*}_{1},x^{*}_{2})$, then, it also contains the pairs $(t_{1}x^{*}_{1},t_{2}x^{*}_{2})$ for all $t_{1}>0$ and $t_{2}>0$. 2. (ii)

$\overline{N}{}^{c}_{A,B}(\bar{x})\subset\overline{N}_{A,B}(\bar{x})\subset\bigcup\limits_{\begin{subarray}{c}(x_{k})\to\bar{x}\end{subarray}}\overline{N}_{A}(\bar{x};(x_{k}))\times\overline{N}_{B}(\bar{x};(x_{k}))\subset\overline{N}_{A}(\bar{x})\times\overline{N}_{B}(\bar{x})$.

Proof

(i) We start with the ‘moreover’ assertion. If a pair $(x^{*}_{1},x^{*}_{2})$ belongs to either of the sets $\overline{N}_{A,B}(\bar{x})$ and $\overline{N}{}^{c}_{A,B}(\bar{x})$, and $(a_{k})\subset A\setminus B$, $(b_{k})\subset B\setminus A$, $(x_{k})\subset X$ and $(x_{1k}^{*}),(x_{2k}^{*})\subset X^{*}$ are the corresponding sequences from Definition 3, then it is straightforward from Definition 3 that, for any $t_{1}>0$ and $t_{2}>0$, the sequences $(a_{k})$, $(b_{k})$, $(x_{k})$, $(t_{1}x_{1k}^{*})$ and $(t_{2}x_{2k}^{*})$ also satisfy all the conditions in the corresponding part of Definition 3. Hence, the pair $(t_{1}x^{*}_{1},t_{2}x^{*}_{2})$ also belongs to the respective set $\overline{N}_{A,B}(\bar{x})$ or $\overline{N}{}^{c}_{A,B}(\bar{x})$. In particular, taking $t_{1}=t_{2}$, we conclude that both the sets are cones.

If a sequence of pairs $(x^{*}_{1i},x^{*}_{2i})$ belongs to either of the sets $\overline{N}_{A,B}(\bar{x})$ and $\overline{N}{}^{c}_{A,B}(\bar{x})$ and the sequences $(x^{*}_{1i})$ and $(x^{*}_{2i})$ converge to $x_{1}^{*}$ and $x_{2}^{*}$, respectively, then, based on the corresponding sequences from Definition 3 and using the standard ‘diagonal’ procedure, one can easily construct new sequences satisfying all the conditions in the corresponding part of Definition 3, and the sequences in $X^{*}$ converge to $x_{1}^{*}$ and $x_{2}^{*}$. Thus, the two cones making each of the sets $\overline{N}_{A,B}(\bar{x})$ and $\overline{N}{}^{c}_{A,B}(\bar{x})$ are closed.

(ii) All the inclusions are direct consequences of Definition 3 and the one in Remark 4.2. The limiting procedure employed in part (ii) of Definition 3 is more restrictive than the one in part (i). This observation implies the first inclusion. If $(x^{*}_{1},x^{*}_{2})\in\overline{N}_{A,B}(\bar{x})$, then $x^{*}_{1}\in\overline{N}_{A}(\bar{x};(x_{k}))$ and $x^{*}_{2}\in\overline{N}_{B}(\bar{x};(x_{k}))$ for some sequence $(x_{k})\subset X$ converging to $\bar{x}$. Hence, the second inclusion. The last inclusion is a consequence of the observation (18). ∎

The next example shows that the last two inclusions in Proposition 2(ii) can be strict.

Example 1

Let $X=\mathbb{R}^{2}$ with the Euclidean norm, $A=\{(t,0)\mid t\geq 0\}$, $B=\{(t,t)\mid t\geq 0\}$ and $\bar{x}=(0,0)$.

Set $a_{k}=b_{k}:=\bar{x}$, $x_{k}:=\left(-\frac{1}{k},-\frac{1}{k}\right)$. We obviously have $x_{k}\to\bar{x}$ as $k\to\infty$, $x^{*}:=(-1,-1)\in N_{A}(\bar{x})\cap N_{B}(\bar{x})$, and $x_{k}-\bar{x}=\frac{1}{k}x^{*}$, i.e., vector $x^{*}$ is parallel to $x_{k}-\bar{x}$. Hence, all the conditions in the definition in Remark 4.2 are satisfied for each of the sets $A$ and $B$, and consequently, $x^{*}\in\overline{N}_{A}(\bar{x};(x_{k}))\cap\overline{N}_{B}(\bar{x};(x_{k}))$. However, $(x^{*},x^{*})\notin\overline{N}_{A,B}(\bar{x})$ because it is not possible to satisfy the conditions in Definition 3(i) with the pair $(x^{*},x^{*})$ and any $a_{k}\neq\bar{x}$ and $b_{k}\neq\bar{x}$.

With $x_{1}^{*}:=(0,1)\in{N}_{A}(\bar{x})=\overline{N}_{A}(\bar{x})$ and $x_{2}^{*}:=(-1,0)\in{N}_{B}(\bar{x})=\overline{N}_{B}(\bar{x})$ it is not possible to find a single sequence $(x_{k})$ converging to $\bar{x}$ to satisfy all the conditions in Definition 3(i). Hence, $(x_{1}^{*},x_{2}^{*})\notin\bigcup\limits_{\begin{subarray}{c}(x_{k})\to\bar{x}\end{subarray}}\overline{N}_{A}(\bar{x};(x_{k}))\times\overline{N}_{B}(\bar{x};(x_{k}))$.  $\triangle$

Thanks to Definition 3, definitions (16) and (15) admit simpler representations:

[TABLE]

with the convention that the minimum over the empty set equals $1$.

Remark 5

1. Formulae (19) and (20) take into account that in finite dimensions the sets under both minima are compact, and the minima of $\|x^{*}_{1}+x^{*}_{2}\|$ over these sets are attained.

2. It is immediate from (19) and (20) that conditions under $\min$ there can be complemented by the inequalities $x^{*}_{1}\neq 0$ and $x^{*}_{2}\neq 0$.  $\triangle$

The following limiting criteria of intrinsic transversality in finite dimensions are straightforward.

Theorem 3.1

Suppose $A,B\subset X$ are closed and $\bar{x}\in A\cap B$. The following conditions are equivalent:

1. (i)

$\{A,B\}$* is intrinsically transversal at $\bar{x}$;* 2. (ii)

there exists a number $\alpha\in]0,1[$ such that ${\|x^{*}_{1}+x^{*}_{2}\|>\alpha}$ for all $(x^{*}_{1},x^{*}_{2})\in\overline{N}_{A,B}(\bar{x})$ satisfying $\|x^{*}_{1}\|+\|x^{*}_{2}\|=1$; 3. (iii)

$\left\{x^{*}\in X^{*}\mid(x^{*},-x^{*})\in\overline{N}_{A,B}(\bar{x})\right\}\subset\{0\}$.

Moreover, the exact upper bound of all $\alpha$ in (ii) equals ${\rm itr}[A,B](\bar{x})$.

Proof

The equivalence of (i) and (ii) as well as the ‘moreover’ estimate are immediate from comparing Definitions 2(ii) and 3(i).

If (i) does not hold, i.e., ${\rm itr}[A,B](\bar{x})=0$, then there exits a pair $(x^{*}_{1},x^{*}_{2})\in\overline{N}_{A,B}(\bar{x})$ such that $\|x^{*}_{1}\|+\|x^{*}_{2}\|=1$ and $\|x^{*}_{1}+x^{*}_{2}\|=0$; this violates (iii). Conversely, if (iii) is violated, i.e., there exists an $x^{*}\neq 0$ such that $(x^{*},-x^{*})\in\overline{N}_{A,B}(\bar{x})$, then the pair $(x^{*}_{1},x^{*}_{2})\in\overline{N}_{A,B}(\bar{x})$ with $x^{*}_{1}=\frac{x^{*}}{2\left\|x^{*}\right\|}$ and $x^{*}_{2}=-\frac{x^{*}}{2\left\|x^{*}\right\|}$ satisfies $\|x^{*}_{1}\|+\|x^{*}_{2}\|=1$ and $\|x^{*}_{1}+x^{*}_{2}\|=0$, which yields ${\rm itr}[A,B](\bar{x})=0$, i.e., (i) does not hold. ∎

Theorem 3.2

Suppose $A,B\subset X$ are closed and convex, and $\bar{x}\in A\cap B$. The following conditions are equivalent:

1. (i)

$\{A,B\}$* is subtransversal at $\bar{x}$;* 2. (ii)

there exists a number $\alpha\in]0,1[$ such that ${\|x^{*}_{1}+x^{*}_{2}\|>\alpha}$ for all $(x^{*}_{1},x^{*}_{2})\in\overline{N}{}^{c}_{A,B}(\bar{x})$ satisfying $\|x^{*}_{1}\|+\|x^{*}_{2}\|=1$; 3. (iii)

$\left\{x^{*}\in X^{*}\mid(x^{*},-x^{*})\in\overline{N}{}^{c}_{A,B}(\bar{x})\right\}\subset\{0\}$.

Moreover, the exact upper bound of all $\alpha$ in (ii) equals ${\rm str}[A,B](\bar{x})={\rm str}_{c}[A,B](\bar{x})$.

Intrinsic transversality in Euclidean spaces.

From now on we assume that $X$ is equipped with the Euclidean norm. We will identify $X^{*}$ with $X$, use $\left\langle\cdot,\cdot\right\rangle$ to denote the scalar product, and write $v_{1}$, $v_{2}$,… instead of $x^{*}_{1}$, $x^{*}_{2}$,…We start with formulating several technical lemmas which are used in the proofs of the results in this section. They are consequences of the geometry of Euclidean space and are likely to be well known. As the author has not been able to find proper references, short proofs are provided for completeness.

Lemma 1

Let $(u_{k})$ and $(v_{k})$ be sequences in a Euclidean space. The following two conditions are equivalent:

1. (i)

$\left\langle u_{k},v_{k}\right\rangle-\left\|u_{k}\right\|\left\|v_{k}\right\|\to 0$; 2. (ii)

$\left\|v_{k}\right\|u_{k}-\left\|u_{k}\right\|v_{k}\to 0$.

Proof

Observe that

[TABLE]

The equivalence of the two conditions follows. ∎

Lemma 2

Let $(u_{k})$ and $(v_{k})$ be sequences of nonzero vectors in a Euclidean space. If both $\left(\frac{u_{k}}{\|u_{k}\|}\right)$ and $\left(\frac{v_{k}}{\|v_{k}\|}\right)$ converge to a (unit) vector $u$, then the sequence $\left(\frac{u_{k}+v_{k}}{\|u_{k}+v_{k}\|}\right)$ also converges to $u$.

Proof

Let

[TABLE]

Then $u_{k}+v_{k}\neq 0$ for all sufficiently large $k$, because otherwise the first condition above yields

[TABLE]

which contradicts the second condition. Thus,

[TABLE]

∎

Lemma 3

Let $u$ and $v$ be nonzero vectors in a Euclidean space. Then

[TABLE]

The idea of the proof below originates in the proof of (KruTha13, , Proposition 5).

Proof

Let $u,v\in X\setminus\{0\}$.

[TABLE]

The proof is completed. ∎

To simplify the comparison of various conditions in the rest of the article, we first reformulate Definition 3 using the Euclidean space notation stipulated above.

Definition 4

Suppose $X$ is a Euclidean space, $A,B\subset X$, and $\bar{x}\in A\cap B$.

1. (i)

A pair $(v_{1},v_{2})\in X\times X$ is called a pair of relative limiting normals to $\{A,B\}$ at $\bar{x}$, i.e. $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$, if and only if there exist sequences $(a_{k})\subset A\setminus B$, $(b_{k})\subset B\setminus A$, $(x_{k}),(v_{1k}),(v_{2k})\subset X$ such that $x_{k}\neq a_{k}$, $x_{k}\neq b_{k}$ $(k=1,2,\ldots)$, $a_{k}\to\bar{x}$, $b_{k}\to\bar{x}$, $x_{k}\to\bar{x}$, $\frac{\left\|x_{k}-a_{k}\right\|}{\left\|x_{k}-b_{k}\right\|}\to 1$, $v_{1k}\to v_{1}$, $v_{2k}\to v_{2}$ as $k\to\infty$, and

[TABLE]

with the convention that $\frac{0}{0}=1$, 2. (ii)

A pair $(v_{1},v_{2})\in X\times X$ is called a pair of restricted relative limiting normals to $\{A,B\}$ at $\bar{x}$, i.e., $(v_{1},v_{2})\in\overline{N}{}^{c}_{A,B}(\bar{x})$, if and only if there exist sequences $(a_{k})\subset A\setminus B$, $(b_{k})\subset B\setminus A$, $(x_{k}),(v_{1k}),(v_{2k})\subset X$ such that $\left\|x_{k}-a_{k}\right\|=\left\|x_{k}-b_{k}\right\|$ $(k=1,2,\ldots)$, $a_{k}\to\bar{x}$, $b_{k}\to\bar{x}$, $x_{k}\to\bar{x}$, $v_{1k}\to v_{1}$, $v_{2k}\to v_{2}$ as $k\to\infty$, and

[TABLE]

In the Euclidean setting, Definition 4(i) admits several equivalent formulations.

Proposition 3

Suppose $X$ is a Euclidean space, $A,B\subset X$, and $\bar{x}\in A\cap B$.

1. (i)

The last two conditions in (21) in Definition 4(i) are equivalent, respectively, to the following two:

[TABLE] 2. (ii)

Conditions (21) in Definition 4(i) can be replaced by conditions (22) and (23).

Proof

(i) The equivalence is a consequence of Lemma 1 employed either with $u_{k}=\frac{x_{k}-a_{k}}{\left\|x_{k}-a_{k}\right\|}$ and $v_{k}=v_{1k}$, or with $u_{k}=\frac{x_{k}-b_{k}}{\left\|x_{k}-b_{k}\right\|}$ and $v_{k}=v_{2k}$.

(ii) Let sequences $(a_{k})$, $(b_{k})$, $(x_{k})$, $(v_{1k})$ and $(v_{2k})$ satisfy all the conditions in Definition 4(i). Set

[TABLE]

Then

[TABLE]

Thanks to Lemma 1, it follows from the last two conditions in (21) that $v_{1k}^{\prime}-v_{1k}\to 0$ and $v_{2k}^{\prime}-v_{2k}\to 0$ as $k\to\infty$; hence, $v_{1k}^{\prime}\to v_{1}$, $v_{2k}^{\prime}\to v_{2}$, and conditions (22) and (23) are satisfied with $v_{1k}^{\prime}$ and $v_{2k}^{\prime}$ in place of $v_{1k}$ and $v_{2k}$, respectively.

Conversely, let sequences $(a_{k})$, $(b_{k})$, $(x_{k})$, $(v_{1k})$ and $(v_{2k})$ satisfy conditions (22) and (23). Then, for any $k$, there exist $v_{1k}^{\prime}\in N_{A}(a_{k})$ and $v_{2k}^{\prime}\in N_{B}(b_{k})$ such that

[TABLE]

and consequently, $v_{1k}^{\prime}\to v_{1}$, $v_{2k}^{\prime}\to v_{2}$, and (with the convention that $\frac{0}{0}=1$)

[TABLE]

hence, the conditions in Definition 4(i) are satisfied with $v_{1k}^{\prime}$ and $v_{2k}^{\prime}$ in place of $v_{1k}$ and $v_{2k}$, respectively. ∎

Remark 6

1. Thanks to Remark 4.1 and Proposition 3(i), the last two conditions in (21) in Definition 4(i) can be replaced, respectively, by conditions

[TABLE]

When $v_{1}\neq 0$ ($v_{2}\neq 0$), one can write

[TABLE]

2. A replacement similar to the one in Proposition 3(ii) is possible for the relative limiting normals discussed in Remark 4.2: $v\in\overline{N}_{A}(\bar{x};(x_{k}))$ if and only if there exist sequences $(a_{k})\subset A$ and $(v_{k})\subset X$ such that $a_{k}\neq x_{k}$ $(k=1,2,\ldots)$, $a_{k}\to\bar{x}$, $v_{k}\to v$ and

[TABLE]

$\triangle$

Next we show that in the Euclidian space setting the cone $\overline{N}_{A,B}(\bar{x})$ of pairs of relative limiting normals can be replaced, when checking intrinsic transversality in accordance with Theorem 3.1, by the cone $\overline{N}{}^{c}_{A,B}(\bar{x})$ of pairs of restricted relative limiting normals.

Proposition 4

Suppose $X$ is a Euclidean space, $A,B\subset X$, $\bar{x}\in A\cap B$ and $v\neq 0$. Then $(v,-v)\in\overline{N}_{A,B}(\bar{x})$ if and only if $(v,-v)\in\overline{N}{}^{c}_{A,B}(\bar{x})$.

Proof

The ‘if’ part follows immediately from the first inclusion in Proposition 2(ii). Conversely, let $(v,-v)\in\overline{N}_{A,B}(\bar{x})$ and $(a_{k})$, $(b_{k})$, $(x_{k})$, $(v_{1k})$ and $(v_{2k})$ be the corresponding sequences as in Definition 3(i) and such that $v_{1k}\to v$ and $v_{2k}\to-v$. Using Lemma 1, it is not difficult to check that

[TABLE]

Passing to subsequences if necessary, we can assume that either $\|x_{k}-a_{k}\|\geq\|x_{k}-b_{k}\|$ or $\|x_{k}-a_{k}\|\leq\|x_{k}-b_{k}\|$ for all $k=1,2,\ldots$. Without loss of generality, it is sufficient to consider the first case only. For any $k$, choose a $t_{k}\in]0,1]$ such that the point $x_{k}^{\prime}:=a_{k}+t_{k}(x_{k}-a_{k})$ satisfies $\|x_{k}^{\prime}-a_{k}\|=\|x_{k}^{\prime}-b_{k}\|$. This is always possible thanks to the continuity of the norm. Then $x_{k}^{\prime}\to\bar{x}$, and we have

[TABLE]

and either $x_{k}^{\prime}=x_{k}$ or

[TABLE]

and consequently, by Lemma 2,

[TABLE]

It follows from (24) and (25) that

[TABLE]

Thus, sequences $(a_{k})$, $(b_{k})$, $(x_{k}^{\prime})$, $(v_{1k})$ and $(v_{2k})$ satisfy all the conditions in Definition 4(ii). Hence, $(v,-v)\in\overline{N}{}^{c}_{A,B}(\bar{x})$. ∎

Since $\overline{N}{}^{c}_{A,B}(\bar{x})\subset\overline{N}_{A,B}(\bar{x})$ (Proposition 2(ii)), the next corollary strengthens Theorem 3.1 (in the Euclidean space setting).

Corollary 3

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. The following conditions are equivalent:

1. (i)

$\{A,B\}$* is intrinsically transversal at $\bar{x}$;* 2. (ii)

there exists a number $\alpha\in]0,1[$ such that ${\|v_{1}+v_{2}\|>\alpha}$ for all $(v_{1},v_{2})\in\overline{N}{}^{c}_{A,B}(\bar{x})$ satisfying $\|v_{1}\|+\|v_{2}\|=1$; 3. (iii)

$\left\{v\in X\mid(v,-v)\in\overline{N}{}^{c}_{A,B}(\bar{x})\right\}\subset\{0\}$.

Moreover, the exact upper bound of all $\alpha$ in (ii) equals ${\rm str}_{c}[A,B](\bar{x})$.

Comparing Corollary 3 with Theorem 3.2 and taking into account Proposition 1(iii), we arrive at the following equivalences of the three transversality properties for closed convex sets in Euclidian spaces.

Corollary 4

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed and convex, and $\bar{x}\in A\cap B$. The following conditions are equivalent:

1. (i)

$\{A,B\}$* is intrinsically transversal at $\bar{x}$;* 2. (ii)

$\{A,B\}$* is weakly intrinsically transversal at $\bar{x}$;* 3. (iii)

$\{A,B\}$* is subtransversal at $\bar{x}$.*

Remark 7

Transversality is in general stronger than all the properties above, even in the convex setting.

It is well known that in Euclidean spaces Fréchet normals and subdifferentials can be approximated by proximal ones; see e.g. (RocWet98, , Exercise 6.18 and Corollary 8.47). As a result, in many statements proximal normals can replace Fréchet ones. This is true, in particular, when characterising intrinsic transversality. The next proposition is a proximal version of Definition 4(i).

Proposition 5

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Then $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ if and only if there exist sequences $(a_{k})\subset A\setminus B$, $(b_{k})\subset B\setminus A$, $(x_{k}),(v_{1k}),(v_{2k})\subset X$ such that $x_{k}\neq a_{k}$, $x_{k}\neq b_{k}$ $(k=1,2,\ldots)$, $a_{k}\to\bar{x}$, $b_{k}\to\bar{x}$, $x_{k}\to\bar{x}$, $\frac{\left\|x_{k}-a_{k}\right\|}{\left\|x_{k}-b_{k}\right\|}\to 1$, $v_{1k}\to v_{1}$, $v_{2k}\to v_{2}$ as $k\to\infty$, and

[TABLE]

with the convention that $\frac{0}{0}=1$.

Proof

Since the proximal normal cone is always a subset of the Fréchet normal cone, the ‘if’ part is trivial.

Conversely, let $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$. If $v_{1}=0$, for any $a_{k}\to\bar{x}$, one can take $v_{1k}=0\in N_{A}^{p}(a_{k})$ $(k=1,2\ldots)$. Let $v_{1}\neq 0$ and sequences $(x_{k})$, $(a_{k})$ and $(v_{1k})$ with $v_{1k}\neq 0$ satisfy the conditions in Definition 3(i) with $v_{1}$ and $v_{1k}$ in place of $x_{1}^{*}$ and $x_{1k}^{*}$, respectively. For each $k=1,2\ldots$, since $v_{1k}\in N_{A}(a_{k})$, there exists a $\delta>0$ such that

[TABLE]

Take a $t_{k}>0$ such that

[TABLE]

set $x_{k}^{\prime}:=a_{k}+t_{k}v_{1k}$, and choose an $a_{k}^{\prime}\in P_{A}(x_{k}^{\prime})$. Then

[TABLE]

We have $a_{k}^{\prime}\in A\setminus B$, $x_{k}^{\prime}\to\bar{x}$, $a_{k}^{\prime}\to\bar{x}$ as $k\to\infty$. It follows from (29), (30) and (27) that

[TABLE]

and consequently,

[TABLE]

Hence,

[TABLE]

Set

[TABLE]

We obviously have $v_{1k}^{\prime}\in N_{A}^{p}(a_{k}^{\prime})\setminus\{0\}$, $\left\|v_{1k}^{\prime}\right\|=\left\|v_{1k}\right\|$,

[TABLE]

[TABLE]

Thus the sequences $(a_{k}^{\prime})$, and $(v_{1k}^{\prime})$ satisfy the conditions in the proposition. Similarly, given a $v_{2}$ and sequences $(x_{k})$, $(b_{k})$ and $(v_{2k})$ satisfying the conditions in Definition 3(i), one can construct sequences $(b_{k}^{\prime})$, and $(v_{2k}^{\prime})$ satisfying the conditions in the proposition. This concludes the proof. ∎

Remark 8

1. In Proposition 5, one can always assume that $\left\|v_{1k}\right\|=\left\|v_{1}\right\|$, $\left\|v_{2k}\right\|=\left\|v_{2}\right\|$, $(k=1,2,\ldots)$; cf. Remark 4.1.

2. For the set $\overline{N}_{A}(\bar{x};(x_{k}))$ of limiting normals to $A$ at $\bar{x}$ relative to $(x_{k})$ defined in Remark 4.2, similarly to Proposition 5, one can show that $v\in\overline{N}_{A}(\bar{x};(x_{k}))$ if and only if there exist sequences $(a_{k})\subset A$ and $(v_{k})\subset X$ such that

[TABLE]

with the convention that $\frac{0}{0}=1$.  $\triangle$

The next statement is a proximal version of Definition 2(ii). It is a consequence of Theorem 3.1 and Proposition 5.

Theorem 3.3

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Then $\{A,B\}$ is intrinsically transversal at $\bar{x}$ if and only if there exist numbers $\alpha\in]0,1[$ and $\rho>0$ such that ${\|v_{1}+v_{2}\|>\alpha}$ for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$, $x\in\mathbb{B}_{\delta}(\bar{x})$ with $x\neq a$, $x\neq b$, $1-\delta<\frac{\left\|x-a\right\|}{\left\|x-b\right\|}<1+\delta$, and all $v_{1}\in N_{A}^{p}(a)$, $v_{2}\in N_{B}^{p}(b)$ satisfying

[TABLE]

with the convention that $\frac{0}{0}=1$. Moreover, the exact upper bound of all such $\alpha$ equals ${\rm itr}[A,B](\bar{x})$.

4 More characterizations of intrinsic transversality in Euclidian spaces

In Euclidian spaces one can go further than restricting the set of relative limiting normals when computing the dual space intrinsic transversality constant (19) to only nonzero ones as observed in Remark 5.2: it is sufficient to consider only unit normals.

Proposition 6

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Then

[TABLE]

with the convention that the minimum over the empty set equals $2$.

Proof

If there is no pair $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ with $v_{1}\neq 0$ and $v_{2}\neq 0$, then both sides in (31) equal 1.

Given any $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ with $\|v_{1}\|=\|v_{2}\|=1$, we have $(\frac{v_{1}}{2},\frac{v_{2}}{2})\in\overline{N}_{A,B}(\bar{x})$ and $\left\|\frac{v_{1}}{2}\right\|+\left\|\frac{v_{2}}{2}\right\|=1$. Hence, by (19),

[TABLE]

and consequently,

[TABLE]

On the other hand, notice that, when evaluating the minimum in (19), it is sufficient to consider only $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ with $v_{1}\neq 0$ and $v_{2}\neq 0$. Given any $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ with $v_{1}\neq 0$, $v_{2}\neq 0$ and $\|v_{1}\|+\|v_{2}\|=1$, we define $w_{1}=\frac{v_{1}}{\left\|v_{1}\right\|}$ and $w_{2}=\frac{v_{2}}{\left\|v_{2}\right\|}$. Then $(w_{1},w_{2})\in\overline{N}_{A,B}(\bar{x})$, $\left\|w_{1}\right\|=\left\|w_{2}\right\|=1$, and by Lemma 3,

[TABLE]

Hence,

[TABLE]

and consequently, by (19),

[TABLE]

The proof is completed. ∎

Alongside ${\rm itr}[A,B](\bar{x})$, several other constants can be used for characterizing intrinsic transversality in Euclidean spaces:

[TABLE]

with the Euclidean distance in $X\times X$ used in (34) and the conventions that in (32) and (33) the maximum and minimum over the empty set equal [math] and $1$, respectively, and the distance to the empty set in (34) equals the distance to the origin, i.e., $\sqrt{2}$.

The expression $\left\langle v_{1},v_{2}\right\rangle$ in (33) can be interpreted as the cosine of the angle between the vectors $v_{1}$ and $v_{2}$. Taking the minimum means minimizing the cosine or, equivalently, maximizing the angle, forcing the vectors to go in opposite directions, potentially making the angle obtuse (or even equal $-\pi$). In this case, ${\rm itr}_{2}[A,B](\bar{x})>0$. However, in general, unlike ${\rm itr}[A,B](\bar{x})$, ${\rm itr}_{1}[A,B](\bar{x})$ and ${\rm itr}_{3}[A,B](\bar{x})$, constant ${\rm itr}_{2}[A,B](\bar{x})$ can be negative.

The relationships between each of the constants (32), (33) and (34) and the original dual space constant ${\rm itr}[A,B](\bar{x})$ (14) (cf. its equivalent representations in (19) and (31)) are given in the next proposition. They follow from the geometry of Euclidean space.

Proposition 7

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Then

[TABLE]

If either $\overline{N}_{A,B}(\bar{x})$ contains a pair $(v_{1},v_{2})\in X\times X$ of nonzero positively independent vectors (none of the vectors is a positive multiple of the other), or $\overline{N}_{A,B}(\bar{x})=\{(0,0)\}$ or $\overline{N}_{A,B}(\bar{x})=\emptyset$, then

[TABLE]

otherwise ${\rm itr}_{3}[A,B](\bar{x})={\rm itr}[A,B](\bar{x})=1$.

Proof

If there is no pair $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ with $v_{1}\neq 0$ and $v_{2}\neq 0$, then, in accordance with the conventions made, ${\rm itr}[A,B](\bar{x})=1$ (see (31)), ${\rm itr}_{1}[A,B](\bar{x})=0$, ${\rm itr}_{2}[A,B](\bar{x})=-1$; so equalities (35) and (36) hold true. If either $\overline{N}_{A,B}(\bar{x})=\{(0,0)\}$ or $\overline{N}_{A,B}(\bar{x})=\emptyset$, then ${\rm itr}_{3}[A,B](\bar{x})=\sqrt{2}$ and equality (37) holds true.

For any $v_{1},v_{2}\in X$, one has

[TABLE]

From (31), (39) and (32), we obtain

[TABLE]

which proves (35). Similarly, from (31), (38) and (33),

[TABLE]

which proves (36).

Definition (34) can be rewritten as follows:

[TABLE]

with the convention that the minimum over the empty set equals $\sqrt{2}$. We next prove equality (37) in the nontrivial case when $\overline{N}_{A,B}(\bar{x})$ contains a pair $(v_{1},v_{2})$ of nonzero vectors with none of them being a positive multiple of the other. Let the minimum in (40) be attained at some $v\in X$ with $\left\|v\right\|=1$ and $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$. Then $v_{1}$ and $-v_{2}$ are the projections of $v$ on the rays $R_{1}$ and $R_{2}$ determined by $v_{1}$ and $-v_{2}$, respectively. In general, one of the rays or both can be trivial. However, in the the nontrivial case, we can restrict ourselves to the pairs $(v_{1},v_{2})$ described above. Thus, $v_{1}\neq v_{2}$, the ray $R_{1}$ and $R_{2}$ are nontrivial and do not go in opposite directions. It also follows from (40) that $v$ must lie in the plane determined by the ray $R_{1}$ and $R_{2}$ in such a way that $\left\langle v,v_{1}\right\rangle\geq 0$ and $\left\langle v,v_{2}\right\rangle\leq 0$. Since $v_{1}$ and $-v_{2}$ are the projections of $v$ on the rays $R_{1}$ and $R_{2}$, we have

[TABLE]

and with $\|v\|=1$ the expression under the $\min$ in (40) takes the following form:

[TABLE]

Since $v$ minimizes this expression over the unit sphere, we have

[TABLE]

Hence, in view of (41),

[TABLE]

It follows now from (42) that

[TABLE]

In view of (41) and (43), we also have

[TABLE]

The last two equalities imply that either $\left\|v_{1}\right\|=\left\|v_{2}\right\|$, or $\left\|v_{1}-v_{2}\right\|=1$ and $\left\langle v_{1},v_{2}\right\rangle=0$, i.e., the rays $R_{1}$ and $R_{2}$ are orthogonal. In the last case, any pair $v_{1}\in R_{1}$, $v_{2}\in R_{2}$, with $\left\|v_{1}-v_{2}\right\|=1$ minimizes expression (44) (the minimum equals 1), and we choose $v_{1}$ and $v_{2}$ such that $\left\|v_{1}\right\|=\left\|v_{2}\right\|=\frac{1}{\sqrt{2}}$. Thus, in both cases $\left\|v_{1}\right\|=\left\|v_{2}\right\|$, and it follows from (44) that

[TABLE]

where $v_{1}^{\prime}:=\frac{v_{1}}{\left\|v_{1}\right\|}$ and $v_{2}^{\prime}:=\frac{v_{2}}{\left\|v_{2}\right\|}$. Obviously, $(v_{1}^{\prime},v_{2}^{\prime})\in\overline{N}_{A,B}(\bar{x})$, $\|v_{1}^{\prime}\|=\|v_{2}^{\prime}\|=1$, and it follows from (40), (45) and (31) that

[TABLE]

Conversely, let the minimum in (31) be attained at some $(v_{1}^{\prime},v_{2}^{\prime})\in\overline{N}_{A,B}(\bar{x})$ with $\|v_{1}^{\prime}\|=\|v_{2}^{\prime}\|=1$. Choose a unit vector $v$ such that $\left\langle v,v_{1}^{\prime}-v_{2}^{\prime}\right\rangle=\|v_{1}^{\prime}-v_{2}^{\prime}\|$, and let $v_{1}$ and $-v_{2}$ be the projections of $v$ on the rays determined by $v_{1}^{\prime}$ and $-v_{2}^{\prime}$, respectively. We are in a situation as above and, using (45) again, we obtain

[TABLE]

Combining (46) and (47) proves (37).

Now we consider the case when one of the components of $\overline{N}_{A,B}(\bar{x})$ is trivial while the other one is not. Let, e.g., $\overline{N}_{A,B}(\bar{x})=C\times\{0\}$, where $C$ is a nontrivial ($C\neq\emptyset$ and $C\neq\{0\}$) cone in $X$. Then by (40), ${\rm itr}_{3}[A,B](\bar{x})\geq 1$, and, given any $\hat{v}\in C$ with $\|\hat{v}\|=1$, one can take $v=\hat{v}$ to get ${\rm itr}_{3}[A,B](\bar{x})\leq\|\hat{v}\|=1$. Hence, ${\rm itr}_{3}[A,B](\bar{x})=1$. In this case ${\rm itr}[A,B](\bar{x})=1$ by convention.

Finally we consider the remaining case when $\overline{N}_{A,B}(\bar{x})=R\times R$ where $R$ is a ray in $X$ determined by a unit vector $\bar{v}$. By definition (34),

[TABLE]

For any $v$, one of the distances in the above expression is attained at the origin and equals 1. Hence, ${\rm itr}_{3}[A,B](\bar{x})\geq 1$. On the other hand, with $v=\hat{v}$ we have ${\rm itr}_{3}[A,B](\bar{x})\leq d(-\hat{v},R)=1$, and consequently, ${\rm itr}_{3}[A,B](\bar{x})=1$. In this case, by (31), ${\rm itr}[A,B](\bar{x})=\frac{1}{2}\left\|\hat{v}+\hat{v}\right\|=1$. ∎

Remark 9

The only property of $\overline{N}_{A,B}(\bar{x})$ used in the proof of Proposition 7 is the one in Proposition 2(i). The proof is applicable in other situations, e.g., when establishing similar relationships between the dual space constants characterizing the transversality property introduced in Definition 1(ii) (cf. KruTha13 ; KruLukTha ). One only needs to replace $\overline{N}_{A,B}(\bar{x})$ in the above proof with $\overline{N}_{A}(\bar{x})\times\overline{N}_{B}(\bar{x})$ where $\overline{N}_{A}(\bar{x})$ and $\overline{N}_{B}(\bar{x})$ are conventional limiting normal cones (cf. definition (5)) at $\bar{x}$ to the sets $A$ and $B$, respectively.  $\triangle$

Thanks to Propositions 6 and 7, the limiting criteria of intrinsic transversality in Theorem 3.1 can be complemented in the Euclidean space setting by several more criteria collected in the next theorem.

Theorem 4.1

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. The following conditions are equivalent:

1. (i)

$\{A,B\}$* is intrinsically transversal at $\bar{x}$;* 2. (ii)

there exists a number $\alpha\in]0,1[$ such that ${\|v_{1}+v_{2}\|>\alpha}$ for all $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ with $\|v_{1}\|={\|v_{2}\|=1}$; the exact upper bound of all such $\alpha$ equals $2\,{\rm itr}[A,B](\bar{x})$; 3. (iii)

${\rm itr}_{1}[A,B](\bar{x})<2$,

i.e., there exists a number $\alpha<2$ such that ${\|v_{1}-v_{2}\|<\alpha}$ for all $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ with $\|v_{1}\|=\|v_{2}\|=1$; the exact lower bound of all such $\alpha$ equals ${\rm itr}_{1}[A,B](\bar{x})$; 4. (iv)

${\rm itr}_{2}[A,B](\bar{x})<1$,

i.e., there exists a number $\alpha<1$ such that $\left\langle v_{1},v_{2}\right\rangle>-\alpha$ for all $(v_{1},v_{2})\in\overline{N}_{A,B}(\bar{x})$ with $\|v_{1}\|=\|v_{2}\|=1$; the exact lower bound of all such $\alpha$ equals ${\rm itr}_{2}[A,B](\bar{x})$; 5. (v)

${\rm itr}_{3}[A,B](\bar{x})>0$,

i.e., there exists a number $\alpha\in]0,1[$ such that $d\left((v,-v),\overline{N}_{A,B}(\bar{x})\right)>\alpha$ for all $v\in X$ with $\|v\|=1$; the exact upper bound of all such $\alpha$ equals ${\rm itr}_{3}[A,B](\bar{x})$.

Remark 10

1. Instead of the constant ${\rm itr}_{2}[A,B](\bar{x})$, defined by (33), one can employ in Theorem 4.1(iv) its modification:

[TABLE]

It is easy to check that ${\rm itr}_{2}^{\prime}[A,B](\bar{x})=({\rm itr}_{2}[A,B](\bar{x}))_{+}$. Hence, the last constant is always nonnegative, and ${\rm itr}_{2}[A,B](\bar{x})<1$ if and only if ${\rm itr}_{2}^{\prime}[A,B](\bar{x})<1$.

2. In view of Definition 3(i) and Proposition 5, the intrinsic transversality constants admit equivalent sequential representations in terms of Fréchet (or proximal) normals to $A$ and $B$ computed at points near $\bar{x}$. For instance,

[TABLE]

with the convention that the infimum over the empty set in (48) and (49) equals 2 and 1, respectively. Each of the criteria of intrinsic transversality in Theorem 4.1 can be rewritten equivalently in terms of Fréchet (or proximal) normals to $A$ and $B$ computed at points near $\bar{x}$.

3. Thanks to Lemma 1, one can write down several more equivalent representations. For instance,

[TABLE]

with the convention that the infimum over the empty set in (50) and (51) equals 2 and 1, respectively.  $\triangle$

Another pair of constants originated in DruIofLew15 can be of interest:

[TABLE]

[TABLE]

with the convention that the infimum and supremum over the empty set equal 1 and 0, respectively. Thanks to this convention, it always holds $0\leq\widehat{\rm itr}_{1}[A,B](\bar{x})\leq 1$ and $0\leq\widehat{\rm itr}_{2}[A,B](\bar{x})\leq 1$.

Remark 11

1. Points $a\in A\setminus B$ and $b\in B\setminus A$ with either $N_{A}(a)=\{0\}$ or $N_{B}(b)=\{0\}$ can be excluded from definition (4) because at such points either $d\left(\frac{b-a}{\|a-b\|},N_{A}(a)\right)=1$ or $d\left(\frac{a-b}{\|a-b\|},N_{B}(b)\right)=1$.

2. Fréchet normal cones in representations (48), (49), (50), (51), (4) and (53) can be replaced by proximal or limiting ones.  $\triangle$

Proposition 8

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. Then

1. (i)

$(\widehat{\rm itr}_{1}[A,B](\bar{x}))^{2}+(\widehat{\rm itr}_{2}[A,B](\bar{x}))^{2}=1$; 2. (ii)

*if ${\rm itr}[A,B](\bar{x})<\frac{1}{\sqrt{2}}$, then $\widehat{\rm itr}_{1}[A,B](\bar{x})\leq 2{\rm itr}[A,B](\bar{x})\sqrt{1-({\rm itr}[A,B](\bar{x}))^{2}}$; * 3. (iii)

$\widehat{\rm itr}_{1}[A,B](\bar{x})=0$* if and only if ${\rm itr}[A,B](\bar{x})=0$.*

Proof

(i) If there are no points $a\in A\setminus B$ and $b\in B\setminus A$ in a neighbourhood of $\bar{x}$ with $N_{A}^{p}(a)\neq\{0\}$ and $N_{B}^{p}(b)\neq\{0\}$, then $\widehat{\rm itr}_{1}[A,B](\bar{x})=1$ and $\widehat{\rm itr}_{2}[A,B](\bar{x})=0$ in view of the conventions made and Remark 11; hence, equality (i) holds.

Let $a\in A\setminus B$ and $b\in B\setminus A$, $v_{1}\in N_{A}^{p}(a)$, $v_{2}\in N_{B}^{p}(b)$, and $\left\|v_{1}\right\|=\left\|v_{2}\right\|=1$. Denote

[TABLE]

and set $\alpha_{1}:=\left\langle v_{1},u\right\rangle$ and $\alpha_{2}:=-\left\langle v_{2},u\right\rangle$. If $\alpha_{1}\geq 0$, then $d(u,\mathbb{R}_{+}v_{1})=\sqrt{1-\alpha_{1}^{2}}$; otherwise $d(u,\mathbb{R}_{+}v_{1})=1$. Hence, in both cases it holds $d^{2}(u,\mathbb{R}_{+}v_{1})+(\alpha_{1})_{+}^{2}=1$. Similarly, $d^{2}(-u,\mathbb{R}_{+}v_{2})+(\alpha_{2})_{+}^{2}=1$. Equality (i) follows from the definitions.

(ii) Let ${\rm itr}[A,B](\bar{x})<\gamma<\frac{1}{\sqrt{2}}$ and choose a $\gamma^{\prime}>0$ and an $\varepsilon>0$ such that ${\rm itr}[A,B](\bar{x})<\gamma^{\prime}<\gamma$ and

[TABLE]

which is possible because the function $\gamma^{\prime}\mapsto\gamma^{\prime}\sqrt{1-(\gamma^{\prime})^{2}}$ is increasing on $[0,\frac{1}{\sqrt{2}}]$. By the second representation in (50), there exist points $a\in(A\setminus B)\cap\mathbb{B}_{\varepsilon}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\varepsilon}(\bar{x})$, $x\in\mathbb{B}_{\varepsilon}(\bar{x})$, $v_{1}\in N_{A}^{p}(a)$, $v_{2}\in N_{B}^{p}(b)$ such that $a\neq x$, $b\neq x$, $1-\varepsilon<\frac{\left\|x-a\right\|}{\left\|x-b\right\|}<1+\varepsilon$, $\left\|v_{1}\right\|=\left\|v_{2}\right\|=1$, $\left\|u_{1}-v_{1}\right\|<\varepsilon$, $\left\|u_{2}-v_{2}\right\|<\varepsilon$, and $\left\|u_{1}+u_{2}\right\|<2\gamma^{\prime}$, where

[TABLE]

Employing the notations (54) and (56), set

[TABLE]

The relationship between the numbers $\alpha$ and $\beta$ is straightforward:

[TABLE]

Let $\mathfrak{A}$, $\mathfrak{A}_{1}$ and $\mathfrak{A}_{2}$ stand for the angles between $u_{1}$ and $u_{2}$, $u$ and $u_{1}$, and $u_{2}$ and $-u$, respectively (measured counterclockwise). Then $\mathfrak{A}+\mathfrak{A}_{1}+\mathfrak{A}_{2}=\pi$, $\sin\mathfrak{A}_{1}\geq 0$, $\sin\mathfrak{A}_{2}\geq 0$, and

[TABLE]

Hence,

[TABLE]

By assumption, $\beta<2\gamma<\sqrt{2}$. It follows from (59) that $\alpha<0$ (i.e. angle $\mathfrak{A}$ is obtuse), and consequently, $\alpha_{1}>0$ and $\alpha_{2}>0$. Since $\alpha_{1}\leq 1$ and $\alpha_{2}\leq 1$ (see (57)), we have $\alpha_{1}\alpha_{2}\leq\hat{\alpha}:=\min\{\alpha_{1},\alpha_{2}\}$, and in view of (60), $-\hat{\alpha}\leq\alpha$. Using (59) again, we have

[TABLE]

In view of (58) and (61) and taking into account that the function $\mu\mapsto\sqrt{\mu(2-\mu)}$ is increasing on $[0,1]$, it holds

[TABLE]

Set $\hat{v}_{1}:=\alpha_{1}v_{1}$, $\hat{v}_{2}:=\alpha_{2}v_{2}$, $\hat{u}_{1}:=\alpha_{1}u_{1}$ and $\hat{u}_{2}:=\alpha_{2}u_{2}$, and notice that $\hat{v}_{1}\in N_{A}^{p}(a)$, $\hat{v}_{2}\in N_{B}^{p}(b)$, $\left\|\hat{u}_{1}-u\right\|=\sqrt{1-\alpha_{1}^{2}}$ and $\left\|\hat{u}_{2}+u\right\|=\sqrt{1-\alpha_{2}^{2}}$. Hence, in view of (62) and (55),

[TABLE]

It follows from the definitions (4) and (54) that $\widehat{\rm itr}_{1}[A,B](\bar{x})\leq 2\gamma\sqrt{1-\gamma^{2}}$. Letting $\gamma\downarrow{\rm itr}[A,B](\bar{x})$, we arrive at the claimed inequality.

(iii) If ${\rm itr}[A,B](\bar{x})=0$, then $\widehat{\rm itr}_{1}[A,B](\bar{x})=0$ in view of part (ii). Let $\widehat{\rm itr}_{1}[A,B](\bar{x})=0$. By the definition (4), for any $\varepsilon>0$, there exist points $a\in(A\setminus B)\cap\mathbb{B}_{\varepsilon}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\varepsilon}(\bar{x})$, $v_{1}\in N_{A}^{p}(a)$ and $v_{2}\in N_{B}^{p}(b)$ such that $\left\|u-v_{1}\right\|<\varepsilon$ and $\left\|u+v_{2}\right\|<\varepsilon$ where $u$ is given by (54). Without loss of generality, we can assume that $\left\|v_{1}\right\|=\left\|v_{2}\right\|=1$. Set $x:=(a+b)/2$. Then $x\neq a$, $x\neq b$, $\left\|x-a\right\|=\left\|x-b\right\|$, and employing the notations (56), $u_{1}=u$ and $u_{2}=-u$. Hence, $\left\|u_{1}-v_{1}\right\|<\varepsilon$ and $\left\|u_{2}-v_{2}\right\|<\varepsilon$, and it follows from the first representation in (50) that ${\rm itr}[A,B](\bar{x})=0$. ∎

Remark 12

For the expression in the right-hand side of the inequality in Proposition 8(ii), we have the following estimates:

[TABLE]

as long as $0\leq{\rm itr}[A,B](\bar{x})\leq 1$. It equals 0 if and only if either ${\rm itr}[A,B](\bar{x})=0$ or ${\rm itr}[A,B](\bar{x})=1$. It equals 1 if and only if ${\rm itr}[A,B](\bar{x})=\frac{1}{\sqrt{2}}$.  $\triangle$

Thanks to Proposition 8, the criteria of intrinsic transversality in Theorems 3.1 and 4.1 can be complemented by several more characterisations collected in the next theorem.

Theorem 4.2

Suppose $X$ is a Euclidean space, $A,B\subset X$ are closed, and $\bar{x}\in A\cap B$. The following conditions are equivalent:

1. (i)

$\{A,B\}$* is intrinsically transversal at $\bar{x}$;* 2. (ii)

$\widehat{\rm itr}_{1}[A,B](\bar{x})>0$,

i.e., there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that

[TABLE]

for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$, or equivalently,

there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that

[TABLE]

for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$, and all $v_{1}\in N_{A}(a)$ and $v_{2}\in N_{B}(b)$;

the exact upper bound of all such $\alpha$ equals $\widehat{\rm itr}_{1}[A,B](\bar{x})$. 3. (iii)

$\widehat{\rm itr}_{2}[A,B](\bar{x})<1$,

i.e., there exist numbers $\alpha\in]0,1[$ and $\delta>0$ such that

[TABLE]

for all $a\in(A\setminus B)\cap\mathbb{B}_{\delta}(\bar{x})$, $b\in(B\setminus A)\cap\mathbb{B}_{\delta}(\bar{x})$, and all $v_{1}\in N_{A}(a)$ and $v_{2}\in N_{B}(b)$ with $\left\|v_{1}\right\|=\left\|v_{2}\right\|=1$;

the exact lower bound of all such $\alpha$ equals $\widehat{\rm itr}_{2}[A,B](\bar{x})$.

Remark 13

1. Conditions $N_{A}(a)\neq\{0\}$ and $N_{B}(b)\neq\{0\}$ can be added in Theorem 4.2(i) (cf. Remark 11).

2. Thanks to Theorem 4.2 and taking into account Remark 11.2, Definition 2(ii) of intrinsic transversality formulated in a general normed linear space, in the Euclidean space setting reduces to the original definition of this property introduced recently by Drusvyatskiy et al. DruIofLew15 (see (DruIofLew15, , Definition 3.1 and formula (3.1)) and (NolRon16, , formula (5))).

3. The six constants providing quantitative characterisations of the intrinsic transversality property of $\{A,B\}$ at $\bar{x}$ make two distinct groups: 1) ${\rm itr}[A,B](\bar{x})$, ${\rm itr}_{1}[A,B](\bar{x})$, ${\rm itr}_{2}[A,B](\bar{x})$, ${\rm itr}_{3}[A,B](\bar{x})$ and 2) $\widehat{\rm itr}_{1}[A,B](\bar{x})$, $\widehat{\rm itr}_{2}[A,B](\bar{x})$. Within each group, the constants can be easily converted from one into another thanks to Proposition 7 and Proposition 8(i). The constants belonging to different groups are not convertible. We only have a one-sided estimate in Proposition 8(ii) complemented by the fact in Proposition 8(iii) that constants ${\rm itr}[A,B](\bar{x})$ and $\widehat{\rm itr}_{1}[A,B](\bar{x})$ can equal zero only simultaneously. Fortunately the last fact is sufficient for detecting intrinsic transversality qualitatively.

4. Compared to $\widehat{\rm itr}_{1}[A,B](\bar{x})$, the definition (16) of ${\rm itr}[A,B](\bar{x})$ contains an additional parameter: $x$ which in a sense determines the “directions” of the normal vectors $x_{1}^{*}$ and $x_{2}^{*}$. This explains why the constants belonging to different groups are not convertible (see the previous remark) and seems to be an advantage of the definition (16) when characterizing the intrinsic transversality property quantitatively as it eliminates normal vectors which are irrelevant from the point of view of intrinsic transversality.  $\triangle$

The next theorem provides a list of equivalent criteria of subtransversality of a pair of convex sets which follow from Theorems 4.1 and 4.2 in view of Corollary 4.

Theorem 4.3

Suppose $A,B\subset X$ are closed and convex, and $\bar{x}\in A\cap B$. Then $\{A,B\}$ is subtransversal at $\bar{x}$ if and only if one of the conditions (ii)–(v) in Theorem 4.1 or (ii)–(iii) in Theorem 4.2 is satisfied.

5 Conclusions and future work

A connection has been established between the two seemingly different normal cone transversality properties of pairs of nonconvex sets: the one introduced in Kruger et al KruLukTha as a sufficient condition of subtransversality in Asplund spaces and the finite dimensional Euclidean space intrinsic transversality property introduced in Drusvyatskiy et al. DruIofLew15 as a sufficient condition for local linear convergence of alternating projections for solving feasibility problems. It is shown that in Euclidean spaces the properties are equivalent. Several characterizations of this property are established. Two new limiting objects are used in the finite dimensional characterizations: the cone of pairs of relative limiting normals and the cone of pairs of restricted relative limiting normals. They possess certain similarity with the conventional limiting normal cones, but unlike the latter one are defined for pairs of sets. Special attention is given to the convex case.

The following questions need to be answered and have been identified for future research. The readers are welcome to contribute.

Does the dual characterization in Theorem 2.4 reduce to that in Theorem 2.3 when the sets are convex? 2. 2)

Can Proposition 1(v) be extended to general Banach spaces? 3. 3)

The relationship between intrinsic transversality and weak intrinsic transversality should be further investigated. Are they different in general? in finite dimensions? in Euclidean spaces? 4. 4)

When do the sets defined in the two parts of Definition 3 coincide? 5. 5)

An analogue of Theorem 3.1 for weak intrinsic transversality should be formulated. 6. 6)

It is not important for estimating intrinsic transversality, but it would be good to add the case ${\rm itr}[A,B](\bar{x})\geq\frac{1}{\sqrt{2}}$ to Proposition 8 for completeness.

Acknowledgements.

The author thanks Nguyen Hieu Thao for many constructive comments and suggestions regarding several definitions and statements in the article, and the referees for the careful reading of the manuscript and constructive comments and suggestions.