Arnold Diffusion in Multi-Dimensional Convex Billiards

TL;DR

This paper demonstrates that in higher-dimensional convex billiards, trajectories can asymptotically approach the boundary, contrasting with the two-dimensional case where Lazutkin proved this is impossible.

Contribution

It extends Lazutkin's result by showing that Arnold diffusion techniques imply such boundary-approaching trajectories exist in three or more dimensions.

Findings

Existence of boundary-approaching trajectories in higher dimensions.

Use of Arnold diffusion to prove genericity of these trajectories.

Contrast with two-dimensional billiard dynamics.

Abstract

Consider billiard dynamics in a strictly convex domain, and consider a trajectory that begins with the velocity vector making a small positive angle with the boundary. Lazutkin proved that in two dimensions, it is impossible for this angle to tend to zero along trajectories. We prove that such trajectories can exist in higher dimensions. Namely, using the geometric techniques of Arnold diffusion, we show that in three or more dimensions, assuming the geodesic flow on the boundary of the domain has a hyperbolic periodic orbit and a transverse homoclinic, the existence of trajectories asymptotically approaching the billiard boundary is a generic phenomenon in the real-analytic topology.

Equations

$$\Gamma =\Gamma (Q)=\{x\in {\mathbb{R}}^d:Q(x)=0\}$$

$$n(x)=-\frac{\nabla Q(x)}{\Vert \nabla Q(x)\Vert }$$

$$C(x)=\Vert \nabla Q(x){\Vert }^{-1}{\left(\frac{{\partial }^{2}Q}{\partial x_i\partial x_j}(x)\right)}_{i,j=1,\dots ,d}$$

$$S(x)u=-Dn(x)u=C(x)u-⟨C(x)u,n(x)⟩n(x)$$

$$\kappa (x,u)=⟨S(x)u,u⟩=⟨C(x)u,u⟩.$$

$$\pi :T\Gamma$$

$$T_x\Gamma \ni u$$

$$M=M(Q)=\{(x,u)\in T\Gamma :\Vert u\Vert \le 1\}.$$

$$v=v(x,u)=u+\sqrt{1-u^2}n(x),$$

$$\tau (x,u)=t>0\text{such that}x+tv(x,u)\in \Gamma .$$

$$\{\begin{array}{l}\bar{x}=x+\tau (x,u)v\\ \bar{u}=v-⟨v,n(\bar{x})⟩n(\bar{x}).\end{array}$$

$$f^{\ast }\lambda -\lambda =-d\tau ,$$

$${\gamma }^{\prime \prime }(t{)}^T={\gamma }^{\prime \prime }(t)-⟨{\gamma }^{\prime \prime }(t),n(\gamma (t))⟩n(\gamma (t)).$$

$$0=\frac{d}{dt}⟨{\gamma }^{\prime }(t),n(\gamma (t))⟩=⟨{\gamma }^{\prime \prime }(t),n(\gamma (t))⟩-\kappa (\gamma (t),{\gamma }^{\prime }(t)).$$

$${\gamma }^{\prime \prime }(t)=\kappa (\gamma (t),{\gamma }^{\prime }(t))n(\gamma (t)).$$

$${\frac{d}{dt}|}_{t=0}{\varphi }^t(x,u)=X(x,u),$$

$$\{\begin{array}{l}\dot{x}=u\\ \dot{u}=\kappa (x,u)n(x).\end{array}$$

$$H(x,u)=\frac{u^2}{2}+\kappa (x,u)\frac{Q(x)}{\Vert \nabla Q(x)\Vert }.$$

$$\Omega=\left(\begin{array}[]{cc}0&I_{d}\\ -I_{d}&0\end{array}\right)$$

$$\underset{n\to \infty }{\mathrm{liminf}}\Vert u^n\Vert =1-\delta ,\underset{n\to \infty }{\mathrm{limsup}}\Vert u^n\Vert =1.$$

$${\frac{d}{dt}|}_{t=0}{\varphi }^t=X.$$

$$T_{A}M=TA\oplus E^s\oplus E^u$$

$$\begin{array}[]{c}\|D\phi^{t}|_{E^{s}}\|\leq C\lambda^{t}\mbox{ for all }t\geq 0,\\ \|D\phi^{t}|_{E^{u}}\|\leq C\lambda^{-t}\mbox{ for all }t\leq 0,\\ \|D\phi^{t}|_{TA}\|\leq C\mu^{|t|}\mbox{ for all }t\in\mathbb{R}.\end{array}$$

$$\begin{array}[]{c}W^{s}(A)=\{x\in M:d(\phi^{t}(x),A)\leq C(x)\tilde{\lambda}^{t}\mbox{ for all }t\geq 0\},\\ W^{u}(A)=\{x\in M:d(\phi^{t}(x),A)\leq C(x)\tilde{\lambda}^{-t}\mbox{ for all }t\leq 0\},\end{array}$$

$$\begin{array}[]{c}W^{s}(x)=\{y\in M:d(\phi^{t}(x),\phi^{t}(y))\leq C(x,y)\tilde{\lambda}^{t}\mbox{ for all }t\geq 0\},\\ W^{u}(x)=\{y\in M:d(\phi^{t}(x),\phi^{t}(y))\leq C(x,y)\tilde{\lambda}^{-t}\mbox{ for all }t\leq 0\}.\end{array}$$

$$W^s(A)=\bigcup _{x\in A}{W}^s(x),W^u(A)=\bigcup _{x\in A}{W}^u(x).$$

$${\varphi }^t(W^s(x))=W^s({\varphi }^t(x)),{\varphi }^t(W^u(x))=W^u({\varphi }^t(x)).$$

$$\begin{array}[]{c}T_{x}W^{s}(x_{+})\oplus T_{x}(W^{s}(A)\cap W^{u}(A))=T_{x}W^{s}(A),\\ T_{x}W^{u}(x_{-})\oplus T_{x}(W^{s}(A)\cap W^{u}(A))=T_{x}W^{u}(A).\end{array}$$

$$s={\pi }^s\circ ({\pi }^u{)}^{-1}:x_{-}⟼x_{+}.$$

$${\pi }_{F(B)}^s=F\circ {\pi }_B^s\circ F^{-1},{\pi }_{F(B)}^u=F\circ {\pi }_B^u\circ F^{-1}.$$

$$\tilde{Q}(x)=\frac{Q(x)}{\Vert \nabla Q(x)\Vert }.$$

Full text

1 Introduction

Let $\Gamma$ be a real-analytic, closed, and strictly convex hypersurface of $\mathbb{R}^{d}$ where $d\geq 3$. The billiard map takes a point $x\in\Gamma$ and an inward pointing velocity vector $v$ of norm $1$ and follows the straight line in the direction of $v$ until the next point of intersection with $\Gamma$. At this point, say $\bar{x}$, the velocity vector transforms according to the optical law of reflection: the angle of reflection $\bar{\alpha}$ equals the angle of incidence $\bar{\beta}$ (see Figure 1).

It is known as a general principle that the dynamics of the billiard map in the limit as the angle of reflection goes to 0 is determined by the geodesic flow on $\Gamma$. We formulate and prove this statement rigorously (see Section 4). Recall the geodesic flow takes a point $x\in\Gamma$ and a tangent vector $u\in T_{x}\Gamma$ and follows the uniquely defined geodesic $x(t)$ with $x(0)=x$ and $\dot{x}(0)=u$. We make the following assumptions on the geodesic flow on $\Gamma$:

1. [A1]

There is a hyperbolic closed geodesic $\gamma$; 2. [A2]

There is a transverse homoclinic geodesic $\xi$ to $\gamma$.

Condition [A2] means that the stable and unstable manifolds of the hyperbolic closed geodesic $\gamma$ intersect transversely, and the geodesic $\xi$ represents an orbit of the geodesic flow lying in their intersection. It is well-known that these conditions are $C^{2}$-open in terms of the Hamiltonian function, and in our case the Hamiltonian depends on the curvature of the hypersurface (i.e. the second derivative). Therefore the set of hypersurfaces satisfying these assumptions is $C^{4}$-open. In particular the set $\mathcal{V}$ of hypersurfaces of $\mathbb{R}^{d}$ for which these assumptions hold is open in the space of all real-analytic, closed, and strictly convex hypersurfaces of $\mathbb{R}^{d}$, equipped with the real-analytic topology (see Section 2 for definitions). Moreover, the fact that $\mathcal{V}$ is nonempty follows from results of [9]. Our main theorem is as follows:

Theorem**.**

For a generic billiard domain in $\mathcal{V}$, there are trajectories of the billiard map such that $\alpha_{n}\to 0$ as $n\to\infty$ where $\{\alpha_{n}\}_{n\in\mathbb{N}}$ denotes the corresponding sequence of angles of reflection.

The word “generic” means that the phenomenon occurs on a residual set of hypersurfaces $\Gamma$ belonging to $\mathcal{V}$ in the real-analytic topology. This topology is very restrictive as bump functions are not real-analytic, and so they cannot be used explicitly when making perturbations. To overcome this difficulty, we use a method introduced by Broer and Tangerman [6]: determine open conditions (in a weaker topology, e.g. $C^{4}$) to be satisfied by a family of perturbations to obtain the desired effect for arbitrarily small values of the parameter; show that these conditions are satisfied by some locally-supported family of perturbations; and approximate the family of perturbed systems sufficiently well by a real-analytic family. Since the conditions are open in a weaker topology, they are satisfied by the real-analytic family. Therefore we obtain the desired effect for arbitrarily small values of the parameter of the real-analytic family. As a consequence of this approach, our theorem is also true in the $C^{k}$ topology for $k=4,5,\ldots,\infty$.

The assumptions [A1,2] are commonly believed to be satisfied in general. In fact, as it is shown in [9] they hold for any $C^{\omega}$-generic convex surface in $\mathbb{R}^{3}$. They are also satisfied for $C^{\omega}$-generic convex hypersurfaces carrying an elliptic closed geodesic in $\mathbb{R}^{d}$ for $d\geq 3$ [9]. The remaining case is when $d>3$ and every closed geodesic is hyperbolic. It is currently unknown whether conditions [A1,2] are $C^{\omega}$-generic in this case, but there have been positive results in a similar setting but with a much weaker topology (for geodesic flows on manifolds with a $C^{2}$-generic metric)[10]. The assumptions [A1,2] have been made before in a similar setting; in the paper [13], it was shown that there are orbits of unbounded energy for certain Hamiltonian systems that are small perturbations of geodesic flows satisfying [A1,2]. Moreover there is a discussion of the plenitude of Riemannian metrics satisfying [A1,2] in section 2 of [13].

The diffusive trajectories described by our main result cannot occur in dimension 2 (assuming sufficient smoothness and strict convexity), as was shown by Lazutkin. Indeed, consider a curve in the billiard domain with the following property: if one segment of a billiard trajectory is tangent to the curve, then every segment of the trajectory is (see Figure 2). Such curves are called caustics, and correspond to closed invariant curves in the phase space of the billiard map. Lazutkin showed the existence of caustics near the billiard boundary in dimension 2 [37]. The corresponding 1-dimensional invariant circles divide the 2-dimensional phase space into invariant regions, keeping the angle of reflection bounded away from zero. See also [17].

Both smoothness and convexity are essential for Lazutkin’s result. Halpern gave an example of a billiard boundary which is strictly convex, but only $C^{2}$-smooth, for which there are billiard trajectories that drift towards the boundary in finite time [31]. In this case we do not have Lazutkin’s caustics, and KAM theory does not apply. It was shown by Mather that if a billiard table in two dimensions is $C^{k}$-smooth for $k\geq 2$ but has a flat point (i.e. a point at which the curvature is 0), then there are billiard trajectories which come arbitrarily close to being tangent with the boundary [40]. As a consequence of this and Birkhoff’s theorem about invariant circles of twist maps being graphs of Lipschitz functions, he showed the absence of caustics in this case.

In dimension 3 and higher, Theorems of Berger and Gruber imply that only ellipsoids have caustics [3, 29]. For every other billiard domain, this result removes an obstacle to the existence of trajectories asymptotically approaching the boundary. Gruber called such trajectories asymptotically terminating on the boundary [28], and proved that the set of such trajectories must have measure 0. The question of existence, however, was left open.

Even though caustics do not exist, invariant tori with a different structure may be present in higher-dimensional billiards (see [36] for a construction of KAM tori for the billiard map in a neighbourhood of an elliptic closed geodesic). In dimension 3, the known examples of invariant tori are of dimension 2 or lower, and so do not divide the 4-dimensional phase space. Then the question of existence of trajectories asymptotically terminating on the boundary becomes the classical question of Arnold diffusion: are there trajectories that move around the invariant tori and drift towards the boundary?

In Arnold’s original paper on the instability of Hamiltonian systems, he constructed an example of a nearly integrable Hamiltonian system with orbits along which a component of the momentum drifts by any prescribed amount, and he conjectured: “I believe that the mechanism of ‘transition chains’ which guarantees that instability in our example is also applicable to the general case”[1]. Arnold’s example is a very special case of a system with a normally hyperbolic invariant cylinder, the stable and unstable manifolds of which have a homoclinic intersection. The dynamics near such an object in the general setting is the subject of the modern theory of Arnold diffusion. Our work belongs to the same realm.

Assumptions [A1,2] imply the existence of a normally hyperbolic invariant manifold $\bar{A}$ for the geodesic flow, the stable and unstable manifolds of which have a transverse homoclinic intersection. We show (see Section 4) that when the initial angle of reflection $\alpha_{1}$ is close to 0, the billiard map can be considered as a small perturbation of a time-shift of the geodesic flow, but with fluctuating speed. We adapt the theory of Fenichel [18, 19, 20] to infer that the billiard map inherits the normally hyperbolic invariant manifold, and that the transverse homoclinic intersection persists. Therefore this is the a priori chaotic setting as defined in [11]. The union of the normally hyperbolic invariant manifold with several homoclinic manifolds is called the homoclinic channel [14]. We prove that, generically, there are trajectories of the billiard map that move up the channel, and that the angle of reflection tends to 0 for these trajectories. These trajectories will mostly stay in a neighbourhood of the hyperbolic closed geodesic $\gamma$, occasionally making excursions near a transverse homoclinic intersection.

Note that the genericity obtained in the main theorem relies on making perturbations within the class $\mathcal{V}$ of strictly convex, analytic hypersurfaces satisfying [A1,2]. However, we do not know of any examples of hypersurfaces in $\mathcal{V}$ that do not contain trajectories tending to the boundary, and indeed it is possible that these trajectories exist for every hypersurface in the class $\mathcal{V}$. One would need to adopt a different, potentially non-perturbative, approach to this problem in order to make such an observation.

A related concept is that of Fermi acceleration [5, 15, 23]. It takes place in the setting of nonautonomous Hamiltonian systems, and is characterised by unbounded growth of energy. In the context of billiards, an example of a nonautonomous system would be the billiard dynamics inside a table with moving walls, for instance. There have been several results in recent years establishing Fermi acceleration in such circumstances [16, 21, 22]. In our case the system is autonomous, and so the billiard map conserves energy. However the energy conserved by the billiard map is, in general, different to that of the geodesic flow, and it is the energy of the geodesic flow that drifts when we consider the billiard map as a perturbation of a time shift of the geodesic flow.

The geometric techniques used here rely on the scattering map of a normally hyperbolic invariant manifold, introduced in [11]. It is known that there are actual orbits (of the billiard map, in this case) that shadow orbits of the scattering map (see e.g. [24, 25]). Therefore one can prove the existence of diffusive trajectories by destroying the invariant curves of scattering maps on an invariant cylinder. This idea was first considered in [42], and developed further in [24, 38, 43]. Variational techniques pioneered by Mather [41] have also had success in proving generic existence of diffusive orbits. These results however typically use a lemma of Cheng and Yan which requires essentially the use of bump functions [7]. This has presented an obstacle to producing results in the real-analytic category. We overcome this difficulty using techniques from [24].

Our results also imply the generic existence of oscillatory motions. In other words, generically there exist trajectories whose corresponding sequence of angles of reflection has positive limit superior, but limit inferior equal to 0 (see Theorem 2). See for example [30, 39, 45] for results regarding the existence of oscillatory motions in the three body problem.

*Acknowledgements.   *The authors would like to thank Misha Bialy, Amadeu Delshams, Marian Gidea, Alexey Glutsyuk, Vadim Kaloshin, Tomás Lázaro, André Neves, Felix Schlenk, and Tere Seara for useful discussions. The first author has received funding for this project from an EPSRC grant, and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant Agreement No 757802). The second author has received funding for this project from Leverhulme Trust RPG-2021-072 and RScF 19-11-00280 and 19-71-10048.

2 Geometry of the Domain, Dynamics, and Results

2.1 The Domain

Let $d\geq 3$, and consider a closed and strictly convex hypersurface $\Gamma$ of $\mathbb{R}^{d}$ given by

[TABLE]

where $Q:\mathbb{R}^{d}\to\mathbb{R}$ is real-analytic. For $x\in\Gamma$, we assume the unit normal vector

[TABLE]

is inward-pointing. Clearly this assumption can be satisfied by changing the sign of $Q$ if necessary. The curvature matrix is the Hessian of $Q$ divided by the norm of the gradient of $Q$:

[TABLE]

Let $u\in T_{x}\Gamma$. The shape operator $S(x):T_{x}\Gamma\to T_{x}\Gamma$, defined using the derivative of the unit normal $n(x)$,

[TABLE]

sends a tangent vector to the tangential component of its image under $C(x)$. This enables a definition of the normal curvature at $x$ in the direction of $u$ via

[TABLE]

Strict convexity of $\Gamma$ means $\kappa$ is strictly positive whenever $u$ is nonzero. Let

[TABLE]

denote the canonical projection along fibres of the tangent bundle. We use the notation $(x,u)\in T\Gamma$ to mean $x\in\Gamma$ and $u\in T_{x}\Gamma$, so that $\pi(x,u)=x$.

2.2 The Billiard Map

The billiard map takes a point on the surface $\Gamma$ and an inward-pointing velocity vector of norm 1, and follows the velocity vector to the next point of collision with the boundary. At this point, the new velocity vector is obtained via the optical law of reflection: the angle of reflection equals the angle of incidence. Billiard dynamics were first considered by Birkhoff [4]. See also e.g. [8][46][47] for modern expositions.

The phase space $M$ of the billiard map consists of the closed unit ball around the origin of each tangent space to $\Gamma$,

[TABLE]

For a point $(x,u)\in M$, the inward-pointing velocity vector is

[TABLE]

where $u^{2}=\|u\|^{2}$. Note that $\|v\|=1$ since $u$ is a tangent vector to $\Gamma$ at $x$ and $n(x)$ is normal to $\Gamma$ at $x$. The boundary $\partial M$ of the phase space is the set of points $(x,u)\in M$ for which $\|u\|=1$, and the interior is $\mathrm{Int}(M)=M\setminus\partial M$. Define the free flight time $\tau:\mathrm{Int}(M)\to\mathbb{R}$ between collisions with the boundary as

[TABLE]

This is well-defined on the interior of $M$ by strict convexity. Extend it continuously to the boundary of $M$ by setting $\tau|_{\partial M}\equiv 0$. It can be shown that $\tau|_{\mathrm{Int}(M)}$ is real-analytic under our assumptions on $\Gamma$, but analyticity of $\tau$ fails on $\partial M$ in these coordinates (see Section 4). Since $\|v\|=1$, the flight time $\tau(x,u)$ is equal to the Euclidean distance from $x$ to the next point of collision with the boundary. Define the billiard map $f:M\to M$ by $(\bar{x},\bar{u})=f(x,u)$ where

[TABLE]

The second equation of (11) is simply the optical law of reflection. Clearly $f|_{\partial M}\equiv\mathrm{Id}$ since the flight time vanishes identically on the boundary. Moreover, a point $(x,u)\in M$ is a fixed point of the billiard map if and only if $\|u\|=1$.

The billiard map satisfies the following reversibility property: if $\mathcal{I}:T\Gamma\to T\Gamma$ denotes the operator $\mathcal{I}(x,u)=(x,-u)$ then $f^{-1}=\mathcal{I}\circ f\circ\mathcal{I}$. It follows that the billiard map is bijective.

Remark*.*

If $\lambda=-u\,dx$ is the Liouville 1-form, then the standard symplectic form on $T\Gamma$ is given by $\omega=d\lambda=dx\wedge du$. Direct differentiation of (11) yields

[TABLE]

and so the billiard map is exact symplectic.

2.3 The Geodesic Flow

The geodesic flow on $\Gamma$ takes a point $(x,u)\in T\Gamma$, and follows the uniquely defined geodesic starting at $x$ in the direction of $u$ at a constant speed of $\|u\|$. Typically, it is introduced via the Hamiltonian function $H(x,u)=\frac{1}{2}g(x)(u,u)$ where $g$ is a Riemannian metric, and $(x,u)$ are intrinsic coordinates on the hypersurface $\Gamma$. In our case we use the induced metric and the coordinates of the ambient Euclidean space $\mathbb{R}^{d}$, so a different formulation is required.

Consider a smooth curve $\gamma:[a,b]\to\Gamma$. The tangential component $\gamma^{\prime\prime}(t)^{T}$ of its acceleration is given by

[TABLE]

Since $\gamma^{\prime}(t)\in T_{\gamma(t)}\Gamma$ and $n(\gamma(t))$ is normal to $\Gamma$ at $\gamma(t)$ for all $t\in[a,b]$ we have

[TABLE]

The curve $\gamma$ is a geodesic if and only if $\gamma^{\prime\prime}(t)^{T}=0$, so from (13) and (14) we see that $\gamma$ is a geodesic if and only if

[TABLE]

It follows that the geodesic flow $\phi^{t}:T\Gamma\to T\Gamma$ is defined by

[TABLE]

where the vector field $X(x,u)=(\dot{x},\dot{u})$ is given by

[TABLE]

Consider the function $H:T\Gamma\to\mathbb{R}$ given by

[TABLE]

It is not hard to see that $X=\Omega\nabla H$ where

[TABLE]

is the standard symplectic matrix. Therefore the geodesic flow is the Hamiltonian flow associated with the Hamiltonian function $H$.

Notice that the second term of (18) vanishes identically on $T\Gamma$, and so the energy $\frac{u^{2}}{2}$ is conserved. Since the Hamiltonian $H$ is homogeneous of second order in $u$, the dynamics of the geodesic flow is the same on every energy level. Despite the second term of the Hamiltonian vanishing on the phase space, its contribution to the second equation of motion in (17) is nontrivial. In particular, if we did not include the second term, the $x$-component would travel along straight lines in $\mathbb{R}^{d}$, and would not remain on the hypersurface $\Gamma$.

As mentioned in Section 1, we assume that there is a hyperbolic periodic orbit $\gamma$ of the geodesic flow, and a transverse homoclinic orbit $\xi$. Assumption [A1] ensures that the geodesic flow has a normally hyperbolic invariant manifold consisting of the hyperbolic orbit $\gamma$ on an interval of energy levels, and assumption [A2] ensures that the stable and unstable manifolds of the normally hyperbolic manifold have a transverse homoclinic intersection. It was shown in [10] that for a $C^{2}$-open and dense set of $C^{\infty}$-smooth Riemannian metrics, assumptions [A1,2] are satisfied. In [35], it was proved that these assumptions hold for a $C^{\infty}$-open and dense set of positively curved Riemannian metrics on $\mathbb{S}^{2}$. There are two significant differences between these cases and ours. First, the analytic category requires special treatment as bump functions cannot be used explicitly. Second, making perturbations only by perturbing the manifold rather than the Riemannian metric is significantly more restrictive, so results about generic metrics do not apply. In the case of closed, strictly convex and real-analytic hypersurfaces of Euclidean space, the paper [9] offers two positive results: for surfaces in $\mathbb{R}^{3}$, conditions [A1,2] are $C^{\omega}$-open and dense; and for hypersurfaces of $\mathbb{R}^{d}$ with an elliptic closed geodesic, [A1,2] are satisfied $C^{\omega}$-generically. We use the notation $\gamma$, $\xi$ liberally to refer either to orbits in $T\Gamma$, or to curves in $\Gamma$.

2.4 Results

Let $\mathcal{V}$ denote the set of all real-analytic functions $Q:\mathbb{R}^{d}\to\mathbb{R}$ such that the set $\Gamma=\Gamma(Q)$ defined as in (1) satisfies:

• •

$\Gamma$ is a closed and strictly convex hypersurface of $\mathbb{R}^{d}$; and

• •

Assumptions [A1,2] hold on $\Gamma$.

We define the real-analytic topology on $\mathcal{V}$ as follows. Let $K\subset\mathbb{R}^{d}$ be a compact set, and $\hat{K}$ a compact complex neighbourhood of $K$. If $Q_{1},Q_{2}\in\mathcal{V}$, by definition they admit holomorphic extensions $\hat{Q}_{1},\hat{Q}_{2}$ on $\hat{K}$. We say that $Q_{1},Q_{2}$ are close on the compact set $K$ in the real-analytic topology if $\hat{Q}_{1},\hat{Q}_{2}$ are uniformly close on $\hat{K}$.

Recall that a set is residual if it is a countable intersection of open dense sets. We now restate our main theorem.

Theorem 1**.**

There is a residual subset $\bar{\mathcal{V}}$ of $\mathcal{V}$ such that for every $Q\in\bar{\mathcal{V}}$, there is $(x,u)\in M=M(Q)$ with $\|u\|<1$ and $\|u^{n}\|\to 1$ as $n\to\infty$, where $(x^{n},u^{n})=f^{n}(x,u)$.

A billiard trajectory $\{(x^{n},u^{n})=f^{n}(x,u)\}_{n\in\mathbb{N}_{0}}$ is said to be oscillatory if there is $\delta>0$ such that

[TABLE]

The methods of this paper also imply the generic existence of oscillatory motions.

Theorem 2**.**

Let $\bar{\mathcal{V}}$ denote the residual subset of $\mathcal{V}$ described in Theorem 1. Then for every $Q\in\bar{\mathcal{V}}$ there is $(x,u)\in M=M(Q)$ for which the trajectory $\{(x^{n},u^{n})=f^{n}(x,u)\}_{n\in\mathbb{N}_{0}}$ is oscillatory.

Note that the trajectories described in Theorem 1 (and obviously those of Theorem 2) cannot occur in a finite number of iterations: since the billiard map is bijective, the preimage of a fixed point (i.e. $\|u\|=1$) is a fixed point. In fact, the process is extremely slow. Since the proof is valid in the analytic category, the Nekhoroshev estimates apply, giving an exponential lower bound on the stability time of finite segments of diffusive trajectories [44]. The methods used to obtain diffusive trajectories in this paper are abstract, and do not provide any upper bounds on diffusion time. However, one factor that may point to a particularly long diffusion time is the use of several secondary homoclinic geodesics to $\gamma$ (whose existence is guaranteed by the existence of $\xi$, due to standard results; see for example Theorem 6.5.5 of [34]).

Let us describe these trajectories. We prove that, under our hypotheses, there is a normally hyperbolic invariant manifold $A$ of the billiard map. The manifold $A$ has the structure of a 2-dimensional cylinder, where the height component can be thought of as the reciprocal of the angle of reflection. The diffusive trajectories occur arbitrarily close to (but not on) this cylinder. Due to normal hyperbolicity, $A$ has stable and unstable invariant manifolds $W^{s,u}(A)$. Moreover, these invariant manifolds have a transverse homoclinic intersection. The union of some small neighbourhoods of the normally hyperbolic cylinder and submanifolds of the transverse homoclinic intersection is called the homoclinic channel. Diffusive trajectories occur in the homoclinic channel, starting near the cylinder, and making excursions near a transverse homoclinic intersection before returning close to $A$ and remaining in a neighbourhood of $A$ for some iterations (see Figure 3). This process is repeated infinitely many times.

In Section 3 we define the following objects: normally hyperbolic invariant manifolds, homoclinic cylinders, and scattering maps. The stable and unstable manifolds of a normally hyperbolic invariant manifold $A$ have invariant foliations by submanifolds. These are called the strong stable and strong unstable foliations. Each point $x$ on the manifold $A$ uniquely defines a leaf $W^{s}(x)$ of the strong stable foliation and a leaf $W^{u}(x)$ of the strong unstable foliation. Suppose now that $y$ is a point of transverse homoclinic intersection of $W^{s,u}(A)$. The implicit function theorem then implies that there is a neighbourhood $B$ of $y$ in $W^{s}(A)\cap W^{u}(A)$ such that the homoclinic intersection is transverse at each point in $B$. Suppose moreover that the intersection at $z$ is strongly transverse (this will be defined precisely later) for each $z\in B$. Let $A^{\prime}$ denote the subset of $A$ consisting of points $x$ for which the leaf $W^{u}(x)$ of the strong unstable foliation intersects $B$ at exactly one point. The scattering map $s_{B}:A^{\prime}\to A$ is defined as follows: from $x\in A^{\prime}$, follow the leaf $W^{u}(x)$ of the strong unstable foliation until it reaches $B$. At this point there is a unique leaf of the strong stable foliation passing through $B$. Follow this leaf back to $A$. The resulting point is $s_{B}(x)$ (see Figure 4). The scattering map was introduced by Delshams, de la Llave, and Seara [11, 12], and is the key tool for analysing dynamics near a transverse homoclinic intersection.

In Section 4, we prove that the billiard map has a normally hyperbolic invariant cylinder $A$, with height component inversely proportional to the angle of reflection, and such that the stable and unstable manifolds of $A$ have a transverse homoclinic intersection. The invariant manifold $A$ is 2-dimensional, and the stable and unstable manifolds are each $d$-dimensional. Each leaf of the strong stable and strong unstable foliations is $(d-2)$-dimensional. We prove the existence of $A$ by using the fact that, as mentioned above, the geodesic flow under assumptions [A1,2] has such objects. Expanding the Taylor series of the billiard map in powers of the flight time $\tau$, we find that the term of order $\tau$ in this expansion is the vector field of the geodesic flow. The problem with taking this approach directly is that the ‘small parameter’ $\tau$ is in fact a function, and this function has unbounded $u$-derivative as $\|u\|\to 1$. This implies that the approximation of the billiard map by a time shift of the geodesic flow is only good in the $C^{0}$ topology, in these coordinates. Therefore we fix some small constant value $\tau_{*}>0$ of the flight time, and make a change of coordinates so that the billiard map is real-analytic in some subset of the phase space. If we expand the billiard map now in powers of $\tau_{*}$, we find that the term of order $\tau_{*}$ is a vector field $Z$, whose flow traces the geodesics of $\Gamma$, but at a fluctuating (nonzero) speed. Moreover $Z$ preserves an integral, and so has a normally hyperbolic invariant cylinder that consists of the orbit defined by the hyperbolic closed geodesic $\gamma$ on an interval of energy levels (i.e. an interval of values of the integral). The angular component of the cylinder is an angular variable on the closed geodesic $\gamma$, and the height corresponds to the energy level. We conclude that the billiard map is $O(\tau_{*}^{2})$ close to the time-$\tau_{*}$ shift of the flow of $Z$. Fenichel theory guarantees [18, 19, 20] the persistence of the normally hyperbolic invariant manifold $A$ for any dynamics $C^{1}$-close enough to $Z$. Since we can make $\tau_{*}$ as small as we like by considering trajectories nearer to the boundary, we get the desired persistent manifold for the billiard map.

Notice that as $\tau\to 0$, the billiard map tends to the identity. Therefore as we move up the cylinder $A$, the hyperbolicity parameters become weaker. To deal with this, we look at a large (i.e. $1/\tau$) iterate of the billiard map, and compare this to the time-1 shift of the geodesic flow. Of course this means that we must exclude the case $\tau=0$, which corresponds to the upper boundary of the normally hyperbolic invariant manifold $A$. As a consequence, the resulting persistent manifold $A$ for the billiard map is a non-compact non-uniformly normally hyperbolic invariant manifold. Aside from this, the non-uniformity of hyperbolicity parameters does not explicitly impact our proof, although it does mean that diffusive trajectories become slower the further up the cylinder they move. Non-compactness, however, means that we must adapt some proofs of results we use from [24] and [25].

We conduct an analysis of the homoclinic cylinders in Section 5. We show in this section that it is sufficient to make perturbations along (most of) a fundamental domain of the billiard map on a homoclinic trajectory. We also give conditions for the existence of diffusive pseudo-orbits, and describe how there are orbits of the billiard map that shadow pseudo-orbits arbitrarily well.

In Section 6 we show how to perturb the billiard map by perturbing the function $Q$ that defines the manifold $\Gamma$. We make a perturbation of size $\epsilon$, and compute the first order expansion of the map resulting from this change of the surface. Since the billiard map is symplectic, the first order term is a Hamiltonian vector field, corresponding to some Hamiltonian $H_{\mathrm{pert}}$. Similarly, the first order term in the expansion of the scattering map in powers of $\epsilon$ is a Hamiltonian vector field. Delshams, de la Llave, and Seara proved a formula (Theorem 31 of [14]) for the Hamiltonian function of this vector field in terms of $H_{\mathrm{pert}}$ and the unperturbed billiard dynamics. We make use of this formula here.

In Section 7, we use the machinery developed in [24] and [25] to determine conditions which, when satisfied, guarantee the existence of diffusive trajectories. We then prove that these conditions are satisfied generically in the real-analytic category. Since we have a homoclinic cylinder for the billiard map, there is a horseshoe near the normally hyperbolic manifold $A$. This in turn implies the existence of infinitely many independent homoclinic cylinders (see e.g. Proposition 2 of [24]). Take 8 such cylinders, and construct an iterated function system $\{f,s_{1},\ldots,s_{8}\}$ on $A$, consisting of the 8 scattering maps and the restriction of the billiard map to $A$. Orbits of the IFS are often called pseudo-orbits, and it can be shown that there are orbits of the map $f$ that shadow pseudo-orbits arbitrarily well [25]. An adaptation of a result from [24] implies that if the IFS has no essential invariant curves on the cylinder $A$, then there exist diffusive pseudo-orbits: orbits of the IFS that drift up the cylinder $A$. We then prove that the absence of invariant essential curves of the IFS is a generic property. This is done by embedding the billiard map in a two-parameter family of real-analytic billiard maps. In Lemma 18, we give open conditions on this family which, if satisfied, imply that the set of parameters for which there are no invariant essential curves is of full measure. The lemma is proved by contradiction, using a combinatorial argument that requires 8 scattering maps. It is based on a result in [24], and finishes the proof of Theorem 1. The idea of using families of real-analytic mappings to approximate a system perturbed by a bump function was used in [6, 9, 24, 27].

3 The Scattering Map of a Normally Hyperbolic Invariant Manifold: Definitions

In this section, let $M$ be a sufficiently smooth manifold, and $\phi^{t}:M\to M$ a sufficiently smooth flow with

[TABLE]

Let $A\subset M$ be a compact invariant manifold (possibly with boundary) for $\phi^{t}$, in the sense that for each $x\in A$, we have $\phi^{t}(x)\in A$ for all $t\in\mathbb{R}$. We say that $A$ is a normally hyperbolic invariant manifold if there is $0<\lambda<\mu^{-1}<1$, a positive constant $C$, and an invariant splitting of the tangent bundle

[TABLE]

such that111This is actually stronger than the standard definition. Notice that the rate of expansion $\lambda$ is equal to the rate of contraction. Such invariant manifolds are called symmetrically normally hyperbolic. Moreover, the standard definition of normally hyperbolic invariant manifolds allows the parameters $\lambda,\mu$ to depend on the point in $A$. However the definition given here is sufficient for our case.:

[TABLE]

Moreover, $A$ is called an $r$-normally hyperbolic invariant manifold whenever $0<\lambda<\mu^{-r}<1$ for $r\geq 1$. This is also referred to as a large spectral gap condition. Taking integer values of $t$ in (23) gives the definition of a normally hyperbolic invariant manifold for the map $F=\phi^{1}$.

Due to the normal hyperbolicity of $A$, there exist stable and unstable manifolds, defined by

[TABLE]

where $C$ is a constant that is allowed to depend on $x$, and $\tilde{\lambda}\in[\lambda,\mu^{-1})$ may be chosen close to $\lambda$. On the stable and unstable manifolds we have, respectively, the strong stable and strong unstable foliations. The leaves of the foliations corresponding to $x\in A$ are

[TABLE]

The tangent spaces of the leaves of the foliations are given by $T_{x}W^{s}(x)=E^{s}_{x}$ and $T_{x}W^{u}(x)=E^{u}_{x}$. Two leaves of the strong stable or strong unstable foliation corresponding to different points of $A$ are disjoint, and

[TABLE]

The foliations are invariant in the sense that for any $x\in A$ and $t\in\mathbb{R}$ we have

[TABLE]

Since we assume $A$ to be sufficiently smooth (i.e. more than $r$ times continuously differentiable), the stable and unstable manifolds of $A$ are $C^{r}$-smooth, where $r$ is the size of the spectral gap. So too are the leaves of the strong stable and strong unstable foliations [32]. Moreover, the foliations are $C^{r-1}$-regular, meaning that the fields of tangents to the leaves are $C^{r-1}$-smooth.

It was proved by Fenichel in the 70’s that open normally hyperbolic invariant manifolds are stable under small perturbations: for any vector field $Y$ in some $C^{1}$-small neighbourhood of $X$, there is a persistent $C^{1}$-smooth open normally hyperbolic invariant manifold, $C^{1}$-close to $A$ [18, 19, 20]. Moreover, if $X$ is $C^{r}$-smooth, $A$ is an $r$-normally hyperbolic invariant manifold, and $Y$ is $C^{r}$-close to $X$, then the persistent manifold is $C^{r}$-smooth and is $C^{r}$-close to $A$. Moreover, the stable and unstable manifolds of the perturbed manifold are $C^{r}$-close to those of $A$. In the case of open normally hyperbolic manifolds, invariance means that the vector field is tangent to the manifold at each point.

Similar results hold for maps: if $F$ is a $C^{r}$ smooth map of a smooth manifold $M$ with an open $r$-normally hyperbolic invariant manifold $A$, then there is a $C^{r}$ neighbourhood of $F$ such that for any map $G$ from this neighbourhood, the invariant manifold $A$ persists as an open $C^{r}$-smooth normally hyperbolic invariant manifold [32].

Now suppose there is a homoclinic point $x\in(W^{s}(A)\cap W^{u}(A))\setminus A$. We call $x$ a transverse homoclinic point if $W^{s}(A)$ and $W^{u}(A)$ are transverse at $x$, in which case we write $x\in(W^{s}(A)\pitchfork W^{u}(A))\setminus A$. If $x$ is a transverse homoclinic point, the implicit function theorem implies that there is an open neighbourhood $V$ of $x$ contained in $W^{s}(A)\cap W^{u}(A)$ such that every $y\in V$ is a transverse homoclinic point.

Given a transverse homoclinic point $x\in(W^{s}(A)\pitchfork W^{u}(A))\setminus A$, there are unique $x_{\pm}\in A$ such that $x\in W^{s}(x_{+})\cap W^{u}(x_{-})$. The homoclinic intersection at $x$ is said to be strongly transverse if

[TABLE]

In this case we define the holonomy maps $\pi^{s,u}:V\to A$ to be projections along the leaves of the strong stable and strong unstable foliations, so $\pi^{s}(x)=x_{+}$ and $\pi^{u}(x)=x_{-}$. Notice that the holonomy maps are locally invertible whenever the strong transversality condition holds. Moreover they are $C^{r-1}$-smooth by the regularity of the foliations. Therefore, taking $V$ to be small enough so that (28) holds at each point, the holonomy maps are $C^{r}$-diffeomorphisms onto their respective images. Therefore we can define the scattering map from $\pi^{u}(V)\to\pi^{s}(V)$:

[TABLE]

Now, consider the case of a map $F=\phi^{1}$. Assume that $A$ has the structure of a cylinder, so there is a diffeomorphism $h:\mathbb{T}\times[0,1]\to A$. Let $B\subset(W^{s}(A)\pitchfork W^{u}(A))\setminus A$ be a smooth connected two-dimensional submanifold.

• Definition

We say that $B$ is a homoclinic cylinder if the scattering map $s:A\longrightarrow A$ relative to $B$ is a diffeomorphism.

Suppose the map $F$ is exact symplectic. Recall that a number is said to be Diophantine if it is sufficiently poorly approximable by rationals. Suppose the boundary curves $\gamma^{\pm}$ of the cylinder $A$ are KAM curves, meaning the restriction of the time-1 shift of the flow to $\gamma^{\pm}$ is smoothly conjugate to a rotation by a Diophantine angle. KAM theory implies that they survive all exact symplectic $C^{4}$-small perturbations (see e.g. Appendix 8 of [2]) of an exact symplectic map. We can extend the normally hyperbolic invariant manifold $A$ to an open normally hyperbolic invariant manifold $A^{\prime}$. Due to Fenichel theory, the open cylinder $A^{\prime}$ persists under small perturbations, and due to KAM theory the boundary curves of $A$ persist as invariant curves of the exact symplectic map on the perturbed open cylinder. If $A$ is a 2-dimensional manifold, this implies the persistence of the cylinder $A$ under small exact symplectic perturbations. Moreover the leaves of the strong stable and strong unstable foliations have smooth dependence on the corresponding point in $A$, and so the perturbed foliations will be close to those of the unperturbed system. Therefore the perturbed stable and unstable manifolds themselves will be close to the unperturbed invariant manifolds. This reasoning also implies that a tranverse homoclinic intersection of stable and unstable manifolds remains transverse under small perturbations [32].

It is clear that if $B$ is a homoclinic cylinder, then so too is $F^{n}(B)$ for any $n\in\mathbb{Z}$. The following identities are an immediate consequence of the invariance (27) of the strong stable and strong unstable foliations:

[TABLE]

We also have analogous statements for time shifts of the flow $\phi^{t}$. Similar expressions hold for $F^{n}(B)$ for any $n\in\mathbb{Z}$. Furthermore, combining (29) and (30) gives an analogous expression for the scattering map $s_{F^{n}(B)}$.

4 Billiard Trajectories Near the Boundary

The goal of this section is to show that the billiard map inherits a normally hyperbolic invariant manifold from the geodesic flow, and to analyse the homoclinic intersection of its stable and unstable manifolds. This is done by showing that, in some coordinates and for sufficiently small $\tau_{*}>0$, the billiard map $f$ is approximated up to order $\tau_{*}^{2}$ by the time-$\tau_{*}$ shift of the flow of some vector field $Z$ in the $C^{r}$-topology for any finite $r$. We show that $Z$ has a normally hyperbolic invariant manifold that does not depend on $\tau_{*}$ as a result of assumptions [A1,2]. Therefore, shrinking $\tau_{*}$ if necessary, we can use the theory of Fenichel to imply the persistence of the normally hyperbolic invariant manifold for $f$ [18, 19, 20]. This process is repeated a countable number of times, and the resulting invariant manifolds are glued together to obtain a noncompact normally hyperbolic invariant manifold for $f$.

Let $Q\in\mathcal{V}$ and let $\Gamma=\Gamma(Q)$, $M=M(Q)$ be as in (1), (8) respectively. In this section we do not make any perturbations to the surface, so we may assume without loss of generality that $\|\nabla Q(x)\|=1$ for all $x\in\Gamma$. Indeed, we can replace $Q$ by

[TABLE]

Then $\tilde{Q}(x)=0$ if and only if $Q(x)=0$, so $\Gamma(\tilde{Q})=\Gamma$. Moreover if $x\in\Gamma$ then

[TABLE]

since $Q$ vanishes identically on $\Gamma$, and so the gradient of $\tilde{Q}$ has norm 1 on $\Gamma$. This assumption simplifies computations since the unit normal takes the form

[TABLE]

which implies that the shape operator is

[TABLE]

4.1 Estimates Near the Boundary

We consider trajectories near the boundary by taking $\delta>0$ small and restricting our attention to the subset

[TABLE]

of the billiard map’s phase space.

Lemma 3**.**

Let $(x,u)\in M_{\delta}$. Write $\tau=\tau(x,u)$, and recall $v=u+\sqrt{1-u^{2}}\,n(x)$.

1. (i)

We have

[TABLE]

where

[TABLE] 2. (ii)

Write $(\bar{x},\bar{u})=f(x,u)$. Then

[TABLE]

where $X$ is the vector field of the geodesic flow, given by (17). 3. (iii)

With $n(x)$ as in (33),

[TABLE]

where $r$ is the vector-valued function with components

[TABLE] 4. (iv)

With $R(x,v)$ as in (37),

[TABLE]

Proof.

These statements are proved by taking Taylor expansions in powers of $\tau$. Notice that

[TABLE]

From (33), (34), (42), and the fact that $Q(x)=Q(\bar{x})=0$ we get

[TABLE]

Dividing by $\tau$ and rearranging terms gives (i), where the terms of higher order are uniformly bounded due to the real-analyticity of $Q$.

For (ii) we must compute the term of order $\tau$ in the expansion of $(\bar{x},\bar{u})$. From (36) we get

[TABLE]

Also by (42),

[TABLE]

and moreover

[TABLE]

where $D$ denotes the derivative, and where (36) was used. It follows that

[TABLE]

Comparing (48) and (53) with (17) completes the proof of (ii).

From (33) we get

[TABLE]

which is (iii).

For part (iv) notice that the second equation of (11) combined with basic geometry implies that

[TABLE]

which, along with (42), gives

[TABLE]

Comparing (37) and (40) we see that

[TABLE]

Using this together with (39),

[TABLE]

From (36), (42), (57), and (60) we get

[TABLE]

which completes the proof of the lemma. ∎

Remark*.*

Equation (38) implies that the billiard map is approximated up to order $\tau^{2}$ by the time-$\tau$ shift of the geodesic flow. Equation (36), however, implies that the $u$-derivative of $\tau$ is unbounded as $\|u\|$ tends to 1. Therefore the approximation of the billiard map by a time shift of the geodesic flow is only good in the $C^{0}$ topology, which is not enough to use the theory of Fenichel. However, if we scale up $\tau$ by dividing by a small constant $\tau_{*}>0$ and consider it as a variable $z$ rather than a function, normalise $u$ to have norm 1 (i.e. let $w=\|u\|^{-1}u$), and consider the billiard map as a function of $x,w$, and $z$, then it is real-analytic and near to the identity by (36) and (41), and estimates similar to those contained in Lemma 3 hold in the $C^{r}$-topology for any finite $r$ with respect to the new constant small parameter $\tau_{*}$. This process is carried out in the next section - see the change of coordinates (67).

4.2 Normally Hyperbolic Invariant Manifold for the Billiard Map

Fix some small $\tau_{*}>0$ and consider the coordinates $(x,w,z)$ on the subset $M_{\delta}$ (see equation (35)) of the phase space $M$ where

[TABLE]

Here, we are replacing the radial variable $\|u\|$ by the scaled variable $z$. Fix any $b>a>0$ and let

[TABLE]

Note that $\mathcal{D}_{*}\subset M$. Consider the vector field $Z(x,w,z)=(\dot{x},\dot{w},\dot{z})$ where

[TABLE]

where $R$ is given by (37). Comparing $Z$ with $X$ (see (17)), it is clear that the orbits of $Z$ trace the geodesics of $\Gamma$, but at a fluctuating speed $z$. The following lemma is the general principle mentioned in the introduction: the dynamics of the billiard map in the limit as the angle of reflection goes to 0 is determined by the geodesic flow.

Lemma 4**.**

On $\mathcal{D}_{*}$ we have

[TABLE]

in the $C^{r}$-topology for any finite $r$.

Proof.

The second equation of (67) implies that

[TABLE]

This together with the first equation of (67) gives

[TABLE]

From (36) and (72) we get

[TABLE]

and so

[TABLE]

Write $(\bar{x},\bar{w},\bar{z})=f(x,w,z)$. From (38) and (72) we get

[TABLE]

and

[TABLE]

Let us determine $\bar{z}$. Equations (34), (75), and (77) imply

[TABLE]

since $S(x)w$ is tangent to $\Gamma$ and $n(x)$ is normal. Therefore

[TABLE]

Moreover (36), (41), (72), and (74) imply

[TABLE]

and

[TABLE]

Combining the last three equations with the second equation of (67),

[TABLE]

It follows that

[TABLE]

where the vector field $Z(x,w,z)$ is as in (69). ∎

Recall that $\gamma:\mathbb{T}\to\Gamma$ is a hyperbolic closed geodesic of length 1, and $\xi:\mathbb{R}\to\Gamma$ is a transverse homoclinic geodesic, both parametrised by arclength.

Lemma 5**.**

The vector field $Z$ has an integral $y$ defined by

[TABLE]

and the set

[TABLE]

is an $r$-normally hyperbolic invariant manifold for the flow of $Z$ for any finite $r$.

Proof.

Notice that, from (69), and by a similar computation to (82),

[TABLE]

It follows that

[TABLE]

which implies that $z=C\kappa(x,w)^{-\frac{4}{3}}$ for some positive constant $C$. Therefore the quantity $y$ defined by (93) is constant along trajectories. Since $z=y^{-1}\kappa(x,w)^{-\frac{4}{3}}$ it is bounded along trajectories, and bounded away from 0 for strictly positive $y$. It follows that the vector field $Z$ follows the geodesics on $\Gamma$ at a fluctuating speed that is bounded away from 0. Therefore $\bar{A}_{*}$ as defined in (94) is a normally hyperbolic invariant manifold for $Z$.

Notice that every point $(q,y)\in\bar{A}_{*}$ is periodic with minimal period $l=l(y)$ depending only on $y$. We have

[TABLE]

where

[TABLE]

is a positive constant depending only on the periodic orbit $\gamma$. Then we can introduce coordinates $(\theta,y)$ so that the cylinder is written as

[TABLE]

where $\phi^{t}_{Z}$ denotes the time-$t$ shift of the flow of $Z$. In this equation the time $t$ along the orbit is equal to $\theta l(y)$, and so we get $\dot{\theta}=Ky$. Therefore on $\bar{A}_{*}$ in $(\theta,y)$ coordinates we have

[TABLE]

and

[TABLE]

It follows that the differential of the time-$t$ shift of the flow of $Z$ is

[TABLE]

which has a double eigenvalue of 1. Therefore there is an infinitely large spectral gap in these coordinates, and so $\bar{A}_{*}$ is an $r$-normally hyperbolic invariant manifold. ∎

Remark*.*

By choosing $b>a>0$ appropriately in equation (68) we can guarantee that $\bar{A}_{*}\subset\mathcal{D}_{*}$. We can extend the normally hyperbolic invariant manifold $\bar{A}_{*}$ to an open normally hyperbolic invariant manifold $\bar{A}_{*}^{\prime}\subset\mathcal{D}_{*}$. Now, equation (70) implies that $f^{N}$ is approximated up to terms of order $\tau_{*}$ by the time-1 shift $\phi^{1}_{Z}$ of the vector field $Z$ uniformly in the $C^{r}$ topology for $r$ finite, where $N$ is the integer part of $\tau_{*}^{-1}$. Fenichel theory implies that there is a $C^{r}$ neighbourhood of $\phi^{1}_{Z}$ such that for any map in this neighbourhood, the open $r$-normally hyperbolic invariant manifold $\bar{A}_{*}^{\prime}$ persists as an open $C^{r}$-smooth normally hyperbolic invariant manifold [18, 19, 20]. Shrinking $\tau_{*}$ as required, we can ensure that $f^{N}$ lies in this $C^{r}$ neighbourhood of $\phi^{1}_{Z}$, and so we obtain an open $C^{r}$-smooth normally hyperbolic invariant manifold $A_{*}^{\prime}$ for $f^{N}$ that is $O(\tau_{*})$-close to $\bar{A}_{*}^{\prime}$ in the $C^{r}$-topology for $r$ finite. Moreover, this manifold $A_{*}^{\prime}$ is itself an open normally hyperbolic invariant manifold for the billiard map $f$. Choosing the boundary curves of $\bar{A}_{*}$ to be KAM curves, we find that the manifold $A_{*}^{\prime}$ contains a compact $C^{r}$ normally hyperbolic invariant manifold $A_{*}$ of $f$ that is $O(\tau_{*})$-close to $\bar{A}_{*}$ in the $C^{r}$ topology, as explained in Section 3. Furthermore, the results imply that when we perturb the surface $\Gamma$, the normally hyperbolic invariant manifold $A_{*}$ of the billiard map persists. This is because the invariant manifold $\bar{A}_{*}$ for $Z$ persists in the tangent bundle of the perturbed hypersurface, and so we can apply the results again to obtain a normally hyperbolic invariant manifold for the billiard map on the perturbed manifold.

The next result shows that we can choose a sequence $\{\bar{A}_{n}\}_{n\in\mathbb{N}}$ of normally hyperbolic invariant cylinders for the flow of $Z$ such that: the cylinders overlap; as we move up the cylinders $\|u\|\to 1$; and the corresponding persistent cylinders $A_{n}$ of the billiard map fit together to form a manifold, itself having the structure of a noncompact cylinder (see Figure 5).

Proposition 6**.**

We can choose a sequence of constants $\{\tau_{n}\}_{n\in\mathbb{N}}$ such that $\tau_{n}\to 0$ as $n\to\infty$, and so that the corresponding cylinders $A_{n}$ for the billiard map fit together to form a noncompact nonuniformly normally hyperbolic invariant manifold. Moreover, $\|u\|\to 1$ as we move up the cylinder.

Proof.

Let $\bar{A}_{n}$ denote the cylinder $\bar{A}_{*}$ in (94) corresponding to $\tau_{*}=\tau_{n}$. We choose the sequence $\{\tau_{n}\}_{n\in\mathbb{N}}$ so that the top of the cylinder $\bar{A}_{1}$ which is $y_{1}=\frac{3}{2}$ matches up with the middle of $\bar{A}_{2}$ which is $y_{2}=1$ (see equation (94)). If we can choose $\tau_{1},\tau_{2}$ so that $y_{1},y_{2}$ correspond to the same value of $\|u\|$ at $(\gamma(q),\gamma^{\prime}(q))$ for each $q\in\mathbb{T}$, then the cylinders match up. We have

[TABLE]

so we need

[TABLE]

or more generally,

[TABLE]

which tends to 0 as $n\to\infty$. Therefore we choose $\tau_{1}>0$ sufficiently small so that Fenichel theory and KAM theory guarantee the persistence of the cylinder $\bar{A}_{1}$ for the billiard map, along with its transverse homoclinic intersection. Then define $\tau_{n}$ according to (105). Since the sequence is strictly decreasing, and since $\bar{A}_{n}$ as defined in (94) does not depend explicitly on $\tau_{n}$, each cylinder $\bar{A}_{n}$ persists for the billiard map, along with its transverse homoclinic intersection. Clearly $\|u\|\to 1$ as we move up the cylinders since, from (103), and for any $q\in\mathbb{T}$,

[TABLE]

Now, we must prove that

[TABLE]

has the structure of a cylinder. It is clear that $\mathrm{Int}(\bar{A}_{n})\cap\mathrm{Int}(\bar{A}_{n+1})\neq\emptyset$. Recall that each closed essential curve on $\bar{A}_{n}$ corresponding to constant $y$ is an invariant circle of the flow of $Z$ with frequency $\rho=\rho(y)$ depending only on $y$. Moreover, $\rho$ is strictly monotone in $y$. KAM theory ensures that curves corresponding to Diophantine frequencies survive perturbation and so are invariant circles for the billiard map. We may assume without loss of generality (as we may slightly shift the boundary curves of the cylinders $\bar{A}_{n}$ if necessary) that each cylinder $\bar{A}_{n}$ is bounded by curves with Diophantine frequencies.

Let $\bar{\gamma}_{1}$ be the lower boundary curve of $\bar{A}_{n+1}$ corresponding to $y=y_{1}$. For any $\epsilon>0$ we can find $\eta_{2}\in(y_{1},y_{1}+\epsilon)$ such that $\rho(\eta_{2})$ is Diophantine with respect to the map $f^{N}$. Let $\bar{\gamma}_{2}$ be the invariant circle corresponding to $y=\eta_{2}$. Then there are invariant circles $\gamma_{1}$, $\gamma_{2}$ for the billiard map that are $\tau_{n}$ close in the $C^{r}$-topology to $\bar{\gamma}_{1}$, $\bar{\gamma}_{2}$, respectively by KAM theory since the perturbation is exact symplectic. Clearly we must have $\gamma_{1},\gamma_{2}\subset A_{n}\cap A_{n+1}$. Let $D_{1}$ be the region in $A_{n}$ bounded by $\gamma_{1}$ and $\gamma_{2}$. Then $D_{1}$ is an invariant set for the billiard map. There is a characterisation of the local stable (respectively unstable) manifold of $A_{n+1}$ as the set of points whose forward (resp. backward) orbit never leaves a neighbourhood $V$ of $A_{n+1}$ [32]. Therefore the manifold $A_{n+1}$ is the set of points whose forward and backward orbits never leave $V$. By choosing $\epsilon$ small enough, we can guarantee that $D_{1}\subset V$ since $A_{n}$ and $A_{n+1}$ are $\tau_{n}$-close in the $C^{r}$-topology. It follows that $D_{1}\subset A_{n+1}$, and so the cylinders $A_{n}$ and $A_{n+1}$ agree between the curves $\gamma_{1}$ and $\gamma_{2}$. Now choose curves $\bar{\gamma}_{3},\ldots,\bar{\gamma}_{N}$ for some $N\in\mathbb{N}$ such that $\bar{\gamma}_{N}$ is the upper boundary of $\bar{A}_{n}$, and such that each $\bar{\gamma}_{j}\subset\bar{A}_{n}$ corresponds to $y=\eta_{j}$ where $\rho(\eta_{j})$ is a Diophantine number and $0<\eta_{j}-\eta_{j-1}<\epsilon$. Then this argument applies to the area between each pair of consecutive curves $\gamma_{j},\gamma_{j+1}$, and so the cylinders $A_{n},A_{n+1}$ agree where they overlap. This clearly applies for each $n\in\mathbb{N}$. ∎

5 Homoclinic Cylinders and Pseudo-Orbits

In this section we construct an iterated function system consisting of the restriction of the billiard map to the non-compact normally hyperbolic invariant manifold together with $n$ scattering maps. An orbit of the iterated function system is called a pseudo-orbit. We give sufficient conditions for the existence of diffusive pseudo-orbits, and explain how there are orbits of the billiard map that shadow pseudo-orbits arbitrarily well.

5.1 Analysis of Hyperbolic and Homoclinic Cylinders

It was shown in Section 4 that in $(x,w,z)$-coordinates defined by (67), the billiard map $f$ is approximated up to order $\tau_{*}^{2}$ by the time $\tau_{*}$-shift of the flow of the vector field $Z$, given by (69). Recall it was shown in the proof of Lemma 5 that we can write the normally hyperbolic invariant manifold $\bar{A}_{*}$ of the vector field $Z$ as in equation (99), which gives us natural coordinates $(\theta,y)$. Moreover, each point $(\theta,y)\in\bar{A}_{*}$ is a periodic point of the flow of the vector field $Z$ (which is given by (100), (101)) with minimal period $l(y)=K^{-1}y^{-1}$. Since $K>0$ depends only on $\gamma$, we can replace $Ky$ by $y$ and write

[TABLE]

Then the vector field and the flow take the simple form

[TABLE]

and

[TABLE]

Notice that the speed $z$ of the flow of $Z$ is now given by

[TABLE]

Since the geodesic flow is Hamiltonian, the stable and unstable manifolds of the hyperbolic closed geodesic $\gamma$, restricted to an energy level in the $2(d-1)$-dimensional manifold $M$, are of dimension $d-1$. Thus the 2-dimensional normally hyperbolic invariant manifold $\bar{A}_{*}$ has $d$-dimensional stable and unstable manifolds $W^{s,u}(\bar{A}_{*})$. On the stable (resp. unstable) manifold there is the strong stable (resp. strong unstable) foliation, each leaf of which is $(d-2)$-dimensional, and is uniquely determined by a point $(\theta,y)\in\bar{A}_{*}$ (see Section 3 for definitions).

Lemma 7**.**

The stable and unstable manifolds $W^{s,u}(\bar{A}_{*})$ of the normally hyperbolic invariant manifold $\bar{A}_{*}$ of the flow of $Z$ intersect strongly transversely along the set

[TABLE]

where $\xi$ is the homoclinic geodesic, and we can introduce coordinates $(\theta,y)$ on $B$ such that the holonomy maps

[TABLE]

defined as projections onto $\bar{A}_{*}$ along leaves of the strong stable and strong unstable foliations, are given by

[TABLE]

where $a_{\pm}\in\mathbb{R}$ are constants that do not depend on the point $(\theta,y)$.

Proof.

Since the orbit $\xi$ is a transverse homoclinic orbit of the geodesic flow in each energy level, it is a transverse homoclinic orbit for the flow of the vector field $Z$ in each energy level. Therefore the stable and unstable manifolds $W^{s,u}(\bar{A}_{*})$ intersect transversely along $B$. It is clear that this homoclinic intersection is strongly transverse, as it is strongly transverse for the geodesic flow.

It follows that the holonomy maps are well-defined and invertible on $B$ since every point on the homoclinic orbit $\xi$ is the intersection of exactly one leaf of the strong stable foliation and one leaf of the strong unstable foliation. Moreover they are $C^{r}$-smooth due to the regularity of the foliations [32].

Let us introduce the coordinates $(\theta,y)$ on $B$. If $(q,y)\in\mathbb{R}\times\left[\frac{K}{2},\frac{3K}{2}\right]$ defines the point $\left(\xi(q),\xi^{\prime}(q),Ky^{-1}\kappa(\xi(q),\xi^{\prime}(q))^{-\frac{4}{3}}\right)$ in $B$ then there is a unique $\theta\in\mathbb{R}$ such that

[TABLE]

Notice that, in these coordinates, the restrictions of the vector field $Z$ and its flow $\phi^{t}_{Z}$ to the homoclinic manifold $B$ are given by (109) and (110) respectively. For the rest of the proof we work in $(\theta,y)$ coordinates.

Consider the point $(0,y)\in B$, and let $(a_{+},y_{+})=\bar{\pi}^{s}_{B}(0,y)$. Since $y$ is constant along trajectories of the flow, it is constant along leaves of the strong stable foliation. Since the holonomy map is defined via projection along leaves of the strong stable folation, we have $y_{+}=y$. Now let $(\theta,y)\in B$ be another point with a different $\theta$-value and the same $y$-value. Then there is $t_{*}\in\mathbb{R}$ such that

[TABLE]

where we have used (110). From (30), (110), (115), and (116) we get

[TABLE]

so $a_{+}$ does not depend on $\theta$. Moreover it does not depend on $y$ since the dynamics of the flow (110) is the same on every energy level $y=const$. The second equation of (114) is proved analogously. ∎

Notice that the scattering map relative to the strip $B$ has (infinitely) many branches. The following corollary of Lemma 7 gives us a closed-form expression for the scattering map.

Corollary 8**.**

The scattering map

[TABLE]

defined by $\bar{s}_{B}(\theta,y)=\bar{\pi}^{s}_{B}\circ\left(\bar{\pi}^{u}_{B}\right)^{-1}$, is given by

[TABLE]

where

[TABLE]

Recall that a manifold $\bar{B}_{*}\subset\left(W^{s}(\bar{A}_{*})\pitchfork W^{u}(\bar{A}_{*})\right)\setminus\bar{A}_{*}$ is called a homoclinic cylinder if the scattering map

[TABLE]

is a diffeomorphism, where

[TABLE]

are the holonomy maps. It is clear from Lemma 7 that the set

[TABLE]

is a homoclinic cylinder to $\bar{A}_{*}$. Observe that $\bar{A}_{*}$ is a topological cylinder, while $\bar{B}_{*}$ is not.

Let us now discuss the normally hyperbolic invariant cylinder of the billiard map $f$. Recall that $f=g+O(\tau_{*}^{2})$ in $(x,w,z)$ coordinates. As a result of this, it was shown in Section 4 that there is a normally hyperbolic invariant cylinder $A_{*}$ for $f$ that is $O(\tau_{*})$ close to $\bar{A}_{*}$. Therefore we only know a priori that $f|_{A_{*}}$ is $O(\tau_{*})$ close to $g|_{\bar{A}_{*}}$, where $g$ is the time-$\tau_{*}$ shift of the flow of $Z$. The following lemma improves this approximation; the coordinates $((\theta,y),\alpha,\beta)$ introduced in this lemma are referred to as “Fenichel coordinates”.

Lemma 9**.**

There are coordinates $((\theta,y),\alpha,\beta)$ in a neighbourhood of $\bar{A}_{*}$ such that $(\theta,y)$ are coordinates both on $\bar{A}_{*}$ and on the normally hyperbolic invariant cylinder $A_{*}$ of the billiard map $f$. Moreover the restriction of $f$ to $A_{*}$ is given by

[TABLE]

where the higher order terms are uniformly bounded in the $C^{r}$ topology.

Proof.

Since $\bar{A}_{*}$ is an $r$-normally hyperbolic invariant manifold, there is a neighbourhood $U$ of $\bar{A}_{*}$ in $M$ in which we can introduce $C^{r}$ coordinates $((\theta,y),\alpha,\beta)$, where $\alpha=(\alpha_{1},\ldots,\alpha_{d-2})$ and $\beta=(\beta_{1},\ldots,\beta_{d-2})$, which straighten the stable and unstable manifolds of $\bar{A}_{*}$ in the following sense [33]. Fix a point $(\theta_{0},y_{0})$ on $\bar{A}_{*}$. Then we have:

• •

$W^{s}(\bar{A}_{*})=\{\alpha=0\}$

• •

$W^{u}(\bar{A}_{*})=\{\beta=0\}$

• •

$W^{s}(\theta_{0},y_{0})=\{((\theta,y),\alpha,\beta)\in U:(\theta,y)=(\theta_{0},y_{0}),\alpha=0\}$

• •

$W^{u}(\theta_{0},y_{0})=\{((\theta,y),\alpha,\beta)\in U:(\theta,y)=(\theta_{0},y_{0}),\beta=0\}$.

The vector field $Z$ in these coordinates is given by $Z((\theta,y),\alpha,\beta)=((\dot{\theta},\dot{y}),\dot{\alpha},\dot{\beta})$ with

[TABLE]

where

[TABLE]

Since this is a $C^{r}$ change of coordinates, and since $f=g+O(\tau_{*}^{2})$ in $(x,w,z)$ coordinates, we have $f=g+O(\tau_{*}^{2})$ uniformly in $((\theta,y),\alpha,\beta)$ coordinates in the $C^{r}$ topology.

Notice that, in $((\theta,y),\alpha,\beta)$ coordinates, the normally hyperbolic invariant cylinder of $Z$ is given by

[TABLE]

Fenichel theory implies that the normally hyperbolic invariant cylinder $A_{*}$ of $f$ can be written as a graph over $\bar{A}_{*}$. Thus there are maps $\nu,\eta:\mathbb{T}\times\left[\frac{K}{2},\frac{3K}{2}\right]\to\mathbb{R}^{d-2}$ that are uniformly $O(\tau_{*})$ close to 0 in $C^{r}$ such that

[TABLE]

It follows that we can use the same $(\theta,y)$ coordinates on both $\bar{A}_{*}$ and $A_{*}$, and since $f=g+O(\tau_{*}^{2})$ uniformly in $C^{r}$ in the Fenichel coordinates $((\theta,y),\alpha,\beta)$, we see, using (127), that

[TABLE]

since $\nu,\eta$ are $O(\tau_{*})$ close to 0, which is (126). ∎

Now, there is a homoclinic strip $B_{*}$ to $A_{*}$ that is $O(\tau_{*})$ close to $B$ (where $B$ is the homoclinic strip for the vector field $Z$ defined in Lemma 7) as a result of the implicit function theorem. The strip $B_{*}$ is a connected component of the transverse homoclinic intersection of the stable and unstable manifolds of $A_{*}$, and it is $f$-invariant. Denote by $s_{B_{*}}:A_{*}\to A_{*}^{\prime}$ the multibranched scattering map defined by the homoclinic strip $B_{*}$, where $A_{*}^{\prime}\subset A$ is an open normally hyperbolic invariant manifold containing $A_{*}$, and $A$ is the noncompact cylinder comprising the union of all compact invariant cylinders $A_{n}$. Since the perturbed homoclinic strip $B_{*}$ is $O(\tau_{*})$ close to $B$ in the $C^{r}$-topology, we have that any particular branch of the map locally satisfies

[TABLE]

uniformly in the $C^{r}$ topology for $r$ finite. The map $s_{B_{*}}$ has infinitely many branches along the strip $B_{*}$. If we restrict the homoclinic channel to subsets of $B_{*}$, the corresponding scattering map may be a single-branched mapping defined on a subset $U$ of $A_{*}$, as long as $U$ does not contain an essential curve of $A_{*}$. However, it is possible that the perturbation from the time-$\tau_{*}$ map of the vector field $Z$ to the billiard map $f$ introduces monodromy for the scattering map $s_{B_{*}}$.

In Section 7 we introduce a perturbation scheme that requires the scattering map to be globally defined on a cylinder. Since it is not clear that this is the case for $s_{B_{*}}$, we instead focus on a fundamental domain of $f$ on its normally hyperbolic cylinder $A_{*}$. We glue the boundaries of the fundamental domain to make it a cylinder, and introduce the modified scattering maps, which we prove to be exact symplectic, globally defined diffeomorphisms that are homotopic to the identity on the glued fundamental domain.

Denote by $D$ the fundamental domain of $f$ on $A_{*}$ contained between the line $\{\theta=0\}$ and its image under $f$. We assume that the left boundary $\{\theta=0\}$ is contained in $D$ and the right boundary is not.

Denote by $(t,y)$ the time-energy coordinates of the vector field $Z$ in a neighbourhood of $D$ in $A_{*}$, so $y$ is the integral defined by (111), $t$ is the time along orbits of the flow of $Z$, the symplectic form is $dt\wedge dy$, and $Z(t,y)=(1,0)$.

Lemma 10**.**

We can introduce coordinates $(\hat{\varphi},\hat{y})$ in a small neighbourhood of the closure of the fundamental domain $D$ in $A_{*}$ such that

[TABLE]

where $a,b$ are $O(\tau_{*})$-close to $\frac{K}{2}$, $\frac{3K}{2}$ respectively, and where the coordinate transformation $(t,y)\mapsto(\hat{\varphi},\hat{y})$ is $O(\tau_{*})$-close to the identity in the $C^{1}$ topology. Moreover in these coordinates the restriction of the symplectic form to $A_{*}$ is $d\hat{\varphi}\wedge d\hat{y}$, and the billiard map takes the form

[TABLE]

Proof.

In the coordinates $(t,y)$, the billiard map takes the form

[TABLE]

for some bounded $C^{r}$ functions $q,p$. Moreover, $f$ preserves the restriction of the symplectic form $\omega$ to $A_{*}$. Denote by $\alpha,\beta$ the hyperbolic part of the coordinates (see Lemma 9). Since $\alpha|_{A_{*}},\beta|_{A_{*}}=O(\tau_{*})$, $\omega|_{A_{*}}$ takes the form $h(t,y)dt\wedge dy$ where $h=1+O(\tau_{*}^{2})$ in the $C^{r-1}$ topology, because the $\alpha,\beta$-subspaces in the tangent space at each point in $\bar{A}_{*}$ are $\omega$-orthogonal to $\bar{A}_{*}$. Let

[TABLE]

so that $f$ maintains the form (135) in the new coordinates, and preserves the symplectic form $\omega=dt\wedge d\tilde{y}$.

Now, in a small neighbourhood of the fundamental domain $D$ in $A_{*}$, consider a function

[TABLE]

where $p$ comes from (135), $\eta$ is a smooth function that is identically equal to 1 in a small neighbourhood of 0, and vanishes identically near the right boundary of $D$. Notice that the $t$-derivative of $\eta$ is of order $\tau_{*}^{-1}$. Clearly $\check{y}$ is $O(\tau_{*}^{2})$-close to $\tilde{y}$ in $C^{0}$, and its $\tilde{y}$-derivative is $1+O(\tau_{*}^{2})$, but its $t$-derivative is $O(\tau_{*})$. By construction, we have

[TABLE]

so $\check{y}$ is an integral of $f$ in a neighbourhood of the left boundary of $D$. In the variables $(t,\check{y})$ $f$ takes the form

[TABLE]

where $r$ is bounded along with its first derivatives. Moreover the symplectic form is

[TABLE]

where the coordinate $\varphi$ is defined by

[TABLE]

Then $f$ preserves the standard symplectic form in the new coordinates, and is given by

[TABLE]

Finally, replacing $\varphi,\check{y}$ by

[TABLE]

respectively, we obtain coordinates in which the symplectic form is $d\hat{\varphi}\wedge d\hat{y}$, and $f$ takes the form

[TABLE]

Since each coordinate transformation we have made is (at least) $O(\tau_{*})$ close to the identity in $C^{1}$, the lemma is proved. ∎

Now, drop the ‘hat’ notation, and write $(\varphi,y)$ instead of $(\hat{\varphi},\hat{y})$. Since $D$ is a fundamental domain of the near-identity map $f$ on the cylinder $A_{*}$, we can find a (connected) preimage $\Delta=\left(\pi_{B_{*}}^{u}\right)^{-1}\left(D\right)$ of $D$ on the homoclinic strip $B_{*}$. Then the scattering map $s_{B_{*}}$, relative to a neighbourhood of $\Delta$ in $B_{*}$, is an exact symplectic diffeomorphism onto its image $s_{B_{*}}(D)$ when restricted to a neighbourhood of $D$ in $A_{*}$. This is due to results of [14] combined with the facts that $f$ is an exact symplectic diffeomorphism, and that the restriction of the symplectic form $\omega$ to $A_{*}$ is nondegenerate. Denote by $D^{\prime}$ the extension of the fundamental domain $D$ to the open normally hyperbolic invariant manifold $A_{*}^{\prime}$. Define the function $n:s_{B_{*}}\left(D\right)\to\mathbb{N}$ by

[TABLE]

for $(\varphi,y)\in D$. This is well-defined because $D^{\prime}$ is a fundamental domain of $f$ in $A_{*}^{\prime}$. We then define the modified scattering map $\tilde{s}:D\to D^{\prime}$ by

[TABLE]

The following result follows from the definition of the modified scattering map. As the proof is lengthy, it is deferred to an appendix.

Lemma 11**.**

If we consider $D$ as a cylinder by identifying points on the line $\{\varphi=0\}$ with their image under $f$, and use the coordinates $(\varphi,y)\in[0,\tau_{*})\times\left[a,b\right]$, then the modified scattering map $\tilde{s}:D\to\tilde{s}(D)\subset D^{\prime}$ is an exact symplectic $C^{r}$-diffeomorphism onto its image, and its image contains an essential curve in $D^{\prime}$.

Remark*.*

In the statement of Lemma 11, it is claimed that the modified scattering map is a $C^{r}$-diffeomorphism. If the cylinder $A_{*}$ is only $C^{r}$ smooth, then actually the scattering map is only $C^{r-1}$, and so the modified scattering map is also only $C^{r-1}$. However, we are able to choose $r$ to be any finite positive integer (shrinking $\tau_{*}$ if necessary). Therefore if we initially choose $r+1$, then the modified scattering map is $C^{r}$ smooth, as stated in the lemma.

In the coming chapters we analyse the effect that a perturbation of the hypersurface $\Gamma$ near a fundamental domain of $f$ on the homoclinic strip $B_{*}$ has on the modified scattering map.

Now, the billiard map $f$ does not map the fundamental domain $D$ to itself. Let us define the inner map $\Phi:D\to D$ as follows. For $(\varphi,y)\in D$ let

[TABLE]

This is well-defined since $D$ is a fundamental domain of $f$ on $A_{*}$. Then we define

[TABLE]

Lemma 12**.**

The map $\Phi:D\to D$ is an exact symplectic $C^{r}$-diffeomorphism and satisfies a twist property.

Proof.

The proof that $\Phi$ is an exact symplectic diffeomorphism is similar to the proof of Lemma 11, and so we do not include it here. We show that $\Phi$ satisfies a twist property.

Denote by

[TABLE]

the fundamental domain of $g=\phi^{\tau_{*}}_{Z}$ in $\bar{A}_{*}$, where $(\theta,y)$ are the coordinates on the cylinder $\bar{A}_{*}$ defined at the beginning of the section. Define

[TABLE]

and define the return map of $g$ to $\bar{D}$ as

[TABLE]

Let $(\theta_{0},y_{0})\in\bar{D}$. Then there is a unique $t_{0}\in[0,\tau_{*})$ such that

[TABLE]

Therefore $t_{0}=\theta_{0}\,y_{0}^{-1}$. Recall that the period of $(\theta_{0},y_{0})$ under the flow of $Z$ is $y_{0}^{-1}$. It follows that the image of $(\theta_{0},y_{0})$ under the time-$(y_{0}^{-1}-t_{0})$ map of $Z$ is the point $(0,y_{0})$. Let $t_{1}=(t_{0}-y_{0}^{-1})\mod\tau_{*}$. Then $\tau_{*}\bar{N}(\theta_{0},y_{0})=(y_{0}^{-1}-t_{0})+t_{1}$, and so

[TABLE]

We have

[TABLE]

and so $\Psi$ satisfies a twist condition in time-energy coordinates. By Lemma 9, $f,g$ are $O(\tau_{*}^{2})$-close. Therefore $\Psi,\Phi$ are $O(\tau_{*})$-close, and so $\Phi$ also satisfies a twist condition. Since the coordinate transformation is $O(\tau_{*})$-close to the identity, the twist condition is also satisfied in $(\hat{\varphi},\hat{y})$ coordinates (see Lemma 10). ∎

5.2 Shadowing of Pseudo-Orbits

Let $\{A_{n}\}_{n\in\mathbb{N}}$ denote the sequence of normally hyperbolic invariant cylinders for the billiard map $f$ obtained in Proposition 6, and let

[TABLE]

denote the noncompact cylinder.

Standard results imply that the existence of a single transverse homoclinic geodesic $\xi$ implies the existence of infinitely many (see e.g. Theorem 6.5.5 of [34]). Suppose we take $N$ such geodesics $\xi_{1},\ldots,\xi_{N}$. Let $B_{1},\ldots,B_{N}$ denote the corresponding homoclinic strips for the billiard map, and consider the scattering maps $s_{j}=s_{B_{j}}$. We now construct an iterated function system $\left\{f|_{A},s_{1},\ldots,s_{N}\right\}$ on $A$, and call orbits of the IFS pseudo-orbits.

The following theorem is a generalisation of a theorem of Moeckel [42]. See also [24, 38]. The proof here relies on the global definition of the modified scattering maps, but recent progress has relaxed this assumption [26]. The theorem tells us that, given a prescribed sequence $\{A_{m_{n}}\}$ of consecutive cylinders, we can find a pseudo-orbit $\{\mu_{n}\}$ visiting the cylinders $A_{m_{n}}$ in the prescribed order.

Theorem 13**.**

(Diffusive and Oscillatory Pseudo-Orbits) For each $n\in\mathbb{N}$, denote by $D_{n}$ the fundamental domain in $A_{n}$ defined by (133). Denote by $\tilde{s}_{j}$ the modified scattering maps, and by $\Phi$ the inner map, defined by (146), (148), respectively. Suppose for each $n\in\mathbb{N}$ the maps $\Phi,\tilde{s}_{1},\ldots,\tilde{s}_{N}$ have no common invariant essential curves on the glued fundamental domain $D_{n}$.

Let $\{m_{n}\}_{n\in\mathbb{N}}\subseteq\mathbb{N}$ be such that $m_{n+1}=m_{n}\pm 1$ for each $n$. Then there is a pseudo-orbit $\{\mu_{n}\}_{n\in\mathbb{N}}\subset A$ of the iterated function system $f,s_{1},\ldots,s_{N}$ and $\{k_{n}\}_{n\in\mathbb{N}}\subset\mathbb{N}$ with

[TABLE]

for each $n\in\mathbb{N}$, and

[TABLE]

for each $j=k_{n}+1,k_{n}+2,\ldots,k_{n+1}-1$.

Proof.

The theorem is proved by applying the proof of Theorem 3 of [24] inductively to the fundamental domain $D_{m_{n}}$ in each compact invariant subcylinder $A_{m_{n}}$. Indeed, it follows from Lemma 12 that the inner map $\Phi$ is an exact symplectic twist map, and from Lemma 11 that the modified scattering maps $\tilde{s}_{j}$ are exact symplectic diffeomorphisms (onto their images), and are homotopic to the identity. Therefore the assumptions of Theorem 3 of [24] are satisfied.

Let $\gamma_{\pm}$ denote the essential curves bounding the bottom and top of the cylinder $D_{m_{1}}$, and assume $m_{2}=m_{1}+1$. Construct a curve $\tilde{\gamma}$ as follows: the curves $\Phi(\gamma_{-}),\tilde{s}_{1}(\gamma_{-}),\ldots$, $\tilde{s}_{N}(\gamma_{-})$ divide $D_{m_{1}}$ into connected components. If one of these curves intersects $\gamma_{+}$ then we have already found a pseudo-orbit connecting $A_{m_{1}}$ and $A_{m_{2}}$. If not, let $\tilde{\gamma}$ denote the lower boundary of the connected component of $D_{m_{1}}$ whose upper boundary is $\gamma_{+}$. Write $\tilde{\gamma}=F({\gamma_{-}})$. Here $F$ is just an operation on closed curves. By Theorem 3 of [24], by iterating $F$ on $\gamma_{-}$, we can obtain a curve $F^{n_{1}}(\gamma_{-})$ that intersects $\gamma_{+}$, which in turn gives a pseudo-orbit connecting the bottom of $D_{m_{1}}$ to the top. Since $A_{m_{1}}$ and $A_{m_{2}}$ overlap (see Proposition 6), the connecting pseudo-orbit lands in $D_{m_{2}}$, since it is a fundamental domain of $f$ in $A_{m_{2}}$. Therefore we can perform the process again to show that there is some $n_{2}\in\mathbb{N}$ such that $F^{n_{1}+n_{2}}(\gamma_{-})$ intersects the upper boundary curve of $D_{m_{2}}$. Similarly if $m_{2}=m_{1}-1$ we can find pseudo-orbits starting on $\gamma_{+}$ and ending on $\gamma_{-}$, thus connecting $A_{m_{1}}$ and $A_{m_{2}}$ in that case. Since orbits of the iterated function system $\Phi,\tilde{s}_{1},\ldots,\tilde{s}_{N}$ correspond to orbits of the iterated function system $f,s_{1},\ldots,s_{N}$, we can repeat the process inductively to obtain Theorem 13. ∎

Lemma 14**.**

(Shadowing Lemma) Let $\epsilon>0$, and let $\{\mu_{n}\}_{n=0}^{\infty}$ be a pseudo-orbit, so for each $n=0,1,\ldots$, we have either $\mu_{n+1}=f(\mu_{n})$, or there is an $\alpha_{n}\in\{1,\ldots,N\}$ such that

[TABLE]

Then there is an orbit $\{\nu_{n}\}_{n=0}^{\infty}$ of $f$ in $M$ such that for some $\{m_{n}\}_{n=0}^{\infty}\subset\mathbb{N}$ we have

[TABLE]

and

[TABLE]

The Lemma is equivalent to Lemma 3 of [24] and Lemma 3.11 of [25], so we do not include it here. In those papers it is assumed that the manifold $A$ is compact. The purpose of compactness is to allow application of the Poincaré recurrence theorem. In our case, the structure of $A$ as a countable union of compact invariant subcylinders allows us to adapt the proof as we can apply the recurrence theorem in each compact invariant subcylinder.

It follows from Theorem 13 and Lemma 14 that if we can show that the property of the inner map $\Phi$ and the $N$ modified scattering maps having no common invariant essential curves in each fundamental domain $D_{n}$ in $A$ is a residual property of $Q\in\mathcal{V}$, then Theorems 1 and 2 are proved.

6 Effect of a Perturbation on the Billiard and Scattering Maps

6.1 Perturbing the Billiard Map

In order to perturb the billiard map, we perturb the surface $\Gamma$. Recall that $\Gamma$ is given implicitly as the set of zeros of a real-analytic function $Q\in\mathcal{V}$. In this section, we will show how the billiard map is affected by compactly supported perturbations of the surface $\Gamma$. Since such perturbations are not real-analytic, we will later approximate the perturbed system $Q+\epsilon\psi$ by a real-analytic family $Q_{\epsilon}\subset\mathcal{V}$.

Fix some $(x,u)\in M$ such that $\|u\|<1$ and let $(\bar{x},\bar{u})=f(x,u)$. Let $\epsilon>0$ and $\psi:\mathbb{R}^{d}\to\mathbb{R}$ be a smooth function supported in a neighbourhood of $\bar{x}$. In particular, assume $x\notin\mathrm{supp}(\psi)$. Make the perturbation

[TABLE]

Denote by $f_{\epsilon}$ the perturbed billiard map, and let $(\tilde{x},\tilde{u})=f_{\epsilon}(x,u)$. Referring to (11), we have

[TABLE]

where $\tau_{\epsilon}(x,u)=t>0$ such that $Q_{\epsilon}(x+tv)=0$, and

[TABLE]

Since $\epsilon$ is sufficiently small, the new surface is still strictly convex and so $\tau_{\epsilon}$ is still well-defined. Clearly for the $\tilde{x}$-component, the perturbation is manifested only through $\tau_{\epsilon}$. Therefore the term of order $\epsilon$ in the Taylor series of $\tilde{x}$ is $\Theta(\bar{x},\bar{u})v$, where $\Theta:M\to\mathbb{R}$ is a smooth function. We determine $\Theta$ by computing the first term in the Taylor series of $Q_{\epsilon}(\tilde{x})=0$. Since this term must itself vanish, we have

[TABLE]

Elementary geometrical considerations imply that we can write $v$ in terms of $(\bar{x},\bar{u})$ as $v=\bar{u}-\sqrt{1-\bar{u}^{2}}\,n(\bar{x})$. Therefore we have

[TABLE]

In order to compute the effect of the perturbation on the $u$-component, we write $\tilde{u}=\bar{u}+\epsilon\Pi(\bar{x},\bar{u})+O(\epsilon^{2})$. This computation is more direct: we first compute the expansion of $\tilde{n}(\tilde{x})$ up to first order, using (163) and (167), and then substitute this into the expression for $\tilde{u}$ in (162). Write $\tilde{n}(\tilde{x})=n(\bar{x})+\epsilon V(\bar{x},\bar{u})+O(\epsilon^{2})$. A straightforward computation shows that

[TABLE]

Notice that $V(\bar{x},\bar{u})\in T_{\bar{x}}\Gamma$. It follows from (162) that

[TABLE]

We compute

[TABLE]

We thus obtain the following expressions for $\Pi$:

[TABLE]

Equation (174) will be useful later since $V(\bar{x},\bar{u})\in T_{\bar{x}}\Gamma$, whereas the second term is normal to $\Gamma$ at $\bar{x}$.

Now, let

[TABLE]

The time-$\epsilon$ shift of the flow of $\tilde{X}$ maps the phase space $M=M(Q)$ of the unperturbed billiard to the phase space $M_{\epsilon}=M(Q+\epsilon\psi)$ of the perturbed billiard. The following lemma provides us with a way of describing the dynamics of the perturbed billiard map $f_{\epsilon}$ in terms of the coordinates on the unperturbed phase space $M$.

Lemma 15**.**

There is a smooth symplectic map $h:M\to M$ with the following properties.

The map $h$ is $O(\epsilon)$ close to the identity in the $C^{r}$ topology, and coincides with the identity at points $(x,u)\in M$ where $x\notin\mathrm{supp}(\psi)$. 2. 2.

For $(x,u)\in M$ and $n\in\mathbb{N}$ such that $f_{\epsilon}^{n}(x,u)\in M$, we have

[TABLE] 3. 3.

The map $h$ is approximated up to terms of order $\epsilon^{2}$ by the time-$\epsilon$* shift of the flow of the Hamiltonian function*

[TABLE]

Proof.

Let $(x^{0},u^{0})\in M$ such that $x^{0}\notin\mathrm{supp}(\psi)$, and $\bar{x}\in\mathrm{supp}(\psi)$ where $(\bar{x},\bar{u})=f(x^{0},u^{0})$. Let $(\tilde{x},\tilde{u})=f_{\epsilon}(x^{0},u^{0})$, $(x^{1},u^{1})=f_{\epsilon}(\tilde{x},\tilde{u})$, and $(\hat{x},\hat{u})=f^{-1}(x^{1},u^{1})=\mathcal{I}\circ f\circ\mathcal{I}(x^{1},u^{1})$ where the involution operator $\mathcal{I}:T\mathbb{R}^{d}\longrightarrow T\mathbb{R}^{d}$ sends $(x,u)\longmapsto(x,-u)$. We also assume that $x^{1}\notin\mathrm{supp}(\psi)$, as the remaining case can be treated similarly. Consider the near-to-the-identity maps

[TABLE]

Let

[TABLE]

By construction we have

[TABLE]

and

[TABLE]

where $\omega=dx\wedge du$ is the standard symplectic form on $\mathbb{R}^{2d}$. Therefore $h$ is symplectic, and so is approximated up to terms of order $\epsilon^{2}$ by the time-$\epsilon$ shift of some Hamiltonian vector field on $M$ with Hamiltonian $H_{\mathrm{pert}}$ which we must determine.

The map $h_{0}$ is approximated up to terms of order $\epsilon^{2}$ by the time-$\epsilon$ map of the vector field $\tilde{X}$, defined by (176). Let

[TABLE]

and denote by

[TABLE]

the tangential component of $\nabla\psi(\bar{x})$. It can be seen that

[TABLE]

Denote by $\tilde{\phi}^{t}$ the flow defined by

[TABLE]

We have

[TABLE]

Therefore

[TABLE]

and so

[TABLE]

where

[TABLE]

We must compute $X_{\mathrm{pert}}$ and its Hamiltonian. We have

[TABLE]

and

[TABLE]

Therefore

[TABLE]

[TABLE]

and

[TABLE]

It follows that

[TABLE]

Let

[TABLE]

so that

[TABLE]

and

[TABLE]

Let

[TABLE]

where $B$ is to be determined. Then, on $M$,

[TABLE]

We choose $B$ so that, on $M$,

[TABLE]

Therefore

[TABLE]

Since

[TABLE]

we find that

[TABLE]

∎

6.2 Perturbative Computation of the Scattering Map

Recall from Section 5 that $D$ is a fundamental domain for the billiard map $f$ on the normally hyperbolic invariant cylinder $A_{*}$. Moreover, there is an $f$-invariant homoclinic strip $B_{*}$ relative to $A_{*}$ with respect to which the scattering map $s_{B_{*}}:A_{*}\to A_{*}^{\prime}$ is a $C^{r}$ multibranched map into an open normally hyperbolic invariant cylinder $A_{*}^{\prime}$ containing $A_{*}$. On $D$ we defined the modified scattering map $\tilde{s}:D\to D^{\prime}$ relative to a fundamental domain $\Delta$ of $f$ in $B_{*}$. Recall that there are coordinates $(\varphi,y)\in[0,\tau_{*})\times\left[a,b\right]$ on $D$ for some $b>a>0$ defined by Lemma 10.

Recall moreover that $s_{B_{*}}=\pi^{s}\circ(\pi^{u})^{-1}$ where the holonomy maps $\pi^{s},\pi^{u}:B_{*}\to A_{*}$ are $C^{r}$ smooth. Let

[TABLE]

where $n:s_{B_{*}}\left(D\right)\to\mathbb{Z}$ was defined in Section 5.1. Then $\tilde{B}_{*}$ can be considered as a homoclinic cylinder relative to $D$ with respect to the modified scattering map $\tilde{s}$ (despite $\tilde{B}_{*}$ not being a topological cylinder). Indeed, denoting by

[TABLE]

we see that

[TABLE]

Suppose now we make a perturbation $Q\to Q+\epsilon\psi$ to the hypersurface where $\psi:\mathbb{R}^{d}\to\mathbb{R}$ is a smooth function supported near the projection $\pi\circ f(\tilde{B}_{*})\subset\Gamma$ of the homoclinic cylinder $f(\tilde{B}_{*})$ to the hypersurface $\Gamma$. In this section we determine the effect of the $C^{\infty}$ perturbation $\psi$ on the modified scattering map. This is a key computation for our proof: in Section 7 we give conditions on the terms of order $\epsilon$ in the expansion of the perturbed modified scattering map which, if satisfied, imply diffusion. Using the results of this section, we show that these conditions are satisfied by certain families of smooth perturbations. We then approximate the smooth family $Q+\epsilon\psi$ by a real-analytic family $Q_{\epsilon}$ where $Q_{0}\equiv Q$.

Denote by $\tilde{s}_{\epsilon}$ the perturbed modified scattering map. Since $\tilde{s}_{\epsilon}$ has smooth dependence on $\epsilon$, we can write

[TABLE]

Finally, recall that the coordinates $(x,w,z)$ are defined by (67).

Proposition 16**.**

Let $(\varphi,y)\in D$. Then

[TABLE]

and

[TABLE]

where the higher order terms are uniformly bounded in the $C^{0}$ topology, and where $(\bar{x},\bar{w},\bar{z})$ are the $(x,w,z)$ coordinates of the point

[TABLE]

with the modified scattering map $\tilde{s}$ defined by (146).

Proof.

First, we derive a formula for the Hamiltonian of the perturbation of the modified scattering map, by using the Hamiltonian of the perturbation of the unmodified scattering map. By Lemma 10, the restriction of the symplectic form to $A_{*}$ is given by $\omega|_{A_{*}}=d\varphi\wedge dy$. Drop the $\tilde{B}_{*}$ subscript and denote by $s$ the unperturbed scattering map relative to $\tilde{B}_{*}$ (i.e. not the modified scattering map). Since we are interested in the behaviour of $s$ only near a fundamental domain of the near-identity exact symplectic map $f$ in $A_{*}$, we can use the results of [14] to infer that $s$ is locally exact symplectic. Denote by $s_{\epsilon}$ the perturbed scattering map. Then $s_{\epsilon}$ is a deformation of $s$ in the class of exact symplectic diffeomorphisms defined on a subset of the cylinder $A_{*}$. Therefore we can write $s_{\epsilon}=\tilde{\phi}^{\epsilon}\circ s$ where $\tilde{\phi}^{t}$ is the time-$t$ map of some Hamiltonian vector field $\Omega\nabla\tilde{H}_{\mathrm{pert}}$ where $\tilde{H}_{\mathrm{pert}}$ is a Hamiltonian function. Therefore the perturbed scattering map can be written

[TABLE]

Recall that $H_{\mathrm{pert}}$, the Hamiltonian of the perturbation of $f$, is defined by (178). Notice that the assumptions of Theorem 31 of [14] are satisfied. Therefore we have

[TABLE]

where all but one term has vanished due to $\psi$ being supported only near $\pi\circ f(\tilde{B}_{*})$. Indeed, since $\tilde{B}_{*}$ is a fundamental domain of $f$ on the homoclinic strip $B_{*}$, we find that the sets $f^{m}\left(\tilde{B}_{*}\right)$ are mutually disjoint (see also the discussion at the beginning of Section 7 regarding the support of perturbations).

Now, by Lemma 11, the modified scattering map is exact symplectic. Moreover, the perturbed modified scattering map $\tilde{s}_{\epsilon}$ satisfies

[TABLE]

Therefore the term of order $\epsilon$ is a Hamiltonian vector field, and the Hamiltonian of the perturbation of $\tilde{s}_{\epsilon}$ is the same as that of $s_{\epsilon}$ in $(\varphi,y)$ coordinates.

We must compute the derivatives of $\tilde{H}_{\mathrm{pert}}$ with respect to $(\varphi,y)$. Recall that the coordinates $(\varphi,y)$ defined by (133) are $O(\tau_{*})$-close in $C^{1}$ to the time-energy coordinates $(t,y_{Z})$ of the vector field $Z$ (defined by (69)), where $t$ is the time along orbits, and $y_{Z}$ is the integral defined as in (111). By Lemma 10, the Jacobian matrix $J$ of the coordinate transformation $(t,y_{Z})\mapsto(\varphi,y)$ satisfies

[TABLE]

where the higher order terms are uniformly bounded in the $C^{0}$ topology. It follows that

[TABLE]

The idea of the forthcoming computation is to compute the derivatives of $\tilde{H}_{\mathrm{pert}}$ with respect to $t,y_{Z}$, and then use (224), (225) to approximate the $\varphi,y$-derivatives of $\tilde{H}_{\mathrm{pert}}$.

Now, we can write the Hamiltonian $H_{\mathrm{pert}}$ of the perturbation of the billiard map in terms of $(\bar{x},\bar{w},\bar{z})$ coordinates, and therefore in terms of $(\bar{x},\bar{w},\bar{y}_{Z})$ coordinates. Indeed, (67) and (111) imply that

[TABLE]

which gives

[TABLE]

Equations (226) and (227), and the fact that $\bar{u}=\|\bar{u}\|\bar{w}$ combined with equation (178) give the formula for $H_{\mathrm{pert}}$ in terms of $(\bar{x},\bar{w},\bar{y}_{Z})$. Then the derivatives of $H_{\mathrm{pert}}$, on $M$, with respect to $(\bar{x},\bar{w},\bar{y}_{Z})$ are

[TABLE]

[TABLE]

and

[TABLE]

Even though $w$ belongs to the unit sphere in the tangent space to $\Gamma$ at $x$, we take the derivative with respect to $w$ in the usual sense. Recall in Section 4 we assumed that $\|\nabla Q(x)\|=1$ for all $x\in\Gamma$ which gave us the simple formula (34) for the shape operator. Now that we have made perturbations, we may again make this assumption without loss of generality. It follows from this assumption (see (34)) that

[TABLE]

Now, let $(\varphi,y)\in D$, and denote by $(\bar{x},\bar{w},\bar{y}_{Z})$ the point $f\circ(\pi^{s})^{-1}\circ f^{n(\varphi,y)}(\varphi,y)$ in $(x,w,y_{Z})$ coordinates. With $t$ denoting the time along orbits of the vector field $Z$, differentiating (221) and using equations (69), (111), (219), (226), (228), (229), and (231) together with the facts that $\bar{y}_{Z}$ is an integral of $Z$ and $\left\|\nabla Q\right\||_{\Gamma}\equiv 1$ give

[TABLE]

Equations (230) and (235) combined with (224) and (225), and the fact that the term of order $\epsilon$ in the expansion of $\tilde{s}_{\epsilon}$ is $\left(\Omega\nabla\tilde{H}_{\mathrm{pert}}\right)\circ\tilde{s}$ complete the proof of the proposition. ∎

7 Diffusive Orbits

Recall that the existence of the homoclinic strip $B_{*}$ to the normally hyperbolic invariant cylinder $A_{*}$ of the billiard map $f$ hinges on the existence of a transverse homoclinic geodesic $\xi$ to the hyperbolic closed geodesic $\gamma$. As mentioned in Section 5.2, the existence of a single transverse homoclinic geodesic implies the existence of infinitely many. Choose 8 distinct transverse homoclinic geodesics $\xi_{1},\ldots,\xi_{8}$. These homoclinic geodesics give 8 homoclinic strips $B_{*,j}$ for the billiard map that are independent (in the sense that their orbits under $f$ have no intersection), each of which defines a $C^{r}$ multibranched scattering map $s_{j}$ on $A_{*}$. We then defined the modified scattering maps $\tilde{s}_{j}:D\to D^{\prime}$ for some fundamental domain $D$ of $f$ in $A_{*}$, where $D^{\prime}\subset A_{*}^{\prime}$ is an extension of the fundamental domain $D$ to the open normally hyperbolic cylinder $A_{*}^{\prime}$. It was shown that some fundamental domain $\tilde{B}_{*,j}\subset B_{*,j}$ defined by (213) can be considered a homoclinic cylinder for the modified scattering map $\tilde{s}_{j}$ relative to $D$. Moreover, we defined the inner map $\Phi:D\to D$ by (148).

By shrinking $\tau_{*}$ and shifting the fundamental domain $\tilde{B}_{*,j}$ along the homoclinic intersection if necessary, we may assume that each projected set $\pi\circ f\left(\tilde{B}_{*,j}\right)\subset\Gamma$ contains no points belonging to a set of the form $\pi\circ f^{m}\left(\tilde{B}_{*,k}\right)$ where $k\neq j$ and $m\in\mathbb{Z}$. Therefore we can find some small neighbourhood $V_{j}$ of $\pi\circ f\left(\tilde{B}_{*,j}\right)$ for each $j=1,\ldots,8$ such that:

• •

$V_{j}\cap\pi(A_{*})=\emptyset$;

• •

$V_{j}\cap V_{k}=\emptyset$ if $j\neq k$; and

• •

$V_{j}\cap\left(\bigcup_{m\in\mathbb{Z}}\pi\circ f^{m}\left(\tilde{B}_{*,j}\right)\right)=\pi\circ f\left(\tilde{B}_{*,j}\right)$.

In what follows we will make perturbations inside each $V_{j}$ to affect only the scattering map corresponding to $\tilde{B}_{*,j}$, without interfering with the other scattering maps.

Recall that we have a sequence $\{A_{n}\}_{n\in\mathbb{N}}$ of such cylinders so that the set

[TABLE]

is a noncompact normally hyperbolic invariant manifold for $f$. Denote by $D_{n}$ the fundamental domain in $A_{n}$ for each $n\in\mathbb{N}$. Combining the following theorem with Theorem 13 and Lemma 14 completes the proof of Theorems 1 and 2.

Theorem 17**.**

The maps $\Phi,\tilde{s}_{1},\ldots,\tilde{s}_{8}$ having no common invariant essential curves on any of the glued fundamental domains $D_{n}$ is a residual property of $Q\in\mathcal{V}$.

Proof.

Suppose $\Phi,\tilde{s}_{1},\ldots,\tilde{s}_{8}$ have a common invariant essential curve $C\subset D$, where $D$ is the glued fundamental domain in $A_{*}$. We use on $D$ the coordinates $(\varphi,y)$ from Lemma 10, where for convenience we have dropped the ‘hat’ notation. Define

[TABLE]

By Lemma 12, the inner map $\Phi:D\to D$ is a twist map. Therefore we may apply Birkhoff’s theorem: any invariant essential curve of $\Phi$ on $D$ is the graph of a Lipschitz function. It follows that there is a function $y^{\prime}:\mathbb{T}_{*}\to\mathbb{R}$ and $L^{\prime}>0$ such that

[TABLE]

and

[TABLE]

Furthermore, we drop the tilde notation, and denote by $s_{j}$ the modified scattering maps on $D$.

Fix some $L>0$ that bounds from above the Lipschitz constants of all essential invariant curves of the modified scattering maps in $D$, and all maps on $D$ in some $C^{1}$ neighbourhood of the modified scattering maps. Notice that the property of the 8 modified scattering maps having no $L$-Lipschitz common invariant curves in $D$ is an open property of the function $Q$ that defines the hypersurface. We show that by arbitrarily small perturbations of the hypersurface we can destroy all $L$-Lipschitz common invariant curves of the modified scattering maps in $D$ whenever $\tau_{*}>0$ is small enough.

Consider the space of all $L$-Lipschitz functions $y^{\prime}:\mathbb{T}_{*}\to\mathbb{R}$ equipped with the $C^{0}$-topology. Let $\mathcal{L}$ denote the subset consisting of all such functions whose graph is contained in $D$ and is invariant under each $s_{j}$. Clearly $\mathcal{L}$ is compact. If $\mathcal{L}=\emptyset$ then there is nothing to prove, so we assume that $\mathcal{L}\neq\emptyset$. Then for any $\mu>0$ there are finitely many $y_{1},\ldots,y_{q}\in\mathcal{L}$ such that for any $y^{\prime}\in\mathcal{L}$ we have

[TABLE]

for some $p\in\{1,\ldots,q\}$. Inclusion (240) is also true for Lipschitz invariant curves of all maps $\hat{s}_{1},\ldots,\hat{s}_{8}$ in some $C^{1}$ neighbourhood of $s_{1},\ldots,s_{8}$.

Let $R$ be some positive constant such that

[TABLE]

for each $j=1,\ldots,8$ and for all maps $\hat{s}_{1},\ldots,\hat{s}_{8}$ in some $C^{1}$ neighbourhood of $s_{1},\ldots,s_{8}$.

Since the modified scattering maps are defined on a cylinder, perturbations must be periodic in the angular component. To this end consider arcs $I_{1,2,3,4}\subsetneq\mathbb{T}_{*}$ such that $I_{1}\cup I_{2}=I_{3}\cup I_{4}=\mathbb{T}_{*}$. Write $I_{kl}=I_{k}\setminus I_{l}$, and assume that $I_{12},I_{34},I_{21},I_{43}$ are mutually disjoint and lie on $\mathbb{T}_{*}$ in this order (see Figure 6). Since each $I_{k}$ is a proper subset of $\mathbb{T}_{*}=\mathbb{R}/(\tau_{*}\mathbb{Z})$ we have $l(I_{k})<\tau_{*}$.

For $y_{C}\in\mathcal{L}$, let $C=\mathrm{graph}(y_{C})$. Then $C$ is an $L$-Lipschitz $s_{j}$-invariant curve for each $j$. Each arc $I_{k}$ defines a subset of the curve $C$: $\hat{I}_{k}=\{(\varphi,y_{C}(\varphi)):\varphi\in I_{k}\}$. Since $C$ is $s_{j}$-invariant, there is an arc $\bar{I}^{j}_{k}\subsetneq\mathbb{T}_{*}$ such that $s_{j}(\hat{I}_{k})=\{(\varphi,y_{C}(\varphi)):\varphi\in\bar{I}^{j}_{k}\}$. Clearly $l(\bar{I}^{j}_{k})<\tau_{*}$. Let

[TABLE]

Then $E<\tau_{*}$, so

[TABLE]

By the above argument, there are finitely many $y_{1},\dots,y_{q}\in\mathcal{L}$ such that every $L$-Lipschitz common invariant curve of $s_{1},\ldots,s_{8}$ in $D$ lies in one of the cylinders $D_{p}$ as defined in (240) corresponding to this value of $\mu$.

Consider one of these cylinders $D_{p}$ around $y_{p}\in\mathcal{L}$, and let $C_{p}=\mathrm{graph}(y_{p})$. By (241), $s_{j}(D_{p}\cap\{\varphi\in I_{k}\})$ is contained in the $(R\mu)$-neighbourhood of $s_{j}(C_{p}\cap\{\varphi\in I_{k}\})$. This curve is a subset of $C_{p}$, since $C_{p}$ is $s_{j}$-invariant, and corresponds to an interval $\bar{I}^{j}_{k}$ where $l(\bar{I}^{j}_{k})\leq E$. Therefore (243) implies that there is an arc $\hat{I}^{(p)}_{jk}\subsetneq\mathbb{T}_{*}$ such that

[TABLE]

The fact that $\hat{I}^{(p)}_{jk}$ is a proper sub-arc of $\mathbb{T}_{*}$ will allow us to define a perturbation however we like on $\hat{I}^{(p)}_{jk}$ and extend it to the rest of $\mathbb{T}_{*}$ without creating monodromy.

We will now construct a two-parameter family of functions $Q_{\epsilon}$ where $\epsilon=(\epsilon_{1},\epsilon_{2})$ and $Q_{0}=Q$. The modified scattering maps depend smoothly on $\epsilon$. Therefore, there are smooth functions $\Psi_{j}$, $Y_{j}$ for each $j=1,\dots,8$ such that $s_{j}(\varphi,y)=(\Psi_{j}(\varphi,y,\epsilon),Y_{j}(\varphi,y,\epsilon))$.

Consider the perturbed function

[TABLE]

where for now we assume that $\mathrm{supp}(\psi_{j})\cap\Gamma\subset V_{j}$. Later, we will approximate $Q_{\epsilon}$ by a real-analytic family. Let $\Gamma_{\epsilon}=\{x\in\mathbb{R}^{d}:Q_{\epsilon}(x)=0\}$. It follows that

[TABLE]

The following lemma provides conditions on the $\epsilon$-derivatives of the modified scattering maps which, if satisfied, ensure the absence of $L$-Lipschitz common invariant curves of $s_{1},\ldots,s_{8}$ in $D_{p}$. The lemma plays the same role as Lemma 5 in [24]. However we have had to improve the estimates, as the assumptions of that lemma do not directly apply in our case.

Lemma 18**.**

Suppose the scattering maps $s_{1},\ldots,s_{8}$ corresponding to the family $\Gamma_{\epsilon}$ satisfy:

1. (i)

For $j=1,2$, and whenever $\Psi_{j}(\varphi,y,0)\in I_{j}$ we have

[TABLE]

and

[TABLE] 2. (ii)

For $j=3,4$, and whenever $\Psi_{j}(\varphi,y,0)\in I_{j}$ we have

[TABLE]

and

[TABLE] 3. (iii)

Moreover for $j=1,2,3,4$ we have

[TABLE]

whenever $\Psi_{j}(\varphi,y,0)\in I_{j}$, and

[TABLE]

whenever $\Psi_{j+4}(\varphi,y,0)\in I_{j}$.

Then the set of parameters $\epsilon=(\epsilon_{1},\epsilon_{2})$ for which the modified scattering maps $s_{1},\ldots,s_{8}$ have an $L$-Lipschitz common invariant curve in $D_{p}$ has Lebesgue measure 0, and so there are arbitrarily small values of $\epsilon$ for which the modified scattering maps have no $L$-Lipschitz common invariant curve in $D_{p}$.

Proof.

On the space of parameters we use the norm $\|\epsilon\|=\max\{|\epsilon_{1}|,|\epsilon_{2}|\}$. Take two values $\epsilon^{*}$ and $\epsilon^{**}$ of $\epsilon$, and suppose the maps $s_{1},\ldots,s_{8}$ have an $L$-Lipschitz common invariant curve $\mathcal{L}^{*}\subset D_{p}$ at $\epsilon^{*}$, and $\mathcal{L}^{**}\subset D_{p}$ at $\epsilon^{**}$. These curves can be written as $\mathcal{L}^{*}=\mathrm{graph}(y^{*})$ and $\mathcal{L}^{**}=\mathrm{graph}(y^{**})$ where $y^{*},\,y^{**}:\mathbb{T}_{*}\to\mathbb{R}$. We claim that

[TABLE]

The lemma follows from (253). Indeed, suppose it is true, and let $\mathcal{E}\subset\mathbb{R}^{2}$ denote the set of all $\epsilon$ for which the scattering maps have an $L$-Lipschitz common invariant curve in $D_{p}$. Let $\mathcal{Y}$ denote the set of all points $y_{0}\in\mathbb{R}$ for which there is an $L$-Lipschitz common invariant curve $\mathcal{L}^{\prime}=\mathrm{graph}(y^{\prime})\subset D_{p}$ of $s_{1},\ldots,s_{8}$ at some value of $\epsilon$ with $y^{\prime}(0)=y_{0}$. By (253), for each $y_{0}\in\mathcal{Y}$ there is exactly one $\epsilon\in\mathcal{E}$ at which the modified scattering maps have an $L$-Lipschitz common invariant curve in $D_{p}$ that intersects the $y$-axis at $y_{0}$. It follows that there is a well-defined bijective map from $\mathcal{Y}$ to $\mathcal{E}$ sending $y_{0}$ to $\epsilon$. Moreover, (253) also implies that this map is $(\tau^{-1}_{*})$-Lipschitz. Since Lipschitz maps do not increase Hausdorff dimension, it follows that $\mathcal{E}$ is a 1-dimensional set in $\mathbb{R}^{2}$, and so it has Lebesgue measure 0.

We must now establish estimate (253). Suppose, for a contradiction, that it is not true. Without loss of generality, we may assume that

[TABLE]

It follows that $\Delta\epsilon\coloneqq\|\epsilon^{*}-\epsilon^{**}\|=\epsilon^{*}_{1}-\epsilon^{**}_{1}>0$. The supposition that (253) is not true implies that

[TABLE]

We have that $\varphi=0$ is in at least one of $I_{3}$ or $I_{4}$ since $\mathbb{T}_{*}=I_{3}\cup I_{4}$. Assume $0\in I_{3}$. We write

[TABLE]

Since $\mathcal{L}^{*}$, $\mathcal{L}^{**}$ are invariant under $s_{1},\ldots,s_{8}$ at $\epsilon^{*}$, $\epsilon^{**}$ respectively, we have $\bar{y}^{*}=y^{*}(\bar{\varphi}^{*})$ and $\bar{y}^{**}=y^{**}(\bar{\varphi}^{**})$. It follows from the mean value theorem and (241), (246), (249), (254), and (255) that there are $\bar{\epsilon}_{1},\tilde{\epsilon}_{1}\in\left(\epsilon_{1}^{**},\epsilon_{1}^{*}\right)$ such that

[TABLE]

and

[TABLE]

These inequalities together with the mean value theorem again, the $L$-Lipschitz property of $y^{**}$, and (251) imply that

[TABLE]

We have assumed that $y^{*}(0)\geq y^{**}(0)$ and we have just shown that there is $\varphi=\bar{\varphi}^{*}\in\mathbb{T}_{*}$ such that $y^{*}(\varphi)<y^{**}(\varphi)$. It follows that

[TABLE]

Let us define what it means for an arc to be positive or negative. Let $I\subset\mathbb{T}_{*}$ be an arc such that $y^{*}(\varphi)=y^{**}(\varphi)$ for $\varphi\in\partial I$. We say that $I$ is:

• •

Positive if $y^{*}(\varphi)>y^{**}(\varphi)$ for all $\varphi\in\mathrm{Int}(I)$; and

• •

Negative if $y^{*}(\varphi)<y^{**}(\varphi)$ for all $\varphi\in\mathrm{Int}(I)$.

We consider a single point in $\mathcal{L}^{*}\cap\mathcal{L}^{**}$ to be both a positive and negative arc. It follows that the only arcs in $\mathbb{T}_{*}$ that are both positive and negative are points in $\mathcal{L}^{*}\cap\mathcal{L}^{**}$. We will show that (255) implies that we can construct a positive arc and a negative arc in $\mathbb{T}_{*}$ which agree along some nontrivial arc. This is absurd, and will complete the proof by contradiction.

We have already shown via (261) that there is at least one positive arc and one negative arc. Let $J\subset\mathbb{T}_{*}$ be a positive arc, and let

[TABLE]

and

[TABLE]

We will show that if $J\subseteq I_{j}$ for $j\in\{1,2\}$ then there is a positive arc $J^{\prime}\subset\mathbb{T}_{*}$ such that $s_{j}(\mathcal{L}^{*}_{J},\epsilon^{*})\subset\mathcal{L}^{*}_{J^{\prime}}$ and

[TABLE]

where

[TABLE]

Write $s_{j}^{*}(\cdot,\cdot)=s_{j}(\cdot,\cdot,\epsilon^{*})$, and $s_{j}^{**}(\cdot,\cdot)=s_{j}(\cdot,\cdot,\epsilon^{**})$. Let $\varphi\in J$, and let $P=(\varphi,y^{*}(\varphi))\in\mathcal{L}^{*}_{J}$ denote the corresponding point in $\mathcal{L}^{*}_{J}$. Write

[TABLE]

Clearly $P^{*}\in\mathcal{L}^{*}$ and $P^{\prime}\in s_{j}^{**}(\mathcal{L}^{*}_{J})$. Since $J$ is positive we have $y^{**}(\varphi)\leq y^{*}(\varphi)$, and so $P$ is either on or above $\mathcal{L}^{**}$. Moreover since $\mathcal{L}^{**}$ is $s_{j}^{**}$-invariant, $P^{\prime}$ must also lie on or above $\mathcal{L}^{**}$, and so

[TABLE]

The mean value theorem together with (246), (247), and expressions (266) implies that there are $\hat{\epsilon}_{1},\epsilon_{1}^{\prime}\in(\epsilon_{1}^{**},\epsilon_{1}^{*})$ such that

[TABLE]

and

[TABLE]

These inequalities combined with (267) and the $L$-Lipschitz property of $y^{**}$ give

[TABLE]

The first inequality of (270) shows that there is a positive arc $J^{\prime}\subset\mathbb{T}_{*}$ such that $s_{j}^{*}(\mathcal{L}^{*}_{J})\subset\mathcal{L}^{*}_{J^{\prime}}$. Let $\tilde{y}^{*}=y^{*}-y^{**}$. Then $\tilde{y}^{*}$ is 0 on the endpoints of $J^{\prime}$ and strictly positive on $\mathrm{Int}(J^{\prime})$. Moreover $\tilde{y}^{*}$ is $2L$-Lipschitz since $y^{*}$ and $y^{**}$ are $L$-Lipschitz. Let $\tilde{s}_{j}:\mathbb{T}_{*}\to\mathbb{T}_{*}$ denote the map $\tilde{s}_{j}(\varphi)=\Psi_{j}(\varphi,y^{*}(\varphi),\epsilon^{*})$. Then $\tilde{s}_{j}(J)$ is compactly embedded in $J^{\prime}$. Let $\varphi_{1},\varphi_{4}$ denote the endpoints of $J^{\prime}$, and $\varphi_{2},\varphi_{3}$ the endpoints of $\tilde{s}_{j}(J)$. It is possible that $\varphi_{2}=\varphi_{3}$. Without loss of generality, we may assume (by translating our $\varphi$-coordinate on $\mathbb{T}_{*}$ if necessary) that $\varphi_{1}<\varphi_{2}\leq\varphi_{3}<\varphi_{4}$. Notice that the first inequality of (270) holds at $\varphi_{2},\varphi_{3}$ since it holds at all points of $\tilde{s}_{j}(J)$. These facts combined with (251) and (265) imply that

[TABLE]

which is the first inequality of (264).

Due to (267) and the second inequality of (270), the curve $\mathcal{L}^{\prime}=s_{j}^{**}(\mathcal{L}^{*}_{J})$ lies above $\mathcal{L}^{**}_{J^{\prime}}$, and strictly below $\mathcal{L}^{*}_{J^{\prime}}$. Since $s^{**}_{j}(\mathcal{D}_{J})$ is the region bounded by $\mathcal{L}^{\prime}$ and $s^{**}_{j}(\mathcal{L}^{**}_{J})$ it follows that

[TABLE]

where the last equality is due to the symplecticity of $s_{j}^{**}$. This is the second inequality of (264).

Now, let $J\subseteq I_{j}$ be a positive arc where $j\in\{1,2\}$, and construct a sequence of positive arcs $J_{s}$ as follows: $J_{0}=J$, and $\tilde{s}_{j_{s}}(J_{s})\subset J_{s+1}$ where $j_{s}=1$ if $J_{s}\subseteq I_{1}$ and $j_{s}=2$ if $J_{s}\subseteq I_{2}$. The sequence terminates if neither $J_{s}\not\subset I_{1}$ nor $J_{s}\not\subset I_{2}$. Due to the second inequality of (264), $\mathrm{Area}(\mathcal{D}_{J_{s}})$ is strictly increasing as $s$ increases, and so $J_{s_{1}}\neq J_{s_{2}}$ if $s_{1}\neq s_{2}$. Since all the arcs $J_{s}$ are positive, it follows that they are mutually disjoint. Therefore, (237) and the first inequality of (264) guarantee that there cannot be more than $\frac{\tau_{*}}{\delta\Delta\epsilon}$ such arcs, and so the sequence must terminate at some positive arc $J_{+}$. Since the positive arc $J_{+}$ is contained neither in $I_{1}$, nor in $I_{2}$, we have $\emptyset\neq J_{+}\cap\left(\mathbb{T}_{*}\setminus I_{2}\right)=J_{+}\cap\left(I_{1}\setminus I_{2}\right)=J_{+}\cap I_{12}$, and $\emptyset\neq J_{+}\cap\left(\mathbb{T}_{*}\setminus I_{1}\right)=J_{+}\cap\left(I_{2}\setminus I_{1}\right)=J_{+}\cap I_{21}$ (see Figure 6).

Applying similar logic to negative arcs in the intervals $I_{3}$ and $I_{4}$, we construct a negative arc $J_{-}$ such that $J_{-}\cap I_{34}\neq\emptyset$ and $J_{-}\cap I_{43}\neq\emptyset$. But since $I_{12},I_{34},I_{21},I_{43}$ are located in this order on $\mathbb{T}_{*}$, this means that $\mathrm{Int}(J_{+})\cap\mathrm{Int}(J_{-})\neq\emptyset$, which is impossible since one is positive and the other is negative. This contradiction implies (253), and completes the proof of the lemma. ∎

Now, suppose we make a perturbation as in (245) where each $\psi_{j}$ is supported near $\pi\circ f\left(\tilde{B}_{*,j}\right)\subset V_{j}$ (where $V_{j}$ are the neighbourhoods described at the beginning of the section). We will show now that we can choose $\psi_{1},\ldots,\psi_{8}$ so that the modified scattering maps satisfy the assumptions of Lemma 18.

Suppose $j\in\{1,2,3,4\}$, and $(\varphi,y)\in D_{p}$ such that $\Psi_{j}(\varphi,y,0)\in I_{j}$ where $s_{j}=(\Psi_{j},Y_{j})$ are the modified scattering maps. Let

[TABLE]

By (217) and (218) we have

[TABLE]

and

[TABLE]

Assume $\psi_{j}>0$. Using these expressions, we see that conditions (247) and (249) are of the form:

[TABLE]

for $j=1,2$ whenever $\Psi_{j}(\varphi,y,0)\in I_{j}$; and

[TABLE]

for $j=3,4$ whenever $\Psi_{j}(\varphi,y,0)\in I_{j}$. Moreover condition (251) is

[TABLE]

Fix some point in $f\left(\tilde{B}_{*,j}\right)$ and denote by $w^{*}$ its $w$-component. Notice that for all $(\bar{x},\bar{w},\bar{z})\in f\left(\tilde{B}_{*,j}\right)$ we have

[TABLE]

where the terms of order $\tau_{*}$ are uniformly bounded. For constants $C_{j,1}>0$ and $C_{j,2}\in\mathbb{R}$ which are yet to be determined, let

[TABLE]

whenever $\bar{x}\in\pi\circ f\left(\tilde{B}_{*,j}\right)$ corresponds via $f\circ(\pi^{u}_{j})^{-1}\circ f^{n}$ to some $(\varphi,y)\in D$ satisfying $\Psi_{j}(\varphi,y,0)\in I_{j}$. Moreover $\psi_{j}$ is 0 outside $V_{j}$, and $\psi_{j}$ is $C^{\infty}$-smooth. For $j=1,2$ let

[TABLE]

where $T^{1}\Gamma$ is the subset of the tangent bundle consisting of vectors of norm 1. Since $\psi_{j}>0$, we may divide both sides of (281) by $\psi_{j}(\bar{x})$ to see that it is equivalent to

[TABLE]

which is true for sufficiently small $\tau_{*}$. Since the exponential term is nonzero, we may choose $C_{j,1}>0$ large enough so that (283) is satisfied. For $j=3,4$, let

[TABLE]

Then a similar computation shows that $\psi_{j}$ satisfies (282) and (283) for sufficiently small $\tau_{*}$. Similar computations show that similar functions satisfy conditions (248), (250), and, (252).

Lemma 18 thus implies that the modified scattering maps $s_{1},\ldots,s_{8}$ corresponding to this perturbed hypersurface $\Gamma_{\epsilon}$ have no $L$-Lipschitz common invariant curves in the cylinder $D_{p}$ for arbitrarily small values of $\epsilon$. Approximating the perturbed function $Q_{\epsilon}$ sufficiently well by a real-analytic family $\tilde{Q}_{\epsilon}\subset\mathcal{V}$, we can guarantee that for arbitrarily small values of $\epsilon$, the conditions of Lemma 18 are still satisfied for the analytic family. Now, apply this process to clear the $L$-Lipschitz common invariant curves in $D_{1}$. Since the property of the scattering maps having no invariant curves in $D_{1}$ is open, we may repeat the process with sufficiently small values of $\epsilon$ to kill the invariant curves in $D_{2}$ without creating new invariant curves in $D_{1}$. Repeating this process $q$ times, we can simultaneously clear the subcylinders $D_{1},\ldots,D_{q}$ of invariant curves. Since all $L$-Lipschitz common invariant curves of $s_{1},\ldots,s_{8}$ in $D$ were contained in these subcylinders, we have therefore cleared the cylinder $D$ of common invariant curves of the modified scattering maps. This in turn implies that the IFS on the glued fundamental domain $D$ itself has no invariant essential curves for some $\tilde{Q}$ arbitrarily close to $Q$ in $\mathcal{V}$.

We have shown that by making arbitrarily small analytic perturbations, we can destroy the invariant essential curves on the fundamental domain $D$ for all sufficiently small $\tau_{*}$. This implies that the property of the IFS having no invariant essential curves on $D$ is dense in $\mathcal{V}$ for all sufficiently small $\tau_{*}$. As mentioned earlier, it is also open. Choosing the sequence of cylinders $\{A_{n}\}_{n\in\mathbb{N}}$ so that the corresponding sequence $\{\tau_{n}\}_{n\in\mathbb{N}}$ of constants are all sufficiently small, we therefore obtain for each $n\in\mathbb{N}$ an open dense set $\tilde{\mathcal{V}}_{n}\subset\mathcal{V}$ with the property that for all $Q\in\tilde{\mathcal{V}}_{n}$, the corresponding IFS $\{\Phi,s_{1},\dots,s_{8}\}$ has no invariant essential curves on the cylinder $D_{n}$. It follows that

[TABLE]

is the residual set we are looking for.

∎

Appendix. Exact Symplecticity of the Modified Scattering Maps

In this appendix, we prove Lemma 11, which for convenience is restated here.

Lemma**.**

If we consider $D$ as a cylinder by identifying points on the line $\{\varphi=0\}$ with their image under $f$, and use the coordinates $(\varphi,y)\in[0,\tau_{*})\times\left[a,b\right]$, then the modified scattering map $\tilde{s}:D\to\tilde{s}(D)\subset D^{\prime}$ is an exact symplectic $C^{r}$-diffeomorphism onto its image, and its image contains an essential curve in $D^{\prime}$.

Proof.

Let $\Lambda=\mathbb{T}\times\left[E_{-},E_{+}\right]$, where $\mathbb{T}=\mathbb{R}/\mathbb{Z}$ and where $E_{\pm}\in\mathbb{R}$ are such that $E_{-}<E_{+}$. Recall the mapping torus of the billiard map $f$ on $M$ is obtained by taking the direct product of $M$ with the closed unit interval $[0,1]$ and identifying points $(z,1)\in M\times[0,1]$ with points $(f(z),0)\in M\times[0,1]$. The resulting set is a fibre bundle, with base space $\mathbb{T}$, and fibres $M$. Denote by $\widehat{M}$ the product of the mapping torus of $f$ on $M$ with the interval $\left[E_{-},E_{+}\right]$, so $\widehat{M}$ is itself a fibre bundle, with base space $\Lambda$, and fibres $M$. Note that we do not claim that $\widehat{M}$ is symplectomorphic to a direct product of $M$ and $\Lambda$. Notice, however, that it is an exact symplectic manifold. Indeed, recall from Section 2.2 that the billiard map is exact symplectic, in the sense that $f^{*}\lambda-\lambda=-d\tau$, where $\lambda$ is the Liouville 1-form. Let $\zeta(t)$ be a smooth function that is identically 0 near $t=0$ and identically 1 near $t=1$, and define $\hat{\lambda}=\lambda-E\,dt-d\left(\zeta(t)\,\tau\right)$. Then $\hat{\lambda}$ is a Liouville 1-form on $\widehat{M}$, and

[TABLE]

is a symplectic form on $\widehat{M}$, where $\omega$ is the symplectic form on $M$. We can thus define a Hamiltonian function $K:\widehat{M}\to\mathbb{R}$ by $K=E$. This Hamiltonian has the following properties:

• •

The Hamiltonian vector field of $K$ with respect to the symplectic form $\hat{\omega}$ is such that

[TABLE]

where the coordinates $(t,E)$ on $\Lambda$ represent time and energy respectively.

• •

If we fix $E_{0}\in[E_{-},E_{+}]$ and define $\widehat{M}_{0}$ to be the fibre at $t=0,\,E=E_{0}$, then the return map $\hat{f}:\widehat{M}_{0}\to\widehat{M}_{0}$ coincides with the billiard map $f$ on $M$, under the obvious identification of $\widehat{M}_{0}$ with $M$.

Moreover, the set $\widehat{A}$ – defined as the fibre bundle with base space $\Lambda$ and fibres $A_{*}$ – is a normally hyperbolic invariant manifold in $\widehat{M}$ for the Hamiltonian flow $\phi^{t}_{K}$ of $K$, and the set $\widehat{B}$ – defined as the fibre bundle with base space $\Lambda$ and fibres being the homoclinic manifold $B_{*}$ of $f$ in $M$ – is a homoclinic manifold in the transverse homoclinic intersection of $W^{s,u}\left(\widehat{A}\right)$. Denote by $\hat{\pi}^{s,u}:\widehat{B}\to\widehat{A}$ the holonomy maps corresponding to the Hamiltonian flow $\phi^{t}_{K}$.

Let $\Delta=\left(\pi^{u}_{B_{*}}\right)^{-1}(D)$ be a preimage of $D$ under the unstable holonomy map. For positive integers $N_{\pm}$ that we will fix later, let

[TABLE]

and define

[TABLE]

Then the modified scattering map is

[TABLE]

In what follows we rephrase (295) using Poincaré maps of the Hamiltonian flow $\phi^{t}_{K}$, and we use the fact that these return maps are exact symplectic diffeomorphisms of topological cylinders.

Denote by $C_{0}$ the curve $\{\varphi=0\}\subset A_{*}$ in $M$, so that the fundamental domain $D$ is bounded by $C_{0}$ and $f(C_{0})$. It follows that the set $\Delta$ is a fundamental domain of $f$ in $B_{*}$, bounded by

[TABLE]

and $f\left(\widetilde{C}_{0}\right)$. Moreover the sets $\Delta_{\pm}$ are fundamental domains of $f$, in the connected component of $W^{s}(A_{*})\cap W^{u}(A_{*})$ containing $B_{*}$, bounded by

[TABLE]

and $f\left(C_{0}^{\pm}\right)$ respectively.

Fix some $E_{0}\in[E_{-},E_{+}]$, and define the sets:

• •

$\widehat{D}\subset\widehat{A}$ is the set with base points $\{(t,E)\in\Lambda:E=E_{0}\}$ with the points of $C_{0}$ in each fibre;

• •

$\widehat{\Delta}_{\pm}\subset W^{s}\left(\widehat{A}\right)\cap W^{u}\left(\widehat{A}\right)$ is the set with base points $\{(t,E)\in\Lambda:E=E_{0}\}$ with the points of $C_{0}^{\pm}$, respectively, in each fibre;

• •

$\widehat{D}_{+}=\hat{\pi}^{s}\left(\widehat{\Delta}_{+}\right)\subset\widehat{A},\quad\widehat{D}_{-}=\hat{\pi}^{u}\left(\widehat{\Delta}_{-}\right)\subset\widehat{A}.$

Write

[TABLE]

and define the following return maps:

• •

$T_{0}:\{E=E_{0}\}\to\widehat{M}_{0}$ is the return map of $\phi^{t}_{K}$ forward in time to $\{t=0,\,E=E_{0}\}$: i.e. if $z_{0}\in\{E=E_{0}\}$ then there is a unique $t_{0}\in[0,1)$ such that $T_{0}(z_{0})=\phi^{t_{0}}_{K}(z_{0})\in\widehat{M}_{0}$;

• •

$T_{1}:\widehat{\Delta}_{-}\to\widehat{\Delta}_{+}$ is the return map of $\phi^{t}_{K}$ to $\widehat{\Delta}_{+}$.

• •

$T_{+}:\widehat{A}\cap\{E=E_{0}\}\to\widehat{D}$ is the return map of $\phi^{t}_{K}$ to $\widehat{D}$.

Finally, define the map $\hat{s}:\widehat{D}\to\hat{s}\left(\widehat{D}\right)$ by

[TABLE]

Observe that $\widehat{D}$, $\widehat{D}_{\pm}$ and $\widehat{\Delta}_{\pm}$ are topological cylinders in $\widehat{M}$. We will show:

1. (i)

$\hat{s}:\widehat{D}\to\hat{s}\left(\widehat{D}\right)$ is an exact symplectic diffeomorphism onto its image $\hat{s}\left(\widehat{D}\right)$, which contains an essential curve; 2. (ii)

If we glue the boundaries of $D$ by identifying points $x$ on one boundary with points $f(x)$ on the other then

[TABLE]

is an exact symplectic diffeomorphism. 3. (iii)

The modified scattering map $\tilde{s}$ can be written $\tilde{s}=T_{0}\circ\hat{s}\circ\left(\left.T_{0}\right|_{\widehat{D}}\right)^{-1}$.

These three facts together complete the proof of the lemma.

In order to prove (i), notice first that the time-shifts $\phi_{K}^{-N_{\pm}}$ and the return maps $T_{1},T_{+}$ are exact symplectic diffeomorphisms between cylinders since $\phi^{t}_{K}$ is a Hamiltonian flow. We must show that $\hat{\pi}_{\pm}$ are exact symplectic diffeomorphisms between cylinders. We pass to Fenichel coordinates $((q,p),(t,E),\alpha,\beta)$ in a neighbourhood $\mathcal{U}$ of $\widehat{A}$ (see the proof of Lemma 9 for a more complete description of Fenichel coordinates). Here $((q,p),(t,E))$ are coordinates on $\widehat{A}$ where $(q,p)$ are coordinates on $A_{*}$ and $(t,E)\in\Lambda$, and $\alpha,\,\beta$ are coordinates in the hyperbolic directions. In particular, if $z\in\widehat{A}$ then the corresponding leaves of the strong stable and strong unstable foliations are given by

[TABLE]

The local stable manifold $W^{s}_{loc}\left(\widehat{A}\right)$ (respectively local unstable manifold $W^{u}_{loc}\left(\widehat{A}\right)$) is the set of points whose forward orbits (resp. barckward orbits) tend toward $\widehat{A}$ without ever leaving the neighbourhood $\mathcal{U}$ of $\widehat{A}$. Notice that the restriction

[TABLE]

of the stable and unstable holonomy maps to the local stable and unstable manifolds, in Fenichel coordinates, are simply projections onto the $((q,p),(t,E))$ variables. Now choose $N_{\pm}$ sufficiently large so that

[TABLE]

It follows that the maps $\hat{\pi}_{\pm}$ are just the identity mapping in Fenichel coordinates, and therefore are exact symplectic diffeomorphisms between cylinders. We have thus proved (i).

The proof of (ii) is immediate from the facts that the return map of a Hamiltonian flow is exact symplectic, that $D$ is a fundamental domain of $f$, and that the return map $\hat{f}:\widehat{M}_{0}\to\widehat{M}_{0}$ coincides with the billiard map.

Finally, let us prove (iii). In this part of the proof, we consider the billiard map $f$ and holonomy maps $\pi_{\pm}$ as maps on (subsets of) $\widehat{M}_{0}$ in the natural way without changing notation. From the definitions we have

[TABLE]

for each $m\in\mathbb{Z}$. We prove

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Combining (306), (307), (308), (309), (310) with (295) and (299) yields (iii) and completes the proof of the lemma.

Let us prove (306). Let $z_{0}\in D$. Then there is a unique $t_{0}\in\mathbb{T}$ such that

[TABLE]

Then, since $N_{-}\in\mathbb{Z}$, we have

[TABLE]

Next, we prove (307). We prove the formula for $\pi_{+}$ as the formula for $\pi_{-}$ is analogous. Let $z_{1}\in\Delta_{+}$. Then there is a unique $t_{1}\in\mathbb{T}$ such that

[TABLE]

Since $\Delta_{+}\subset W^{s}_{loc}(A_{*})$, $z_{1}$ is of the form

[TABLE]

in Fenichel coordinates, where $(q,p)$ denote coordinates on the cylinder $A_{*}$. Therefore $\pi_{+}(z_{1},0,E_{0})=((q_{1},p_{1}),(0,E_{0}))$. Now,

[TABLE]

It thus follows that

[TABLE]

which is (307).

Notice that (308) and (309) are analogous to (306), so it remains to prove (310). Let $z_{2}\in s_{B_{*}}(D)$, and let $n_{2}=n(z_{2})\in\mathbb{N}$. Define

[TABLE]

and observe that

[TABLE]

Therefore, since $T_{+}$ is a return map, it can be seen that

[TABLE]

at points of the type (322). Now, there is a unique $t_{2}\in\mathbb{T}$ such that

[TABLE]

Using (323), it follows that

[TABLE]

∎