GIT Stability of Henon Maps

TL;DR

This paper analyzes the geometric invariant theory (GIT) stability of generalized Henon maps in projective space, revealing conditions under which these maps are unstable, semistable, or not stable, and classifying unstable cases.

Contribution

It provides a comprehensive GIT stability classification of Henon maps in various dimensions and degrees, including a full classification of unstable maps in the case of degree 2 and dimension 2.

Findings

Henon maps are GIT unstable if N≥3 or d≥3.

They are semistable but not stable when N=d=2.

Complete classification of unstable maps in Rat_2^2.

Abstract

In this paper we study the locus of generalized degree $d$ Henon maps in the parameter space $\operatorname{Rat}_d^N$ of degree $d$ rational maps $\mathbb{P}^N\to\mathbb{P}^N$ modulo the conjugation action of $\operatorname{SL}_{N+1}$. We show that Henon maps are in the GIT unstable locus if $N\ge3$ or $d\ge3$, and that they are semistable, but not stable, in the remaining case of $N=d=2$. We also give a general classification of all unstable maps in $\operatorname{Rat}_2^2$.

Tables

Table 1. Table 1. Exponents of α 𝛼 \alpha used in computing μ ​ ( f ¯ 𝑷 , L r , s , t ) 𝜇 subscript ¯ 𝑓 𝑷 subscript 𝐿 𝑟 𝑠 𝑡 \mu({\bar{f}}_{\boldsymbol{P}},L_{r,s,t})

Table 2. Table 2. The exponent of α 𝛼 \alpha on each monomial of F L := L ∘ F ∘ L − 1 assign superscript 𝐹 𝐿 𝐿 𝐹 superscript 𝐿 1 F^{L}:=L\circ F\circ L^{-1}

Table 3. Table 3. The exponent of α 𝛼 \alpha on each monomial of F L superscript 𝐹 𝐿 F^{L} assuming 2 ​ r ≥ s > r > 0 2 𝑟 𝑠 𝑟 0 2r\geq s>r>0

Equations

$$f_{P}(x,y)=\bigl{(}ay,bx+P(y)\bigr{)}\quad\text{with $a,b\in K^{*}$, $P(y)\in K[y]$, $\deg P=d\geq 2$.}$$

$$f_{P}(x_{1},\ldots,x_{N})=\bigl{(}b_{2}x_{2},b_{3}x_{3},\ldots,b_{N}x_{N},b_{1}x_{1}+P(x_{N})\bigr{)}$$

$$P_{i+1}\in K[x_k,\dots ,x_i],k\le i\le N,\underset{k\le i\le N}{\max }\mathrm{deg}{P}_{i+1}=d.$$

$$f_{\boldsymbol{P}}:=\Bigl{(}\overbrace{b_{2}x_{2},\ldots,b_{k}x_{k}}^{\text{$k-1$ terms}},\overbrace{b_{k+1}x_{k+1}+P_{k+1}({\boldsymbol{x}}),\ldots,b_{N}x_{N}+P_{N}({\boldsymbol{x}})}^{\text{$N-k$ terms}},b_{1}x_{1}+P_{N+1}({\boldsymbol{x}})\Bigr{)}.$$

$${\bar{f}}_{\mathbf{P}}:{\mathbb{P}}^N⟶{\mathbb{P}}^N.$$

$$f:{\mathbb{A}}^1\setminus 0⟶\{\text{degree d morpshims PN→PN}\},$$

$${\mathbb{A}}^1(K)⟶{\overline{\mathcal{M}}}_d^N(K)\text{given for t=0 by}t⟼⟨f_t⟩.$$

$${\bar{f}}_{d,t}(x,y,z)=[t{x}^d+y{z}^{d-1},x{z}^{d-1}+y^d,z^d]:{\mathbb{P}}^2⟶{\mathbb{P}}^2.$$

$$\underset{t\to 0}{\lim }⟨f_t⟩=⟨f_0⟩,$$

$$f=\left[\sum a_1(\mathbf{i}){\mathbf{X}}^{\mathbf{i}}:\sum a_2(\mathbf{i}){\mathbf{X}}^{\mathbf{i}}:\cdots :\sum a_{N+1}(\mathbf{i}){\mathbf{X}}^{\mathbf{i}}\right]\in {\mathrm{Rat}}_d^N$$

$$f^L=\left[\sum {\alpha }^{e_1(\mathbf{i},L)}{a}_1(\mathbf{i}){\mathbf{X}}^{\mathbf{i}}:\sum {\alpha }^{e_2(\mathbf{i},L)}{a}_2(\mathbf{i}){\mathbf{X}}^{\mathbf{i}}:\cdots :\sum {\alpha }^{e_{N+1}(\mathbf{i},L)}{a}_{N+1}(\mathbf{i}){\mathbf{X}}^{\mathbf{i}}\right].$$

$$\mu (f,L):=\underset{\begin{array}{c}(j,\mathbf{i})\\ a_j(\mathbf{i})\ne 0\end{array}}{\min }{e}_j(\mathbf{i},L).$$

$${\bar{f}}_{\mathbf{P}}:{\mathbb{P}}^N⟶{\mathbb{P}}^N.$$

$${\bar{f}}_{\boldsymbol{P}}=\smash[b]{\Bigl{[}}\overbrace{b_{2}x_{2}x_{N+1}^{d-1}:\cdots:b_{k}x_{k}x_{N+1}^{d-1}}^{\text{coordinates $1,2,\ldots,k-1$}}:\\ \overbrace{b_{k+1}x_{k+1}x_{N+1}^{d-1}+{\bar{P}}_{k+1}({\boldsymbol{x}}):\cdots:b_{N}x_{N}x_{N+1}^{d-1}+{\bar{P}}_{N}({\boldsymbol{x}})}^{\text{coordinates $k,k+1,\ldots,N-1$}}:\\ \underbrace{b_{1}x_{1}x_{N+1}^{d-1}+P_{N+1}({\boldsymbol{x}})}_{\text{coordinate $N$}}:\underbrace{x_{N+1}^{d}}_{\text{coordinate $N+1$}}\Bigr{]}.$$

$$r(k-1)-s(N-k+1)-t=0,r,s,t\ge 0,$$

$$L=L_{r,s,t}:=\left(\begin{array}{ccc}{\alpha }^r{I}_{k-1}& 0& 0\\ 0& {\alpha }^{-s}{I}_{N-k+1}& 0\\ 0& 0& {\alpha }^{-t}\end{array}\right).$$

$$L_{r,s,t}(x_i)=\{\begin{array}{ll}{\alpha }^r{x}_i& \text{for 1≤i≤k−1,}\\ {\alpha }^{-s}{x}_i& \text{for k≤i≤N,}\\ {\alpha }^{-t}{x}_{N+1}& \text{for k=N+1.}\end{array}$$

$${\bar{f}}_{\mathbf{P}}^L:=L_{r,s,t}\circ {\bar{f}}_{\mathbf{P}}\circ L_{r,s,t}^{-1}.$$

$$x_k^{e_k}\cdots x_i^{e_i}{x}_{N+1}^{d-e_k-\cdots -e_i}.$$

$${\alpha }^{(e_k+\cdots +e_i-1)s+(d-e_k-\cdots -e_i)t}.$$

$$s=k-1,t=(k-1)(N+1),r=2N+2-k,$$

$$dt-s=(k-1)\bigl{(}d(N+1)-1\bigr{)}>0.$$

$$-r-s+(d-1)t=\bigl{(}(d-1)(k-1)-2\bigr{)}(N+1)+1>0.$$

$$\text{k≥3 or d≥3}⟹\mu ({\bar{f}}_{\mathbf{P}},L_{r,s,t})>0,$$

$$r=1,s=0,t=1,$$

$$1,0,1,2-m\text{for 0≤m≤2},0,1.$$

$$\mu ({\bar{f}}_{\mathbf{P}},L_{1,0,1})=\min \{0,1,2\}=0,$$

$$\begin{array}{ccc}{\mathbb{P}}^2& \stackrel{F}{\to }& {\mathbb{P}}^2\\ \pi \downarrow & & \downarrow \pi \\ {\mathbb{P}}^1& \stackrel{G}{\to }& {\mathbb{P}}^1\end{array}$$

$$L=L_{r,s}(\alpha ):=\left(\begin{array}{ccc}{\alpha }^r& 0& 0\\ 0& {\alpha }^{s-r}& 0\\ 0& 0& {\alpha }^{-s}\end{array}\right)\text{with r≥s−r≥−s and r,s>0.}$$

$$\begin{array}[]{|c||c|c|c|c|c|c|}\hline\cr&x^{2}&xy&xz\\ \hline\cr\hline\cr\text{$y$-coordinate of $F^{L}$}&-3r+s&-r&-2r+2s\\ \hline\cr\text{$z$-coordinate of $F^{L}$}&-2r-s&-2s&-r\\ \hline\cr\end{array}$$

Full text

The first author is supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education. (NRF-2016R1D1A1B01009208)

1. Introduction

When extending the theory of the dynamics of endomorphisms of $\mathbb{P}^{2}$ to more general rational maps, a standard collection of maps to study are Hénon maps. These maps were originally studied in dimension $2$, where they have the form

[TABLE]

Hénon maps were the first maps shown to exhibit strange attractors in their real dynamics. They have also received considerable attention in the study of arithmetic and algebraic dynamics, since they share many properties with endomorphisms, including the height-boundeness of the set of preperiodic points [2, 9], the existence of a canonical height [3, 4, 5, 7, 13], and equidistribution of periodic points [6].

Various authors have considered generalizations of classical Hénon maps to dimension greater than $2$. For example, dynamics over $\mathbb{R}$ for maps of the form

[TABLE]

have attracted attention; see for exampe [1, 12, 15]. These papers concentrate mainly on real dynamics and quadratic maps. Our aim in this paper is to study stability properties, in the sense of geometric invariant theory, of quite general Hénon type maps, as given in the following definition.

Definition 1.1**.**

Let $K$ be an algebraically closed field of characteristic [math], let $N\geq{k}\geq{2}$ and $d\geq 2$, and fix scalars $b_{1},\ldots,b_{N}\in{K^{*}}$ and polynomials

[TABLE]

The associated generalized Hénon map is the affine automorphism $f_{\boldsymbol{P}}:\mathbb{A}^{N}\to\mathbb{A}^{N}$ defined by

[TABLE]

We denote the extension of $f_{\boldsymbol{P}}$ to a birational map of $\mathbb{P}^{N}$ by

[TABLE]

The set of Hénon maps sits inside the parameter space $\operatorname{Rat}_{d}^{N}$ of degree $d$ rational maps $\mathbb{P}^{N}\to\mathbb{P}^{N}$. In studying dynamics, one looks at the quotient of this space by the natural conjugation action of $\operatorname{SL}_{N+1}$ on the maps parameterized by $\operatorname{Rat}_{d}^{N}$. Geometric invariant theory (GIT) provides subsets of stable and semistable points in $\operatorname{Rat}_{d}^{n}$ for which the $\operatorname{SL}_{N+1}$-quotient has a nice structure. For details of this standard construction, see for example [14]. It is known [8, 11] that all endomorphisms of $\mathbb{P}^{N}$ are in the stable locus, but the full structure of the stable and semistable loci is complicated. It thus seems to be of interest to determine whether Hénon maps, which are such a good testing ground for many dynamical problems, are in these loci. Our main theorem provides an answer to this question.

Theorem 1.2**.**

Let $N\geq{k}\geq 2$ and $d\geq 2$, and let ${\bar{f}}_{\boldsymbol{P}}:\mathbb{P}^{N}\to\mathbb{P}^{N}$ be a degree $d$ Hénon map defined over an algebraically closed field of characteristic [math] as described in Definition 1.1.

• (a)

If $d\geq 3$ or $k\geq 3$, then ${\bar{f}}_{\boldsymbol{P}}$ is in the $\operatorname{SL}_{N+1}$-unstable locus of $\operatorname{Rat}_{d}^{N}$.

• (b)

If $d=k=2$, then ${\bar{f}}_{\boldsymbol{P}}$ is not in the $\operatorname{SL}_{N+1}$-stable locus of $\operatorname{Rat}_{d}^{N}$.

• (c)

If $d=k=N=2$, then ${\bar{f}}_{\boldsymbol{P}}$ is in the $\operatorname{SL}_{N+1}$-semistable locus of $\operatorname{Rat}_{d}^{N}$.

Remark 1.3**.**

We briefly discuss why one might be interested in the stability properties of Hénon maps. Let $K$ be an algebraically closed field. GIT says that the stable and semistable loci $(\operatorname{Rat}_{d}^{N})^{\textup{stab}}$ and $(\operatorname{Rat}_{d}^{N})^{\textup{ss}}$ have $\operatorname{SL}_{N+1}$-quotients that are irreducible algebraic varieties. In particular, the stable quotient $\mathcal{M}_{d}^{N}:=(\operatorname{Rat}_{d}^{N})^{\textup{stab}}//\operatorname{SL}_{N+1}$ has the property that two points $f_{1},f_{2}\in(\operatorname{Rat}_{d}^{N})^{\textup{stab}}(K)$ have the same image in $\mathcal{M}_{d}^{N}(K)$ if and only if there is some $\phi\in\operatorname{SL}_{N+1}(K)$ such that $f_{2}=\phi\circ{f_{1}}\circ\phi^{-1}$. For the semistable quotient $\overline{\mathcal{M}}_{d}^{N}:=(\operatorname{Rat}_{d}^{N})^{\textup{ss}}//\operatorname{SL}_{N+1}$, two points $f_{1},f_{2}\in(\operatorname{Rat}_{d}^{N})^{\textup{ss}}(K)$ have the same image in $\overline{\mathcal{M}}_{d}^{N}(K)$ if and only if the Zariski closure of their orbits $\operatorname{SL}_{N+1}(K)f_{1}$ and $\operatorname{SL}_{N+1}(K)f_{2}$ has a point in common, which is somewhat less satisfactory, but is balanced by the fact that $\overline{\mathcal{M}}_{d}^{N}$ is proper of $K$. So for example, if we are given a $1$-parameter family of endomorphisms

[TABLE]

then the induced map to $\overline{\mathcal{M}}_{d}^{N}$ fills in with an equivalence class of semistable maps at [math], i.e., there is a morphism

[TABLE]

It is thus of interest to understand the semistable maps in $\operatorname{Rat}_{d}^{N}$, since they are the maps that occur as the natural limiting values in families of maps.

For example, let $d\geq 2$, and $t\in\mathbb{A}^{1}(K)$, consider the family of maps

[TABLE]

For $t\neq 0$, the map ${\bar{f}}_{d,t}$ is a morphism, so it is in $(\operatorname{Rat}_{d}^{2})^{\textup{stab}}(K)$, and as $t\to 0$, the points $\langle{f_{t}}\rangle\in\overline{\mathcal{M}}_{d}^{2}(K)$ have a limiting value consisting of the $\operatorname{SL}_{3}(K)$-orbits of various semistable maps in $(\operatorname{Rat}_{d}^{2})^{\textup{ss}}(K)$. Theorem 1.2 says that if $d=2$, then

[TABLE]

but that if $d\geq 3$, then the Hénon map $f_{0}$ is not a limiting value of $\langle{f_{t}}\rangle$. Roughly speaking, this says that for $d=2$, one can study the Hénon map $f_{0}$ as a natural degeneration of the family (4) of morphisms $f_{t}$, but that this is not the case for $d\geq 3$.

We conclude this introduction with a summary of the steps that go into proving Theorem 1.2. We start in Section 2 by setting notation and stating the Hilbert-Mumford numerical criterion for stability [10]. We use this criterion in Section 3 to show that Hénon maps are unstable if $d\geq 3$ or $k\geq 3$, and that they are not stable if $d=k=2$. This leaves the problem of determining if Hénon maps are semistable or unstable when $d=k=2$. Since we do not know how to do this directly, we follow a different path in the case $N=d=k=2$. In Section 4 we show that all unstable maps in $\operatorname{Rat}_{2}^{2}$ are either algebraically unstable or are linearly fibered over $\mathbb{P}^{1}$, and we then show that this precludes their being Hénon maps. Thus the proof of Theorem 1.2(c) follows the Shelock Holmes method: “Once you eliminate the impossible, whatever remains…must be the truth.”

2. The Hilbert–Mumford numerical criterion for stability

In this section, we review and set notation for the Hilbert–Mumford numerical criterion on projective spaces. For a $1$-parameter subgroup ($1$-PS) $L:\mathbb{G}_{m}\to\operatorname{SL}_{N+1}\to\operatorname{Aut}(\mathbb{P}^{N})$, we let $L$ act by conjugation on a rational map

[TABLE]

via $f^{L}:=L(\alpha)\circ{f}\circ{L(\alpha)^{-1}}$. (Here ${\boldsymbol{X}}:=[x_{1}:\cdots:x_{N+1}]$, and we sum over multi-indices ${\boldsymbol{i}}:=(i_{1},\ldots,i_{N+1})$.) After a change of coordinates, we may take $L$ to be diagonal, i.e., the matrix of $L(\alpha)$ is diagonal with entries $\alpha^{r_{1}},\ldots,\alpha^{r_{N+1}}$ for some integers $r_{i}$ that sum to [math]. Then the action takes the form

[TABLE]

The Hilbert–Mumford numerical criterion for stability is then defined in terms of the following numerical invariant:111This is the negative of the $\mu$ that typically appears in the literature, but we find that this version is easier to work with.

[TABLE]

Theorem 2.1** (Numerical criterion, [10]).**

Let $f\in\operatorname{Rat}_{d}^{N}$. Then222More formally, $\operatorname{Rat}_{d}^{N}$ is the complement of a hypersurface in a large projective space $\mathbb{P}^{M}$, and stability is always relative to the ample line bundle $\mathcal{O}_{\mathbb{P}^{M}}(1)$, so we omit it from the notation.

• (a)

$\mu(f,L)>0$* for some $1$-PS $L$  $\Longleftrightarrow$  $f$ is unstable.*

• (b)

$\mu(f,L)\geq 0$* for some $1$-PS $L$  $\Longleftrightarrow$  $f$ is not stable.*

The Hilbert–Mumford criterion suggests if $f$ has a small number of monomials, then $f$ has a higher chance of being unstable. Thus Hénon maps, especially those of high degree or dimension, are likely to be unstable. Theorem 1.2 confirms this intuition.

3. Instability of Hénon maps

In this section we use the Hilbert–Mumford criterion to prove the first two parts of Theorem 1.2, which says that no Hénon map is stable and that most Hénon maps are unstable.

Proof of Theorem 1.2(a,b).

We homogenize equation (3), using the new variable by $x_{N+1}$, to obtain a birational map

[TABLE]

Writing ${\bar{P}}_{i}({\boldsymbol{x}})$ for the homogenization of $P_{i}({\boldsymbol{x}})$, the birational map ${\bar{f}}_{\boldsymbol{P}}:\mathbb{P}^{N}\to\mathbb{P}^{N}$ has the form

[TABLE]

We write $I_{n}$ to denote the $n$-by-$n$ identity matrix. For integers $r,s,t$, not all [math] and satisfying

[TABLE]

we define a 1-PS $L:\mathbb{G}_{m}\to\operatorname{SL}_{N+1}$ by the formula

[TABLE]

Explicitly, the action of $L_{r,s,t}$ on the monomials $x_{1},\ldots,x_{N+1}$ is given by

[TABLE]

We compute the powers of $\alpha$ that appear in front of the monomials in

[TABLE]

The results ae given in Table 1.

In Table 1, only line (IV), which deals with the monomials appearing in the polynomials ${\bar{P}}_{k+1},\ldots,{\bar{P}}_{N+1}$, needs some further explanation. For each $k\leq{i}\leq{N}$, the polynomial $P_{i+1}$ is a polynomial in the variables $x_{k},\ldots,x_{i}$ by definition, so in the homogenized polynomials in ${\bar{P}}_{i+1}$, the monomials are of degree $d$ in the variables $x_{k},\ldots,x_{i},x_{N+1}$, i.e., they are monomials of the form

[TABLE]

The 1-PS $L_{r,s,t}$ mutliplies each of $x_{k},\ldots,x_{i}$ by $\alpha^{-s}$ and multiplies $x_{N+1}$ by $\alpha^{-t}$. The monmomials of the form (6) appear in the $i$th coordinate for some $k\leq{i}\leq{N}$, so conjugation of ${\bar{f}}_{\boldsymbol{P}}$ by $L_{r,s,t}$ multiplies the monomial (6) by

[TABLE]

Letting $m$ denote the quantity $e_{k}+\cdots+e_{i}$, we see that every monomial of every ${\bar{P}}_{i}$ in ${\bar{f}}_{\boldsymbol{P}}$ is multiplied by $\alpha^{(m-1)s+(d-m)t}$ for some $0\leq{m}\leq{d}$.

Case 1: $\boldsymbol{k\geq 3~{}\textbf{or}~{}d\geq 3}$

By definition of Hénon maps, we always have $N\geq{k}\geq 2$ and $d\geq 2$. We set

[TABLE]

so $(r,s,t)$ satisfy (5) and are all strictly positive. It follows immediately that the exponent of $\alpha$ in lines I, II, III, and VI of Table 1 are strictly positive.

For line IV, we observe that if $1\leq{m}\leq{d}$, then $(m-1)s+(d-m)t$ is a sum of two non-negative terms, at least one of which is positive, so the sum is positive. And for $m=0$, using (7) shows that the exponent of $\alpha$ is

[TABLE]

For line V, we use (7) and a little algebra to compute

[TABLE]

Note that this is where we need to assume that $k\geq 3$ or $d\geq 3$, since if $k=d=2$, then $(d-1)(k-1)-2=-1$, so $-r-s+(d-1)t=-N$.

This completes the proof that

[TABLE]

and hence by the numerical criterion that ${\bar{f}}_{{\boldsymbol{P}}}$ is in the $\operatorname{SL}_{N+1}$-unstable locus.

Case 2: $\boldsymbol{k=2~{}\textbf{and}~{}d=2}$

In this case we take

[TABLE]

which satisfy (5). The exponents of $\alpha$ in the six rows of Table 1 are

[TABLE]

Hence

[TABLE]

which by the numerical criterion shows that ${\bar{f}}_{\boldsymbol{P}}$ is not stable. ∎

4. Unstable quadratic affine morphisms on $\mathbb{A}^{2}$

In Section 3 we showed that all Hénon maps are not stable, but we were not able to show that certain quadratic maps were actually unstable. There is a good reason for this, because at least for $N=2$, they are semistable, a fact that we prove in this section. However, since we do not see how to show directly that these maps are semistable, we instead classify unstable quadratic maps on $\mathbb{P}^{2}$, and then we show that this classification precludes an unstable map from being a Hénon map.

Theorem 4.1**.**

Let $F:\mathbb{P}^{2}\to\mathbb{P}^{2}$ be a rational map having the following properties:**

• •

$F:\mathbb{P}^{2}\to\mathbb{P}^{2}$* is a dominant rational map of degree $2$, i.e., $F^{*}\mathcal{O}_{\mathbb{P}^{2}}(1)=\mathcal{O}_{\mathbb{P}^{2}}(2)$.*

• •

$F$* is in the $\operatorname{SL}_{3}$-unstable locus of $\operatorname{Rat}_{2}^{2}$.*

Then one of the following is true:**

• (a)

The map $F$ factors through a non-constant linear projection to $\mathbb{P}^{1}$, i.e., there is a rational map $\pi:\mathbb{P}^{2}\to\mathbb{P}^{1}$ satisfying $\pi^{*}\mathcal{O}_{\mathbb{P}^{1}}(1)=\mathcal{O}_{\mathbb{P}^{2}}(1)$ and a rational map $G:\mathbb{P}^{1}\to\mathbb{P}^{1}$ making the following diagram commute:

[TABLE]

• (b)

The second iterate of $F$ satisfies $\deg(F^{2})\leq 2$, so in particular, the map $F$ is not algebraically stable in the sense of dynamical systems.333In general, if $F:\mathbb{P}^{N}\to\mathbb{P}^{N}$ is a dominant rational map, then $F$ is said to be algebraically stable, in the sense of dynamics, if $\deg(F^{n})=(\deg{F})^{n}$ for all $n\geq 1$.

Proof.

The assumption that $F$ is unstable means that we can find a 1-PS $L:\mathbb{G}_{m}\to\operatorname{SL}_{3}$ satisfying $\mu(F,L)>0$. We choose coordinates to diagonalize $L$ and so that the exponents of $\alpha$ are non-increasing. This gives an $L$ of the form

[TABLE]

Table 2 lists the exponents of $\alpha$ for the monomials of $F^{L}:=L\circ{F}\circ{L}^{-1}$. Then $\mu(F,L_{r,s})$ is the smallest entry in the table whose monomial appears in $F$. Alternatively, our assumption that $\mu(F,L_{r,s})>0$ means that every non-positive entry in Table 2 forces the corresponding monomial to not appear in $F$.

Case 1: $\boldsymbol{r\geq s}$

In this case we look at the $x^{2}$, $xy$ and $xz$ columns and the $y$ and $z$-coordinate rows in Table 2. These six entries are

[TABLE]

The fact that $r,s>0$ immediately implies that four of the entries are negative. Further, the entry $-2r+2s$ is non-positive because we are in the case that $r\geq{s}$, and the entry $-3r+s$ is negative because we normalized $L$ so that $r\geq{s-r}$.

It follows that the monomials $x^{2}$, $xy$, and $xz$ do not appear in the $y$ and $z$-coordinates of $F$, so $F$ has the form

[TABLE]

Thus $F$ factors over $\mathbb{P}^{1}$ as in (8) with

[TABLE]

Case 2: $\boldsymbol{r<s}$

We rewrite Table 2, putting an ${\boldsymbol{\times}}$ in the boxes whose entries are non-positive. The results are compiled in Table 3.

Examining Table 3, we see that $F$ has the form

[TABLE]

Hence $[1,0,0]$ is in the indeterminacy locus of $F$, and $F$ sends the entire line $[u,v,0]$ to the indeterminacy point $[1,0,0]$. It follows that we have a strict inequality $\deg(F^{2})<(\deg{F})^{2}$. Indeed, since the first coordinate of $F$ lacks both an $x^{2}$ and an $xy$ term, we see by a direct calculation that $\deg(F^{2})\leq 2$. ∎

We now have the tools needed to complete the proof of Theorem 1.2.

Corollary 4.2** (Theorem 1.2(c)).**

Let ${\bar{f}}_{P}:\mathbb{P}^{2}\to\mathbb{P}^{2}$ be a Hénon map of degree $2$, i.e., there are scalars $a,b\in{K^{*}}$ and a polynomial $P(y)\in{K[x]}$ of degree $2$ such that ${\bar{f}}_{P}$ is the extension to $\mathbb{P}^{2}$ of the affine automorphism

[TABLE]

Then ${\bar{f}}_{P}$ is $\operatorname{SL}_{3}$-semistable.

Proof.

To ease notation in the proof of Corollary 4.2, we write $f$ and ${\bar{f}}$ for $f_{P}$ and ${\bar{f}}_{P}$. We assume that ${\bar{f}}$ is $\operatorname{SL}_{3}$-unstable and derive a contradiction. This assumption and Theorem 4.1 imply that either $\deg({\bar{f}}^{2})\leq 2$ or ${\bar{f}}$ is linearly fibered over $\mathbb{P}^{1}$. A direct calculation with (9) yields $\deg(f^{2})=4$, so there is a non-constant linear projection $\pi:\mathbb{P}^{2}\to\mathbb{P}^{1}$ and a rational map ${\bar{g}}:\mathbb{P}^{1}\to\mathbb{P}^{1}$ satisfying $\pi\circ{{\bar{f}}}={\bar{g}}\circ\pi$, as illustrated by the commutative diagram (8) with $F={\bar{f}}$ and $G={\bar{g}}$.

For each $t\in\mathbb{P}^{1}(K)$, we let

[TABLE]

be the line lying over $t$, i.e., the Zariski closure of the inverse image of $t$. Then the semi-conjugation $\pi\circ{{\bar{f}}}={\bar{g}}\circ\pi$ implies that444We use the usual convention that if $F:X\to{Y}$ is a rational map between smooth projective varieties with indeterminacy locus $I_{f}$, and if $Z\subset{X}$ is a subvariety of codimension $1$, then its image $F(Z)$ is defined to be $\overline{F(Z\smallsetminus{I_{f}})}$.

[TABLE]

This suggests that we study the effect of ${\bar{f}}$ on lines, as in the next result.

Lemma 4.3**.**

Let $\ell\subset\mathbb{P}^{2}$ be a line. Then its image under the Hénon map ${\bar{f}}$ is as follows:

[TABLE]

Proof.

It is clear that ${\bar{f}}$ sends $\{z=0\}$ to $[0:1:0]$. For all other lines, we work with the affine polynomial map $f$ given by (9). The image of the horizontal line $\ell=\{y=B\}$ is

[TABLE]

Finally, every non-horizontal line has the form $\{x=\lambda{y}+\mu\}$ for some $\lambda,\mu\in{K}$. The image of this line is

[TABLE]

which is an irreducible conic, since $\deg P=2$ by assumption. This concludes the proof of Lemma 4.3. ∎

The fact that $\pi$ is a linear projection implies that the lines $\ell_{t}$ are distinct for distinct $t$. In particular, there is at most one $t\in\mathbb{P}^{1}(K)$ with $\ell_{t}=\{z=0\}$. If there is such a $t$, we denote it by $t_{0}$, and we define a set of points

[TABLE]

and if $\ell_{t}\neq\{z=0\}$ for all $t$, then we set $T=\emptyset$. We note that $T$ is a finite set, since ${\bar{g}}:\mathbb{P}^{1}\to\mathbb{P}^{1}$ is non-constant, and that our definition of $T$ ensures that for all $t\in\mathbb{P}^{1}(K)\smallsetminus{T}$, neither of the lines $\ell_{t}$ and $\ell_{{\bar{g}}(t)}$ is equal to the line $\{z=0\}$.

Let $t\mathbb{P}^{1}(K)\smallsetminus\{t_{0}\}$. Then $\ell_{t}\neq\{z=0\}$, but (10) says that ${\bar{f}}(\ell_{t})$ is contained in the line $\ell_{{\bar{g}}(t)}$, so Lemma 4.3 implies that $\ell_{t}$ is a line of the form $\{y=Bz\}$ and that its image ${\bar{f}}(\ell_{t})$ is a line of the form $x=Az$. This proves:

[TABLE]

Nest suppose that $t\in\mathbb{P}^{1}(K)\setminus{T}$. Then ${\bar{g}}(t)\neq{t_{0}}$, so we may apply the first formula in (11) with $t$ replaced by ${\bar{g}}(t)$ to conclude that

[TABLE]

Combining (11), and (12), we find that for all $t\in\mathbb{P}^{1}(K)\smallsetminus{T}$, we have

[TABLE]

But a line of the form $x=Az$ cannot equal a line of the form $y=Bz$. This contradication concludes the proof that ${\bar{f}}$ is not unstable. ∎

Remark 4.4**.**

The proof of Corollary 4.2, which appears to be somewhat ad hoc, comes down to showing that a Hénon map ${\bar{f}}_{P}:\mathbb{P}^{2}\to\mathbb{P}^{2}$ is not linearly fibered over $\mathbb{P}^{1}$. One might be tempted to exploit the fact that ${\bar{f}}_{P}$ is a birational map with critical locus $\operatorname{Crit}({\bar{f}}_{P})=3\{z=0\}$, and that ${\bar{f}}_{P}$ is the extension of $\mathbb{P}^{2}$ of an affine automorphism $f_{P}:\mathbb{A}^{2}\to\mathbb{A}^{2}$. So it is instructive to keep in mind the affine automorphism

[TABLE]

and its extension $\bar{\phi}:\mathbb{P}^{2}\to\mathbb{P}^{2}$. The map $\bar{\phi}$ is unstable, satisfies $\operatorname{Crit}(\bar{\phi})=3\{z=0\}$, and is linearly fibered over $\mathbb{P}^{1}$ by the map $\pi(x:y:z)=[x:z]$.