Exceptional splitting of reductions of abelian surfaces

TL;DR

This paper proves that abelian surfaces with real multiplication have infinitely many primes of split reduction, using Arakelov intersection theory on Hilbert modular surfaces, extending results known for elliptic curves.

Contribution

It establishes the infinitude of split reduction primes for abelian surfaces with real multiplication, a significant extension of previous elliptic curve results.

Findings

Infinitely many split reduction primes for abelian surfaces with real multiplication

Density-zero set of primes pertaining to reduction is infinite

Utilizes Arakelov intersection theory on Hilbert modular surfaces

Abstract

Heuristics based on the Sato--Tate conjecture suggest that an abelian surface defined over a number field has infinitely many places of split reduction. We prove this result for abelian surfaces having real multiplication. Similar to Charles' theorem on exceptional isogeny of reductions of a given pair of elliptic curves and Elkies' theorem on supersingular reductions of a given elliptic curve, our theorem shows that a density-zero set of primes pertaining to the reduction of abelian varieties is infinite. The proof relies on the Arakelov intersection theory on Hilbert modular surfaces.

Equations

$$L=\left\{\left(\begin{array}{cc}a& \gamma \\ {\gamma }^{\prime }& b\end{array}\right):a\in (D\mathrm{Nm}\mathfrak{a})\mathbb{Z},b\in \mathbb{Z},\gamma \in \mathfrak{a}\right\}$$

$$V=\left\{\left(\begin{array}{cc}a& \gamma \\ {\gamma }^{\prime }& b\end{array}\right):a,b\in \mathbb{Q},\gamma \in F\right\}$$

$$\bigcup _{M\in L,\det (M)=r\mathrm{Nm}\mathfrak{a}}\{(z_1,z_2)\in {\mathbb{H}}^2:a{z}_1{z}_2+\gamma z_1+{\gamma }^{\prime }{z}_2+b=0\}.$$

$$\alpha z_1{z}_2+\beta z_2={\beta }^{\prime }{z}_1-\frac{b}{\sqrt{D}},{\alpha }^{\prime }{z}_1{z}_2+{\beta }^{\prime }{z}_1=\beta z_2+\frac{b}{\sqrt{D}}.$$

$$\alpha z_1{z}_2+{\beta }^{\prime }{z}_1+\beta z_2-\delta =0,\beta {\beta }^{\prime }+\alpha \delta =r>0.$$

$$-\sum _{[B]\in T_{\mathfrak{p}}[A]}\log ||\Psi (\sigma ([B]))|{|}_{\mathrm{Pet}}=o(p\log p)\text{ as }p\to \infty .$$

$$-\log |a{z}_1(B)z_2(B)+\gamma z_1(B)+{\gamma }^{\prime }{z}_2(B)+b|\le O(\log p),\forall [B]\in T_{\mathfrak{p}}([A]).$$

$$|a{z}_1(B)z_2(B)+\gamma z_1(B)+{\gamma }^{\prime }{z}_2(B)+b|<p^{-C_1}$$

$$m{z}_1(A)z_2(A)+\eta z_1(A)+{\eta }^{\prime }{z}_2(A)+l=a{z}_1(B)z_2(B)+\lambda z_1(B)+{\lambda }^{\prime }{z}_2(B)+b.$$

$$|u_{22}|\le |u_{21}{x}_1(A)+u_{22}|+|u_{21}{x}_1(A)|\le |u_{21}{z}_1(A)+u_{22}|+|u_{21}{x}_1(A)|\le C_0^2{C}_6\sqrt{p}+C_0^4{C}_6\sqrt{p}.$$

$$|u_{11}|y_1(A)\le |u_{11}{z}_1(A)+u_{12}|=|(z_1(B)(u_{21}{z}_1(A)+u_{22}))|\le C_7\sqrt{p}.$$

$$|u_{11}|\le C_0{C}_7\sqrt{p},|u_{12}|\le |u_{11}{z}_1(A)+u_{12}|+|u_{11}{x}_1(A)|\le C_7\sqrt{p}+C_0^2{C}_7\sqrt{p}.$$

$$|m_i{z}_1(A)z_2(A)+{\eta }_i{z}_1(A)+{\eta }_i^{\prime }{z}_2(A)+l_i|<p_i^{-C_1}.$$

$$\left|z_2(A)+\frac{{\eta }_i{z}_1(A)+l_i}{{\eta }_i^{\prime }+m_i{z}_1(A)}\right|\le \frac{p_i^{-C_1}}{|{\eta }_i^{\prime }+m_i{z}_1(A)|}\le C_9{p}_i^{-C_1}.$$

$$|f(z_{1}(A))|\leq\big{|}(\eta^{\prime}_{1}+m_{1}z_{1}(A))(\eta^{\prime}_{2}+m_{2}z_{1}(A))\big{|}\cdot\left|\frac{\eta_{1}z_{1}(A)+l_{1}}{\eta^{\prime}_{1}+m_{1}z_{1}(A)}-\frac{\eta_{2}z_{1}(A)+l_{2}}{\eta^{\prime}_{2}+m_{2}z_{1}(A)}\right|\leq(C_{5}p_{2}+2C_{0}C_{5}p_{2})^{2}\cdot(2C_{9}p_{1}^{-C_{1}}).$$

$$2C_9{p}_1^{-C_1}>\left|\frac{{\eta }_1{z}_1(A)+l_1}{{\eta }_1^{\prime }+m_1{z}_1(A)}-\frac{{\eta }_2{z}_1(A)+l_2}{{\eta }_2^{\prime }+m_2{z}_1(A)}\right|=\left|\frac{({\eta }_1{m}_2-{\eta }_2{m}_1)(z_1(A)-\alpha )(z_1(A)-\beta )}{({\eta }_1^{\prime }+m_1{z}_1(A))({\eta }_2^{\prime }+m_2{z}_1(A))}\right|.$$

$$|z_1(A)-z_1(A_{CM})|\le 4C_0{C}_5^4{C}_9(1+2C_0{)}^2{p}_1^{-C_1+6}.$$

$$|z_2(A)-z_2(A_{CM})|$$

$$|{\alpha }_2^2{\Delta }_1-{\alpha }_1^2{\Delta }_2|\ge |({\alpha }_2^{\prime }{)}^2{\Delta }_1^{\prime }-({\alpha }_1^{\prime }{)}^2{\Delta }_2^{\prime }{|}^{-1}\ge (8C_5^2{C}_{11}{)}^{-1}{N}^{-8}.$$

$$|z(A_1)-z(A_2)|$$

$$\{\text{integer }n\in [N^{1/2},N],n\text{ is represented by }{Q}_N^{\prime }\}\le 1+4\sqrt{2N}+8N/|{\Delta }_N^{\prime }{|}^{1/2}=O\left(\frac{N}{(\log N{)}^2}\right).$$

$$-\sum _{[B]\in T_{\mathfrak{p}}[A]}\log ||\Psi (\sigma ([B]))|{|}_{\mathrm{Pet}}\ge {ϵ}_{1}p\log p$$

$$-\sum _{B\in T_{\mathfrak{p}}A}\log |a{z}_1(B)z_2(B)+\gamma z_1(B)+{\gamma }^{\prime }{z}_2(B)+b|=o(p\log p).$$

$$\#\{[B]\in T_{\mathfrak{p}}[A]:\log |a{z}_1(B)z_2(B)+\gamma z_1(B)+{\gamma }^{\prime }{z}_2(B)+b|<C_3\}<ϵp/C_1.$$

$$-\sum _{[B]\in T_{\mathfrak{p}}[A]}\log |a{z}_1(B)z_2(B)+\gamma z_1(B)+{\gamma }^{\prime }{z}_2(B)+b|<-(p+1)C_3+(ϵp/C_1)\cdot C_1\log p$$

$$\sum_{[B]\in T_{\mathfrak{p}}[A]}\big{(}G(z(B))+\sum_{r\in\mathbb{I}^{\prime}}c(r)\sum_{(a,b,\gamma)\in\mathcal{M}_{\Omega,r}}\phi(z)\log|az_{1}(B)z_{2}(B)+\gamma z_{1}(B)+\gamma^{\prime}z_{2}(B)+b|\big{)}<C_{2}\cdot(p+1).$$

$$\sum _{[B]\in T_{\mathfrak{p}}[A]}G(z(B))<C_4(p+1)+({ϵ}_{1}p\log p)/2.$$

$$\prod _{i=1}^m{\mu }_i(M_{v,n_0+(k+1)e_v})\ge {\ell }^{m^{\prime }}\prod _{i=1}^m{\mu }_i(M_{v,n_0+k{e}_v})$$

$$\#\{s\in M_{v,n}\mid Q(s)\leq N\}=O\Big{(}N^{1/2}+\displaystyle{\sum_{j=2}^{m}\frac{N^{j/2}}{\ell^{(j-1)n/e_{v}}}}\Big{)}$$

$$C{d}^{4}v=(C{d}^{4}u+{\ell }^{\lfloor (n-n_0)/e_v\rfloor }{w}_1)+{\ell }^{\lfloor (n-n_0)/e_v\rfloor }{w}_1^{\prime },$$

Full text

1. Introduction

1.1. Infinitely many nonsimple reductions of a given abelian surface

Murty and Patankar conjectured in [MP] that an absolutely simple abelian variety over a number field has absolutely simple reduction for a density one set of primes (up to a finite extension) if and only if its endomorphism ring is commutative. Chavdarov ([Cha]) proved their conjecture in the case of abelian varieties of dimension 2 or 6 whose geometric endomorphism ring is $\mathbb{Z}$. Conditional upon the Mumford–Tate conjecture, Zywina ([Zyw]) established Murty and Patankar’s conjecture in full generality.

It is natural to inquire whether the set of primes (conjecturally a density zero set!) at which a given abelian variety does not have absolutely simple reduction is finite or infinite. Based on the Sato–Tate conjecture for abelian surfaces (see §1.2), it is expected that the (density zero) set of places of nonsimple reduction of a simple abelian surface is infinite. The main result of this paper is the following:

Theorem 1**.**

Let $A$ be an abelian surface over a number field $K$. Suppose that $F\subset\operatorname{End}(A)\otimes\mathbb{Q}$, where $F$ is a real quadratic field. Then $A$ modulo $v$ is not absolutely simple for infinitely many primes $v$ of $K$.

1.2. A heuristic based on the Sato–Tate conjecture

The classical Sato–Tate conjecture addresses the distribution of Frobenius elements involved in the Galois representation on the étale cohomology of a fixed elliptic curve defined over a number field. The work of Katz–Sarnak [KS], Serre [Se12], and Fité–Kedlaya–Rotger–Sutherland ([FKRS]) generalizes this conjecture to higher dimensional abelian varieties. We focus on the case of abelian surfaces with real multiplication and offer a heuristic which indicates that such surfaces have infinitely many places of nonsimple reduction.

For simplicity, assume that $A$ is an abelian surface defined over $\mathbb{Q}$ such that $\operatorname{End}(A)\otimes\mathbb{Q}=\operatorname{End}(A_{\overline{\mathbb{Q}}})\otimes\mathbb{Q}=F$, a real quadratic field. For each prime $\ell$ of good reduction for $A$, the characteristic polynomial of the Frobenius endomorphism of $A$ modulo $\ell$ is of the form $x^{4}+a_{1}x^{3}+a_{2}x^{2}+\ell a_{1}x+\ell^{2}$, with $a_{1},a_{2}\in\mathbb{Z}$. The roots of this polynomial come in complex conjugate pairs $\lambda_{1},\overline{\lambda_{1}},\lambda_{2},\overline{\lambda_{2}}$, and each $\lambda_{i}$ has absolute value $\ell^{1/2}$. Define $s_{i,\ell}=\frac{\lambda_{i}+\overline{\lambda_{i}}}{\sqrt{\ell}}$. The Sato–Tate conjecture for abelian surfaces with real multiplication predicts that the distribution of $(s_{1,\ell},s_{2,\ell})\in[-2,2]\times[-2,2]$, as $\ell$ varies, converges to the measure on $[-2,2]\times[-2,2]$ defined by the function $(\frac{4}{\pi})^{2}\sqrt{4-s_{1}^{2}}\sqrt{4-s_{2}^{2}}$ (for instance, see [Ked]). Assuming fast enough rates of convergence to this measure, the probability that $|s_{1,\ell}-s_{2,\ell}|<\frac{1}{\sqrt{\ell}}$ is approximately the area of the region $V_{\ell}=\{(s_{1},s_{2}):|s_{1}-s_{2}|<\frac{1}{\sqrt{\ell}}\}\subset[-2,2]\times[-2,2]$, which is approximately $\frac{1}{\sqrt{\ell}}$.111More precisely, there exist absolute constants $C_{1},C_{2}>0$ such that the probability that $|s_{1,\ell}-s_{2,\ell}|<\frac{1}{\sqrt{\ell}}$ is greater than $C_{1}/\sqrt{\ell}$ and less than $C_{2}/\sqrt{\ell}$.

If $|s_{1,\ell}-s_{2,\ell}|<\frac{1}{\sqrt{\ell}}$, it is easy to see that $s_{1,\ell}$ and $s_{2,\ell}$ must be equal. Then by Honda–Tate theory, $A$ modulo $\ell$ is not simple. This gives a heuristic lower bound ($\approx\frac{1}{\sqrt{\ell}}$) for the probability that $A$ mod $\ell$ is not simple222This does not take into account those primes modulo which $A$ is simple, but not absolutely simple.. On the other hand, $\displaystyle{\sum_{\ell\text{ prime}}\frac{1}{\sqrt{\ell}}}$ diverges, so $A$ should have infinitely many primes of nonsimple reduction.

1.3. Related results

The Sato–Tate conjectures for elliptic curves and pairs of elliptic curves also suggests that the set of primes in either of the following two situations is infinite:

1. (1)

given an elliptic curve $E$ over a number field, consider primes $v$ such that $E$ mod $v$ is supersingular; 2. (2)

given a pair of non-isogenous elliptic curves $E_{1},E_{2}$ over a number field, consider primes $v$ such that the reductions of $E_{1},E_{2}$ mod $v$ become geometrically isogenous.

As in our case, both sets of primes have density zero (after taking a finite extension). Indeed, Serre conjectured that up to taking a finite extension of the field of definition, a given abelian variety over a number field has ordinary reduction at a density-one set of primes. Katz proved Serre’s conjecture in the case of elliptic curves and abelian surfaces ([Ogus, pages 370–372]). Sawin (in [Sawin]) made explicit the smallest field extension that is required for abelian surfaces. The second set also has density zero (after taking a finite extension) by Faltings’ isogeny theorem ([F85]).

Elkies proved (1) in [Elk1, Elk2] when the elliptic curve is defined over a number field with at least one real embedding and Charles proved (2) in [Ch]. Theorem 1 is an analogue of these two results. Indeed, all three results establish that certain thin sets of primes related to the reduction of abelian varieties are infinite.

1.4. The strategy of the proof

The proof of Theorem 1 builds on the idea of the proof of the main theorem in [Ch], where Charles uses Arakelov intersection theory on the modular curve $X_{0}(1)$ to prove his result. In our case, we use Arakelov intersection theory on the Hilbert modular surface $\mathcal{H}$ (see §2.1 for the precise definition).

Let $[A]\in\mathcal{H}(K)$ denote the point determined by $A$. Loosely speaking, the modular curve embeds canonically into $\mathcal{H}$ (we label its image $\Delta$) and parameterizes the locus of split abelian surfaces (along with the product polarization). A natural strategy is to consider the Arakelov intersection of $\Delta$ with Hecke orbits of $[A]$.333Here we refer to the $1$-cycle, given by taking the Zariski closure of the Hecke orbit of $[A]$ in $\mathcal{H}$ over $\operatorname{Spec}\mathbb{Z}$. The local contribution at a finite prime $v$ is positive precisely when the Hecke orbit of $[A]$ intersects $\Delta$ modulo $v$. The reduction of $A$ modulo $v$ would be geometrically nonsimple for such $v$.

In our proof, we replace $\Delta$ with a compact Hirzebruch–Zagier divisor $\mathcal{T}$ (see §2.1 and §5.1 for definitions). Hirzebruch–Zagier divisors of $\mathcal{H}$ have the feature that the rank of the Néron–Severi group of an abelian surface $B$ increases if $[B]$ lies on these divisors. This has the consequence that over a finite field, an abelian surface is not absolutely simple if it lies on a special divisor. There are two advantages of using a compact Hirzebruch–Zagier divisor $\mathcal{T}$: the first is that we do not have to deal with places of bad reduction for $A$. The second is that we are able to avoid all the cusps of $\mathcal{H}$ in the archimedean contribution to the global Arakelov intersection.

In order to prove Theorem 1, it would suffice to prove that the set of primes which contribute to the intersection is infinite as we vary over infinitely many well-chosen Hecke orbits of $[A]$. There are two steps involved in proving this:

• •

A local step, where we bound the local contribution of the intersection at every place.

• •

A global step, where we compute the growth of the Arakelov intersection of Hecke orbits of $[A]$ with $\mathcal{T}$. The growth is expressed in terms of the degree of the Hecke operators, and is seen to grow asymptotically faster than the local contributions at each place.

Consequently, it follows that more and more primes contribute to this intersection as we vary the Hecke orbit of $A$.

The abelian surfaces parametrized by Hirzebruch–Zagier divisors have extra special endomorphisms (recalled in §2.1). In the non-archimedean case, our methods are very different from the ones used in [Ch]. For a finite place $v$, we use Grothendieck–Messing theory prove statements about the rate of decay of special endomorphisms of $A[\ell^{\infty}]$ modulo higher and higher powers of $v$. This method avoids the use of CM lifts and can be used in other Arakelov-theoretic situations. We use these results and Geometry-of-numbers arguments to bound the number of special endomorphisms of $A$ modulo powers of $v$. This allows us to prove that the $v$-adic contribution grows asymptotically slower than the global intersection for most of the Hecke orbits that we consider. Indeed, if there were too many Hecke orbits $T_{\mathfrak{p}}([A])$ having large $v$-adic intersection with $\mathcal{T}$, then $A$ modulo $v^{n}$ would have too many special endomorphisms as $n\rightarrow\infty$.

The arguments used to bound the archimedean contribution are very different from the ones used to bound the finite contributions. A key step in bounding the archimedean contribution is the following statement: for a fixed infinite place, if $[A]$ is close to two Hirzebruch–Zagier divisors, then $[A]$ must be close to their intersection, which is a CM abelian surface.

In order to prove the global part of our result, it is necessary to relate the global Arakelov intersection of $\mathcal{T}$ and certain Hecke orbits $T_{\mathfrak{p}}([A])$ (see §2.2.1 for the precise definition) to the intersection of $[A]$ and $\mathcal{T}$. We accomplish this in two steps:

• •

We use Borcherds’ theory (briefly recalled after Lemma 5.1.1) to construct a compact special divisor, whose class in the Picard group of a toroidal compactification of $\mathcal{H}$ equals, up to multiplying by a constant in $\mathbb{Z}_{>0}$, the class of the Hodge bundle. Consequently, the global Arakelov intersection of $[A]$ with $\mathcal{T}$ (endowed with a suitable Hermitian metric),444Technically speaking, since we will not make $[A]$ into an arithmetic cycle, here by Arakelov intersection, we mean the height of the $1$-cycle $[A]$ with respect to the Arakelov divisor $\mathcal{T}$, which is endowed with a Hermitian metric by Borcherds’ theory. up to multiplying by a suitable constant, equals the Faltings height of $A$.

• •

We relate the Faltings height of $T_{\mathfrak{p}}([A])$ to the Faltings height of $A$ when $A$ has potentially good reduction at $p$ in Proposition 5.1.6. This extends a result of Autissier ([Au, Theorem 5.1]), which only applies to $A$ with potentially ordinary reduction at $p$.

It follows that the global intersection number grows faster than any local contribution. Hence, infinitely many primes occur in the intersection of $T_{\mathfrak{p}}([A])$ and $\mathcal{T}$ as $p\rightarrow\infty$.

1.5. Organization of the paper

In §2, we recall the definitions of the Hilbert modular surface and the Hirzebruch–Zagier divisors. In §3, we bound the archimedean contribution. We spend §4 counting special endomorphisms and bounding the non-archimedean contribution. We use Borcherds’ theory in §5.1 to choose a compact Hirzebruch–Zagier divisor and relate the Arakelov intersection number to Faltings height. We also extend Autissier’s result to the setting of Hilbert modular surfaces. Finally, we assemble all these results together in §5.2 to prove Theorem 1.

1.6. Notation and conventions

We use $K$ to denote a number field and let $\mathcal{O}_{K}$ be its ring of integers. As in Theorem 1, we use $F$ to denote a fixed real quadratic field with discriminant $D$; its ring of integers is denoted by $\mathcal{O}_{F}$ and $\mathfrak{d}_{F}$ is its different ideal. For $a\in F$, we use $\operatorname{Nm}a$ to denote its $F/\mathbb{Q}$-norm.

The statement of Theorem 1 is invariant under isogeny. Therefore, we will always assume that $A$ has real multiplication by the maximal order $\mathcal{O}_{F}$, and is equipped with an $\mathfrak{a}$-polarization for some (integral) ideal $\mathfrak{a}$ of $\mathcal{O}_{F}$; see [Pa]*§ 2.1 item 2 before Def. 2.1.1 for the definition of an $\mathfrak{a}$-polarization. We may also assume that $A$ has semistable reduction over $K$. For any abelian varieties $B,B^{\prime}$, we use $\operatorname{End}(B)$ and $\operatorname{Hom}(B,B^{\prime})$ to denote the endomorphism ring of $B$ and the $\mathbb{Z}$-module given by homomorphisms from $B$ to $B^{\prime}$.

Let $\mathcal{H}$ be the Hilbert modular surface over $\mathbb{Z}$ associated to $F$ which is the moduli stack of abelian surfaces $B$ with real multiplication by $\mathcal{O}_{F}$ and an $\mathfrak{a}$-polarization. This is a Deligne–Mumford stack and we use $[B]$ to denote the point of $\mathcal{H}$ determined by $B$. Sometimes, we may denote a point of $\mathcal{H}$ by $[B^{\prime}]$; this means that $B^{\prime}$ is the abelian surface determines this given point.

We always use mathcal letters to mean the natural extension over certain ring of integers. For example, we use $\mathcal{A}$ to denote the everywhere semistable semi-abelian scheme over $\mathcal{O}_{K}$ such that $\mathcal{A}_{K}=A_{K}$.

Throughout the text, $v$ means a place of $K$, either archimedean or finite. If $v$ is finite, we use $\mathbb{F}_{v}$ to denote its residue field and $e_{v}$ to denote the degree of ramification of $K$ at $v$. If $\mathcal{A}$ has good reduction at $v$, we use $\mathcal{A}_{v,n}$ to denote $\mathcal{A}$ modulo $v^{n}$. We always use $p$ to denote a prime number which is totally split in the narrow Hilbert class field of $F$ and denote by $\mathfrak{p},\mathfrak{p}^{\prime}$ the two primes ideals of $F$ above $p$. We use $A[p]$ and $A[\mathfrak{p}]$ to denote the $p$-torsion and $\mathfrak{p}$-torsion subgroups of $A$.

Acknowledgements

We thank George Boxer, Francesc Castella, Kęstutis Česnavičius, François Charles, William Chen, Noam Elkies, Ziyang Gao, Chi-Yun Hsu, Nicholas Katz, Ilya Khayutin, Djordjo Milovic, Lucia Mocz, Peter Sarnak, William Sawin, Arul Shankar, Jacob Tsimerman, Tonghai Yang, and Shou-Wu Zhang for useful comments and/or discussions. We are also grateful to Kęstutis Česnavičius, Chao Li, Mark Kisin, and Lucia Mocz for very useful comments on previous versions of this paper. We would like to thank Davesh Maulik for pointing a gap in Theorem 4.1.1 in an earlier version of this paper. The second author, during her stay at the Institute for Advanced Study, was supported by the NSF grant DMS-1128115 to IAS.

2. Hirzebruch–Zagier divisors and Hecke orbits

In this section, we first recall the definition of Hirzebruch–Zagier divisors and their properties and then we specify the Hecke orbits that will be used in the rest of the paper.

2.1. The Hilbert modular surface and the Hirzebruch–Zagier divisors

Recall that we use $\mathcal{H}$ to denote the moduli stack over $\operatorname{Spec}\mathbb{Z}$ that parametrizes abelian surfaces with real multiplication by $\mathcal{O}_{F}$ and an $\mathfrak{a}$-polarization (see [Pa]*Def. 2.1.1). It is a Deligne–Mumford stack. A totally positive element $a\in\mathfrak{a}$ gives rise to a polarization on $A$ and a symplectic form $\psi$ on the Betti cohomology group $W=H^{1}_{B}(A(\mathbb{C}),\mathbb{Q})$ (here we choose an embedding $K\rightarrow\mathbb{C}$). The set of $\operatorname{GSp}(W,\psi)(\mathbb{R})$-conjugate cocharacters of the Hodge cocharacter (from the Hodge decomposition of $W\otimes\mathbb{C}$) of $A_{\mathbb{C}}$ coincides with $\mathbb{H}_{2}^{\pm}$, the upper and lower Siegel half plane of genus $2$. Let $G\subset\operatorname{GSp}(W,\psi)$ be the subgroup (over $\mathbb{Q}$) that commutes with $\mathcal{O}_{F}\subset\operatorname{End}(W)$ and let $G_{1}=\operatorname{Res}^{F}_{\mathbb{Q}}\operatorname{GL}_{2}$. Then $G$ is naturally isomorphic to the subgroup of $G_{1}$ such that for any $\mathbb{Q}$-algebra $R$, the set $G(R)$ consists of matrices with determinant in $R$ (instead of $R\otimes_{\mathbb{Q}}F$). The embedding $G\subset\operatorname{GSp}_{4}$ induces an embedding555One needs to choose suitable level structure to ensure that the natural finite morphism is an embedding. of the Hilbert modular surface $\mathcal{H}_{\mathbb{Q}}=Sh(G,X)$ into $Sh(\operatorname{GSp}(W,\psi),\mathbb{H}_{2}^{\pm})$ (see [vdG]*Chp.   IX.1 and here $X$ is the subset of $\mathbb{H}_{2}^{\pm}$ which consists of cocharacters conjugate to the Hodge cocharacter of $A_{\mathbb{C}}$ under $G(\mathbb{R})$).

Let $\overline{\mathcal{H}}^{\rm{tor}}$ be a toroidal compactification of $\mathcal{H}$ as in [R] (see also [Chai]*§3). The stack $\overline{\mathcal{H}}^{\rm{tor}}$ is regular and proper and $\overline{\mathcal{H}}^{\rm{tor}}\backslash\mathcal{H}$ is a normal crossing divisor (see, for example, [Pa]*2.1.2, 2.1.3 for the regularity of $\mathcal{H}$ and [Chai]Thm. 3.6, 4.3 for the smoothness of the formal neighborhood of the boundary and the property that the boundary is a normal crossing divisor), and hence the arithmetic intersection theory developed by Burgos Gil, Kramer, and Kühn in [BKK] applies to $\overline{\mathcal{H}}^{\rm{tor}}$ (see, for example, [BBK]§1, §6 for a summary of their theory). We will use $[\mathcal{A}]$ (resp. $[A]$) to denote the unique $\mathcal{O}_{K}$-point (resp. $K$-point) of $\mathcal{H}$ corresponding to $A$ (the stack $\overline{\mathcal{H}}^{\rm{tor}}$ being proper allows us to do this).666In general, one needs to pass to a finite field extension to extend a $K$-point on a proper Deligne–Mumford stack to an $\mathcal{O}_{K}$ point; however, since we have assumed that $A$ has a semistable integral model $\mathcal{A}$ over $\mathcal{O}_{K}$, we do not need to pass to a further field extension.

2.1.1*.*

We now summarize some basic facts about $\mathcal{H}_{\mathbb{C}}$ and the Hirzebruch–Zagier divisors. The facts discussed here can be found in [vdG]Chp. I, V, IX, [BBK]§2.3, §5.1 and [Go]*Chp. 2; however, since conventions differ, we will use this subsection to fix our notation. After giving the definition of Hirzebruch–Zagier divisors, we first show that among them, there exists a nonempty compact one and then we give a moduli interpretation of these divisors.

Let $\mathbb{H}$ denote the upper half plane. The two real embeddings of $F$ induce two embeddings $\sigma_{1},\sigma_{2}:\operatorname{SL}_{2}(F)\rightarrow\operatorname{SL}_{2}(\mathbb{R})$. The action of $g\in\operatorname{SL}_{2}(F)$ on $\mathbb{H}^{2}$ is given by $\sigma_{1}(g)$ on the first copy of $\mathbb{H}$ and by $\sigma_{2}(g)$ on the second copy. Since our Hilbert modular surface $\mathcal{H}$ admits a map to $\mathcal{S}$, we have $\displaystyle\mathcal{H}(\mathbb{C})=\Gamma\backslash\mathbb{H}^{2}$, where $\displaystyle\Gamma=\operatorname{SL}_{2}(F)\cap\begin{pmatrix}\mathcal{O}_{F}&(\mathfrak{a}\mathfrak{d}_{F})^{-1}\\ \mathfrak{a}\mathfrak{d}_{F}&\mathcal{O}_{F}\end{pmatrix}$ (see, for example, [Go]*pp. 71).777The Hilbert modular surface $\mathcal{H}$ is connected and hence we may use $\operatorname{Res}^{F}_{\mathbb{Q}}\operatorname{SL}_{2}$ instead of $G$ to study the complex points. Notice that our lattice is different from the default choice in [BBK]. Using their notation, we work with $\Gamma(\mathcal{O}_{F}\oplus\mathfrak{a}\mathfrak{d}_{F})$.

For any $r\in\mathbb{Z}_{>0}$, we recall the definition of the Hirzebruch–Zagier divisors $T(r)$ in $\mathcal{H}_{\mathbb{C}}$ (see for example [vdG]*V.1.3 and [BBK]*sec. 2.3). Let $\gamma^{\prime}$ denote the $\operatorname{Gal}(F/\mathbb{Q})$-conjugate of a given $\gamma\in F$. Consider the lattice

[TABLE]

in the rational quadratic space

[TABLE]

with the quadratic form given by the determinant. The group $\Gamma$ acts on $V$ via $v.g=(g^{\prime})^{t}\cdot v\cdot g$ for $g\in\Gamma,v\in V$ and this action preserves $L$. The quadratic space $V$ is of signature $(2,2)$. One may also view $\mathcal{H}$ as an orthogonal type Shimura variety defined by $\operatorname{SO}(V)$. The divisor $T(r)$ is defined to be the reduced divisor in $\mathcal{H}_{\mathbb{C}}$ whose set of $\mathbb{C}$-points is the image of888This is the definition in [BBK]. The lattice in [vdG] differs by a multiple of the scalar matrix $\sqrt{D}\cdot I$ so these two definitions of $T(r)$ coincide.

[TABLE]

Proposition 2.1.2**.**

The divisor $T(r)$ is nonempty if and only if $r\operatorname{Nm}\mathfrak{a}$ modulo $D$ is $-\operatorname{Nm}\gamma$ for some $\gamma\in\mathfrak{a}$. In this case, $T(r)$ is defined over $\overline{\mathbb{Q}}$ and is either a modular curve or a Shimura curve defined by the indefinite quaternion algebra $\displaystyle\left(\frac{D,-r\operatorname{Nm}\mathfrak{a}}{\mathbb{Q}}\right)$. If $r$ is not the norm of an ideal of $\mathcal{O}_{F}$, then $T(r)$ is a Shimura curve, and hence compact.

Proof.

The first assertion follows from the definition of a Hirzebruch–Zagier divisor. By the discussion on [vdG]*pp. 89–90, the divisor $T(r)$ is the union of Shimura curves defined by the quaternion algebra mentioned above and hence is defined over $\overline{\mathbb{Q}}$. The last assertion follows from [vdG]*Chp. V, 1.7. ∎

Corollary 2.1.3**.**

Let $q$ denote a rational prime inert in $F$. Then the divisor $T(qD)$ is non-empty and compact.

Proof.

As $q$ is inert, $qD$ is not the norm of an ideal of $\mathcal{O}_{F}$. Further, $qD$ is [math] modulo $D$. It follows from Proposition 2.1.2 that $T(qD)$ is compact and nonempty. ∎

Hirzebruch–Zagier divisors parametrize abelian surfaces with extra special endomorphisms. After recalling the definition of special endomorphisms, we give a sketch of the proof of this fact (see Lemma 2.1.6),999We only deals with $T(Dr)$ since these are the divisors that we will use in the proof of Theorem 1. However, after minor modification, the proof shows that any $T(r)$ parametrizes abelian surfaces with an extra special endomorphism. which may be well known to experts. From now on, $B$ denotes an abelian surface over some $\mathbb{Z}$-algebra with an $\mathfrak{a}$-polarization and $\mathcal{O}_{F}\subset\operatorname{End}(B)$. We fix a totally positive element in $\mathfrak{a}\cap\mathbb{Q}$ and this provides a fixed polarization on $B$ and we use this polarization to define the Rosati involution $(-)^{*}$ on $\operatorname{End}(B)\otimes\mathbb{Q}$.

Definition 2.1.4** (see also [KR]*Def. 1.2).**

An $s\in\operatorname{End}(B)$ is a special endomorphism if $a\circ s=s\circ a^{\prime}$ for all $a\in\mathcal{O}_{F}\subset\operatorname{End}(B)$ and $s^{*}=s$.

All the special endomorphisms of $B$ form a sub-$\mathbb{Z}$-module of $\operatorname{End}(B)$. It is well known that the rank of this submodule is at most 4. The following lemma recalls the discussion after [KR]*Def. 1.2.

Lemma 2.1.5**.**

For a special endomorphism $s$, there is a $Q(s)\in\mathbb{Z}$ such that $s\circ s=Q(s)\cdot\operatorname{Id}_{B}$ and hence also $\operatorname{Deg}s=Q(s)^{2}$. The function $Q$ is a positive definite quadratic form on the $\mathbb{Z}$-module of special endomorphisms of $B$.

The $\mathbb{Q}$-vector space in $\operatorname{End}(B)\otimes\mathbb{Q}$ generated by special endomorphisms depends on the choice of the polarization on $B$. However, there are natural isomorphisms between the $\mathbb{Q}$-vector spaces of special endomorphisms defined by different polarizations and the quadratic forms coincide up to multiplying by a fixed scalar determined by the polarizations.

Lemma 2.1.6**.**

The Hirzebruch–Zagier divisor $T(Dr)$ defined in 2.1.1 is the locus of $\mathcal{H}_{\overline{\mathbb{Q}}}$ where the abelian surface has a special endomorphism $s$ with $Q(s)=r\operatorname{Nm}\mathfrak{a}$. In particular, the degree of the endomorphism $s$ is $(r\operatorname{Nm}\mathfrak{a})^{2}$.

Proof.

We only need to check the statement over $\mathbb{C}$. Given a point in $\mathcal{H}(\mathbb{C})$ corresponding to $(z_{1},z_{2})\in\mathbb{H}^{2}$, it corresponds to an abelian surface $B$ with $B(\mathbb{C})=\mathbb{C}^{2}/\mathcal{O}_{F}(z_{1},z_{2})+(\mathfrak{a}\mathfrak{d}_{F})^{-1}$. The Riemann form $E$ on $H_{1}(B,\mathbb{Z})$ is, up to multiplying by a constant $\in\mathbb{Q}_{>0}$, the pull back of the standard alternating form ($\operatorname{Tr}_{F/\mathbb{Q}}$ of $\begin{pmatrix}0&1\\ -1&0\end{pmatrix}$) on $\mathcal{O}_{F}\oplus(\mathfrak{a}\mathfrak{d}_{F})^{-1}$ via the isomorphism $\mathcal{O}_{F}(z_{1},z_{2})+(\mathfrak{a}\mathfrak{d}_{F})^{-1}\cong\mathcal{O}_{F}\oplus(\mathfrak{a}\mathfrak{d}_{F})^{-1}$; see, for example, [vdG]*p. 208 and [BBK]*the discussion after Thm. 5.1. Any endomorphism of $B$ is given by the induced map on $B(\mathbb{C})$ of some $\mathbb{C}$-linear map on $\mathbb{C}^{2}$. For any endomorphism $s$, the condition $f\circ s=s\circ f^{\prime}$ for all $f\in\mathcal{O}_{F}$ is equivalent to the condition that the $\mathbb{C}$-linear map corresponding to $s$ is of the form $(1,0)\mapsto(0,\alpha^{\prime}z_{2}+\beta^{\prime}),(0,1)\mapsto(\alpha z_{1}+\beta)$ where $\alpha\in\mathfrak{a}\mathfrak{d}_{F},\beta\in\mathcal{O}_{F}$. This linear map gives rise to an endomorphism of $B$ if and only if the image of $(z_{1},z_{2})$ is in the period lattice. In other words, there exists $\nu\in\mathcal{O}_{F},\delta\in(\mathfrak{a}\mathfrak{d}_{F})^{-1}$ such that $(z_{2}(\alpha z_{1}+\beta),z_{1}(\alpha^{\prime}z_{2}+\beta^{\prime}))=(\nu z_{1}+\delta,\nu^{\prime}z_{2}+\delta^{\prime})$.

For every component of $T(Dr)$, there exists $M\in L$ in 2.1.1 satisfying $\det(M)=Dr\operatorname{Nm}\mathfrak{a}$. Write $M=\begin{pmatrix}a&\gamma\\ \gamma^{\prime}&b\end{pmatrix}$ where $a\in(D\operatorname{Nm}\mathfrak{a})\mathbb{Z},b\in\mathbb{Z},\gamma\in\mathfrak{a}$. Since $D\operatorname{Nm}\mathfrak{a}\mid\gamma\gamma^{\prime}$, we have $\frac{\gamma}{\sqrt{D}}\in\mathfrak{a}\subset\mathcal{O}_{F}$. Moreover $\frac{a}{\sqrt{D}}\in(\operatorname{Nm}\mathfrak{a})\mathfrak{d}_{F}\subset\mathfrak{a}\mathfrak{d}_{F}$. We take $\alpha=\frac{a}{\sqrt{D}}$ and $\beta=\frac{\gamma^{\prime}}{\sqrt{D}}$. Given $(z_{1},z_{2})\in\mathbb{H}^{2}$ such that $az_{1}z_{2}+\gamma z_{1}+\gamma^{\prime}z_{2}+b=0$, we have

[TABLE]

Hence $(1,0)\mapsto(0,\alpha^{\prime}z_{2}+\beta^{\prime}),(0,1)\mapsto(\alpha z_{1}+\beta)$ is an endomorphism $s$ with $f\circ s=s\circ f^{\prime}$ for all $f\in\mathcal{O}_{F}$.

To check that $s=s^{*}$, it is equivalent to check that for any $u,v\in H_{1}(B,\mathbb{Z})$, one has $E(su,v)=E(u,sv)$. Since we have already checked that $f\circ s=s\circ f^{\prime}$ for all $f\in\mathcal{O}_{F}$, one only needs to check the above equality for $u,v\in\{e_{1}=(z_{1},z_{2}),e_{2}=(1,1)\}\subset\mathbb{C}^{2}$. By construction, $se_{1}=\beta^{\prime}e_{1}-\frac{b}{\sqrt{D}}e_{2}$ and $se_{2}=\alpha e_{1}+\beta e_{2}$ and then we conclude by the fact that $\operatorname{Tr}_{F/\mathbb{Q}}\beta=\operatorname{Tr}_{F/\mathbb{Q}}\beta^{\prime},\,\operatorname{Tr}_{F/\mathbb{Q}}-\frac{b}{\sqrt{D}}=0$, and $\operatorname{Tr}_{F/\mathbb{Q}}\alpha=0$. Moreover, on $\mathbb{C}^{2}$, the composite $s\circ s=\frac{\det(M)}{D}\cdot\operatorname{Id}_{\mathbb{C}^{2}}$. Hence $s$ is a special endomorphism with $Q(s)=r\operatorname{Nm}\mathfrak{a}$.

On the other hand, the moduli space of $B$ with a special endomorphism is $1$-dimensional. Hence the two conditions $z_{2}(\alpha z_{1}+\beta)=\nu z_{1}+\delta$ and $z_{1}(\alpha^{\prime}z_{2}+\beta^{\prime})=\nu^{\prime}z_{2}+\delta^{\prime}$ are linearly dependent. Hence either $\alpha,\delta\in\mathbb{Q},\,\beta=-\nu^{\prime}$ or $\alpha\cdot\sqrt{D},\,\delta\cdot\sqrt{D}\in\mathbb{Q},\,\beta=\nu^{\prime}$. In the first case, we have

[TABLE]

In this case, there is no $(z_{1},z_{2})$ satisfying the above condition (see for example [vdG]*V.4). In the second case, take $a=\alpha\cdot\sqrt{D},b=-\delta\cdot\sqrt{D},\gamma=\beta^{\prime}\cdot\sqrt{D}$. Then $M=\begin{pmatrix}a&\gamma\\ \gamma^{\prime}&b\end{pmatrix}\in L$ and hence $[B]\in T(Dr)$. ∎

Let $\mathcal{T}(r)$ be the Zariski closure of $T(r)$ in $\overline{\mathcal{H}}^{\rm{tor}}$ over $\operatorname{Spec}\mathbb{Z}$.

Corollary 2.1.7**.**

Assume $T(Dr)$ is compact. Then for any finite place $v$, the points on $\mathcal{T}(Dr)_{\overline{\mathbb{F}}_{v}}$ correspond to abelian surfaces which are not absolutely simple. The abelian surfaces parametrized by $\mathcal{T}(Dr)$ admit a special endomorphism $s$ such that $Q(s)=r\operatorname{Nm}\mathfrak{a}$.

Proof.

Since $T(Dr)$ is compact, every point parametrized by it has potentially good reduction. For any given point parametrized by $\mathcal{T}(Dr)_{\mathbb{F}_{v}}$, let $[\mathcal{B}]$ be a lift of the point on $\mathcal{T}(Dr)$ over $\mathcal{O}_{K}$ for some number field $K$. By Lemma 2.1.6, the Néron–Severi rank of $\mathcal{B}_{K}$ is $3$ and hence the Néron–Severi rank of $\mathcal{B}_{\mathbb{F}_{v}}$ is at least $4$. By the classification of the endomorphism algebra of abelian varieties, we see that the Néron–Severi rank of a geometrically simple abelian surface is at most 2 and then conclude that $\mathcal{B}_{\mathbb{F}_{v}}$ is not geometrically simple. The last assertion follows from Lemma 2.1.6 and the fact that the canonical reduction map $\operatorname{End}(\mathcal{B}_{K})\rightarrow\operatorname{End}(\mathcal{B}_{\mathbb{F}_{v}})$ is injective. ∎

2.2. Hecke orbits

The idea of the proof of Theorem 1 is to show that the corresponding Hecke orbits of $[\mathcal{A}]$ intersect certain Hirzebruch–Zagier divisors at more and more places of $K$ as one varies over certain well-chosen Hecke operators.101010We view every Hecke orbit as a horizontal divisor over $K$, so the arithmetic intersection number is a sum over the finite places of $K$. In this subsection, we specify the Hecke orbits which we will use later.

2.2.1*.*

Recall that $p$ is a prime which splits completely in the narrow Hilbert class field of $F$ and $(p)=\mathfrak{p}\mathfrak{p}^{\prime}\subset\mathcal{O}_{F}$. Hence $\mathfrak{p}=(\lambda),\,\mathfrak{p}^{\prime}=(\lambda^{\prime})$ with $\lambda,\lambda^{\prime}\in F$ totally positive and $\lambda\lambda^{\prime}=p$. Let $G^{\operatorname{ad}}_{1}$ be the adjoint group of $G_{1}=\operatorname{Res}^{F}_{\mathbb{Q}}\operatorname{GL}_{2}$. We denote by $G^{\operatorname{ad}}_{1}(\mathbb{R})_{1}$ the image of $G_{1}(\mathbb{R})$ in $G^{\operatorname{ad}}_{1}(\mathbb{R})$ and let $G^{\operatorname{ad}}_{1}(\mathbb{Q})_{1}$ be $G^{\operatorname{ad}}_{1}(\mathbb{Q})\cap G^{\operatorname{ad}}_{1}(\mathbb{R})_{1}$. Since $\lambda$ is totally positive, the image of the diagonal matrix $g_{\mathfrak{p}}:=\operatorname{diag}(1,\lambda)$ under $G_{1}\rightarrow G_{1}^{\operatorname{ad}}$ lies in $G_{1}^{\operatorname{ad}}(\mathbb{Q})_{1}$, so it induces a correspondence $T_{\mathfrak{p}}$ on $\mathcal{H}_{\mathbb{Z}[1/p]}$ (defined in [D77]; see also [Kisin]*sec. 3.2).111111The definition of $T_{\mathfrak{p}}$ depends on the choice of $\lambda$ if we do not pass to a certain finite quotient of $\mathcal{H}$. However, there are only finitely many choices: let $U$ be the unit group of $\mathcal{O}_{F}$ and $U^{+}$ the subgroup of totally positive units; then the number of choices is $\#U^{+}/U^{2}$. Hence we will not specify our choice of $\lambda$ as it does not affect the arguments in this paper. The following lemma provides a moduli interpretation of $T_{\mathfrak{p}}$.

Lemma 2.2.2**.**

We have $\#T_{\mathfrak{p}}[A]=p+1$. Over $\mathbb{Z}[1/p]$, the set $T_{\mathfrak{p}}[\mathcal{A}]$ consists of those points on $\mathcal{H}$ that correspond to a quotient of $A$ by an order $p$ subgroup in $A[\mathfrak{p}]$ endowed the with induced $\mathcal{O}_{F}$-structure and a suitable $\mathfrak{a}$-polarization.121212The choice of $\lambda$ determines the polarization.

Proof.

The first assertion follows from the definitions: $\#T_{\mathfrak{p}}[A]=\#\Gamma/(g_{\mathfrak{p}}^{-1}\Gamma g_{\mathfrak{p}}\cap\Gamma)=\#\mathbb{P}^{1}(\mathbb{F}_{p})=p+1.$

For the second assertion, since the correspondence $T_{\mathfrak{p}}$ is étale, we only need to show the same statement for $T_{\mathfrak{p}}[A]$ over $\mathbb{C}$ for a fixed embedding of $K$ into $\mathbb{C}$. On the one hand, as $\mathcal{O}_{F}$ acts on $A[\mathfrak{p}]$ via $\mathcal{O}_{F}/\mathfrak{p}\cong\mathbb{F}_{p}$ and $\mathbb{Z}\subset\mathcal{O}_{F}$ surjects onto $\mathbb{F}_{p}$, any subgroup of $A[\mathfrak{p}]$ is $\mathcal{O}_{F}$-invariant. Therefore, any quotient of $A$ by an order $p$ subgroup of $A[\mathfrak{p}]$ has the induced $\mathcal{O}_{F}$-structure. Moreover, by [BBK]*Lem. 5.9, any such quotient of $A$ is $\mathcal{O}_{F}$-polarizable.

On the other hand, by [vdG]*p. 208, if a point $(z_{1},z_{2})\in\mathbb{H}^{2}$ corresponds to $A_{\mathbb{C}}$, then $A(\mathbb{C})$ is isomorphic to $\mathbb{C}^{2}/(\mathcal{O}_{F}(z_{1},z_{2})+(\mathfrak{a}\mathfrak{d}_{F})^{-1})$. Then $\operatorname{diag}(1,\lambda)z$ corresponds to $\mathbb{C}^{2}/(\mathcal{O}_{F}(z_{1}/\lambda,z_{2}/\lambda^{\prime})+(\mathfrak{a}\mathfrak{d}_{F})^{-1})$, so the kernel of the isogeny defined by $\operatorname{diag}(1,\lambda)$ is contained in $\ker(\lambda)=A[\mathfrak{p}]$. On the quotient $\mathbb{C}^{2}/(\mathcal{O}_{F}(z_{1}/\lambda,z_{2}/\lambda^{\prime})+(\mathfrak{a}\mathfrak{d}_{F})^{-1})$, the $\mathcal{O}_{F}$-structure is the induced one and the choice of $\lambda$ determines the $\mathcal{O}_{F}$-polarization. Since the other elements in $T_{\mathfrak{p}}$ differ from $\operatorname{diag}(1,\lambda)$ by the action of some element in $\Gamma$ (on $\mathbb{H}^{2}$ and on $A[\mathfrak{p}]$), the set $T_{\mathfrak{p}}$ injects into the set of order $p$ subgroups of $A[\mathfrak{p}]$. Since both sets have cardinality $p+1$, this is in fact a bijection. ∎

3. Archimedean places and equidistribution of Hecke orbits

Let $\Psi$ be a meromorphic Hilbert modular form of parallel weight $k$ over $\overline{\mathbb{Q}}$ such that $\operatorname{Div}(\Psi)$ in $\mathcal{H}_{\mathbb{Q}}$ is given by $\sum_{c\in\mathbb{I}}c_{r}T(r)$, where $k\in\mathbb{N}_{>0}$, $\mathbb{I}$ is a finite set, $c_{r}\in\mathbb{Z}$, $T(r)$ is compact and $D|r$ for all $r\in\mathbb{I}$. In the proof of Theorem 1, we will use Lemma 5.1.1 to construct such meromorphic Hilbert modular form. We assume that $\operatorname{End}(A_{\overline{K}})=\mathcal{O}_{F}$ and hence $T_{\mathfrak{p}}([A])$ does not intersect $T(r)$ in characteristic zero. This is the key case in the proof of Theorem 1. Fix an embedding $\sigma:\overline{K}\rightarrow\mathbb{C}$. Given an abelian surface $B$ corresponding to a point $[B]$ on $\mathcal{H}_{\overline{\mathbb{Q}}}$, we use $\sigma([B])$ to denote the corresponding $\mathbb{C}$-point on $\mathcal{H}$ via base change by $\sigma$.

We set $||\Psi(z)||_{\rm{Pet}}=|\Psi(z_{1},z_{2})(\Im z_{1})^{k/2}(\Im z_{2})^{k/2}|$, where $z=(z_{1},z_{2})\in\mathbb{H}^{2}$. This norm is well-defined outside (the preimage of) $\bigcup_{r\in\mathbb{I}}T(r)$ and invariant under $\Gamma$ (defined in 2.1.1) and hence we will also view $||\Psi||_{\rm{Pet}}$ as a function on $\mathcal{H}_{\mathbb{C}}\backslash\bigcup_{r\in\mathbb{I}}T(r)$. The real analytic function $-\log||\Psi||_{\rm{Pet}}$ is a Green function for $\sum_{c\in\mathbb{I}}c_{r}T(r)$ and endows it with the structure of an arithmetic divisor $\widehat{\sum_{c\in\mathbb{I}}c_{r}T(r)}$.

The goal of this section is to show that for most $p$ as in 2.2.1, the archimedean contribution (in the height of $T_{\mathfrak{p}}([\mathcal{A}])$ with respect to the arithmetic divisor $\widehat{\sum_{c\in\mathbb{I}}c_{r}T(r)}$; we will discuss this height in detail in §5.1)

[TABLE]

The equidistribution theorems for Hecke orbits on Shimura varieties reduces this goal to a suitable upper bound for $-\log||\Psi(\sigma([B]))||_{\mathrm{Pet}}$ for all $[B]\in T_{\mathfrak{p}}[A]$ which is valid for most $p$. The proofs are inspired by Charles’ treatment in the case of the modular curve. Recall that all $T(r)$ ($r\in\mathbb{I}$) are compact, so we avoid dealing with estimates around the cusps.

Throughout this section, $p$ is a prime as in 2.2.1 and $N_{i}(*),C_{i}(*)$ denote constants only depending on $*$. In particular, if there is no $(*)$, it means an absolute constant. After defining the constants, we may abbreviate $N_{i}(*),C_{i}(*)$ as $N_{i},C_{i}$. Given $\eta\in F$, we use $|\eta|<C$ to mean that for any real embedding $\iota:F\rightarrow\mathbb{R}$, the absolute value $|\eta|_{\iota}<C$. We also use $|\cdot|$ to denote the absolute value on $\mathbb{C}$.

3.1. An upper bound of the values of Green function on Hecke orbits

3.1.1*.*

Let $\mathcal{F}\subset\mathbb{H}^{2}$ be the fundamental domain for $\Gamma$ described in [vdG]*I.3 and $\overline{\mathcal{F}}$ its closure (with respect to the complex analytic topology) in $\mathbb{H}^{2}$ (that is, the cusps of $\Gamma\backslash\mathbb{H}^{2}$ are not included). Let $\Omega\subset\overline{\mathcal{F}}$ be a compact domain containing the preimage of $\bigcup_{r\in\mathbb{I}}T(r)$ in $\operatorname{Div}(\Psi)$. Then there exists $C_{0}\in\mathbb{R}_{>0}$ such that for any $(z_{1},z_{2})=(x_{1}+\sqrt{-1}y_{1},x_{2}+\sqrt{-1}y_{2})\in\Omega$, we have $|x_{i}|<C_{0}$ and $C^{-1}_{0}<y_{i}<C_{0}$.

For any $\overline{\mathbb{Q}}$-point $[B]$ in $\mathcal{H}$, we use $z(B)=(z_{1}(B),z_{2}(B))=(x_{1}(B)+\sqrt{-1}y_{1}(B),x_{2}(B),\sqrt{-1}y_{2}(B))$ to denote the preimage of $\sigma([B])$ in $\mathcal{F}$.

Let $G$ be the pull back to $\mathbb{H}^{2}$ of the Green function $-\log||\Psi||_{\text{Pet}}$ of $\sum c_{r}T(r)$. There are only finitely many components of the preimage of $T(r)$ in $\Omega$ and for each component, we pick $(a,b,\gamma)$ such that $\begin{pmatrix}a&\gamma\\ \gamma^{\prime}&b\end{pmatrix}\in L$ with $ab-\gamma\gamma^{\prime}=r\operatorname{Nm}\mathfrak{a}$ as in 2.1.1 such that this component is defined by $az_{1}z_{2}+\gamma z_{1}+\gamma^{\prime}z_{2}+b=0$. We use $\mathcal{M}_{\Omega,r}$ to denote this finite set of $(a,b,\gamma)$. Then by the definition of the Green function, we have that $G+\sum_{r\in\mathbb{I}}c_{r}\sum_{(a,b,\gamma)\in\mathcal{M}_{\Omega,r}}\log|az_{1}z_{2}+\gamma z_{1}+\gamma^{\prime}z_{2}+b|$ is a real analytic function on $\overline{\mathcal{F}}$.

The goal of this subsection is to show that for most $p$, one has that

[TABLE]

Proposition 3.1.2**.**

Let $(a,b,\gamma)$ be a fixed triple in $\mathcal{M}_{\Omega,r}$. Given $C_{1}>23$ and $\epsilon_{3}>0$, there is an $N_{0}(\epsilon_{3},C_{1})>0$ such that for every $N>N_{0}(\epsilon_{3},C_{1})$, the number of primes in $[N^{1/2},N]$ for which there exists some $[B]\in T_{\mathfrak{p}}(\sigma[A])$ such that

[TABLE]

is at most $\epsilon_{3}\#\{\text{primes}\in[N^{1/2},N]\}$.

We extend the idea in [Ch] of relating bad primes and degrees of homomorphisms between well-chosen CM elliptic curves to the setting of Hilbert modular surfaces by using the theory of special endomorphisms. A point $[B]$ on $\mathcal{H}_{\mathbb{Q}}$ is called special if there exist $T(n_{1})$ and $T(n_{2})$, $n_{1},n_{2}\in\mathbb{N}$, $n_{1}n_{2}\notin(\mathbb{N})^{2}$ such that $[B]\in T(n_{1})\cap T(n_{2})$. If $[B]$ is special, then $B$ has complex multiplication. We construct a special point $[A_{\mathrm{CM}}]$ on $\mathcal{H}_{\mathbb{C}}$ close to $\sigma([A])$ and show that if some point in $T_{\mathfrak{p}}(\sigma([A]))$ is close to $\operatorname{Div}(\Psi)=\sum_{r\in\mathbb{I}}c_{r}T(r)$, then $A_{\mathrm{CM}}$ has a special endomorphism of certain degree. Proposition 3.1.2 then follows after analysis of the possible degree of special endomorphisms of $A_{\mathrm{CM}}$. In what follows, we will not specify the dependence of the constants $C_{i}$ in this subsection on the fixed triple $(a,b,\gamma)$.

The following lemma shows that if there exists $[B]\in T_{\mathfrak{p}}(\sigma([A]))$ which is close to $T(r)$, then $\sigma([A])$ is close to $T(pr)$.

Lemma 3.1.3**.**

If there exists $[B]\in T_{\mathfrak{p}}[A]$ such that $|az_{1}(B)z_{2}(B)+\lambda z_{1}(B)+\lambda^{\prime}z_{2}(B)+b|<p^{-C_{1}}$, then there exist $m\in(D\operatorname{Nm}\mathfrak{a})\mathbb{Z},l\in\mathbb{Z}$ and $\eta\in\mathfrak{a}$ such that $ml-\operatorname{Nm}(\eta)=rp\operatorname{Nm}\mathfrak{a}$ and $|mz_{1}(A)z_{2}(A)+\eta z_{1}(A)+\eta^{\prime}z_{2}(A)+l|<p^{-C_{1}}.$ Moreover, we have $|m|,|l|,|\eta|<C_{5}p$.

Proof.

As in 2.2.1, we write $\mathfrak{p}=(\lambda)$ and after multiplying $\lambda$ by an element in $(\mathcal{O}^{\times})^{2}$, we may assume that $C^{-1}_{6}\sqrt{p}<|\lambda|<C_{6}\sqrt{p}$. We may also assume $z(B)\in\Omega$ (this can be done by letting $N$ be large enough).

Let $U=\begin{pmatrix}u_{11}&u_{12}\\ u_{21}&u_{22}\end{pmatrix}\in\begin{pmatrix}1&0\\ 0&\lambda\end{pmatrix}\cdot\Gamma$ be the matrix that maps $z(A)$ to $z(B)$. The set $\Gamma$ acts on $V$ (in 2.1.1) via $g.M=(g^{\prime})^{t}Mg$ and this action preserves $L$. Let $\begin{pmatrix}m&\eta\\ \eta^{\prime}&l\end{pmatrix}=(U^{\prime})^{t}\begin{pmatrix}a&\gamma\\ \gamma^{\prime}&b\end{pmatrix}U\in L$. Then $ml-\eta\eta^{\prime}=\det(U^{\prime})(ab-\gamma\gamma)\det(U)=rp\operatorname{Nm}\mathfrak{a}$ and

[TABLE]

This proves the first assertion.

For the second assertion, we first bound $|u_{ij}|$. Consider the real embedding of $F$ corresponding to the first coordinate of $\mathbb{H}^{2}$ and we will still use $u_{ij}$ to denote its image under this embedding. By definition of $U$, we have $\displaystyle z_{1}(B)=\frac{u_{11}z_{1}(A)+u_{12}}{u_{21}z_{1}(A)+u_{22}}$ and hence $\displaystyle y_{1}(B)=\frac{\lambda y_{1}(A)}{|u_{21}z_{1}(A)+u_{22}|^{2}}$. Since $y_{1}(B)>C_{0}^{-1}$ and $y_{1}(A)<C_{0}$, we have $|u_{21}z_{1}(A)+u_{22}|<C_{0}^{2}C_{6}\sqrt{p}$. Consider the imaginary part of $u_{21}z_{1}(A)+u_{22}$ and notice that $y_{1}(A)>C_{0}^{-1}$. Thus, we have $|u_{21}|<C_{0}^{3}C_{6}\sqrt{p}$. By considering the real part, we obtain

[TABLE]

On the other hand, using the bounds of $|u_{21}|,|u_{22}|$, we have

[TABLE]

Hence we obtain

[TABLE]

The same argument works for the other embedding of $F\rightarrow\mathbb{R}$ by studying $z_{2}(B),z_{2}(A)$. The bounds of $|m|,|l|,|\eta|$ follow from the fact that $|u_{ij}|$ is bounded by $O(p^{1/2})$. ∎

3.1.4*.*

The following lemma shows that if two Hecke orbits of $\sigma([A])$ satisfy the assumption of Lemma 3.1.3, then $\sigma([A])$ is close to a special point on $\mathcal{H}_{\mathbb{C}}$. Recall that for a special point $[B]\in\mathcal{H}(\mathbb{C})$, one defines a quadratic form $Q$, up to $\operatorname{SL}_{2}(\mathbb{Z})$-equivalence, as follows (see, for example, [HZ]*1.1, [vdG]*V.4). Let $L_{[B]}$ be the sub lattice of $L$ in 2.1.1 such that for any $v\in L_{[B]}$, one has $(z_{1}(B)\quad 1)v(z_{2}(B)\quad 1)^{t}=0$. The lattice $L_{[B]}$ is of rank two and equipped with a natural orientation. The restriction of the quadratic form on $L$ to the rank two lattice $L_{[B]}$ is positive definite and it coincides, up to a constant, with the quadratic form on the $\mathbb{Z}$-module of special endomorphisms of $B$. By choosing an oriented basis, one obtains a positive definite binary integral quadratic form $Q$.

Lemma 3.1.5**.**

Assume that $N$ is large enough131313In the proof, we give a constant $N_{3}(C_{1})$ such that being large enough means $N>N_{3}$. and that for primes $p_{1},p_{2}\in[N^{1/2},N],p_{1}<p_{2}$, there exist $[B]\in T_{\mathfrak{p}_{1}}[A],[B^{\prime}]\in T_{\mathfrak{p}_{2}}[A]$ satisfying $|az_{1}(B)z_{2}(B)+\lambda z_{1}(B)+\lambda^{\prime}z_{2}(B)+b|<p_{1}^{-C_{1}}$ and $|az_{1}(B^{\prime})z_{2}(B^{\prime})+\lambda z_{1}(B^{\prime})+\lambda^{\prime}z_{2}(B^{\prime})+b|<p_{2}^{-C_{1}}$. Then there exists a special point $[A_{\mathrm{CM}}]$ on $\mathcal{H}_{\mathbb{C}}$ such that $|z(A)-z(A_{\mathrm{CM}})|<C_{8}N^{-C_{1}+7}$ and the integer coefficient binary quadratic form $Q_{N}$ associated to $[A_{\mathrm{CM}}]$ represents $p_{1}r$ and $p_{2}r$.

Proof.

By Lemma 3.1.3, there exist $m_{i}\in(D\operatorname{Nm}\mathfrak{a})\mathbb{Z},l_{i}\in\mathbb{Z},\eta_{i}\in\mathfrak{a}$ with $|m_{i}|,|l_{i}|,|\eta_{i}|<C_{5}p_{i}$ such that $m_{i}l_{i}-\operatorname{Nm}\eta_{i}=p_{i}r\operatorname{Nm}\mathfrak{a}$ and

[TABLE]

We first show that $|\eta^{\prime}_{i}+m_{i}z_{1}(A)|$ is bounded below by a constant. Indeed, if $m_{i}\neq 0$, one has $|\eta^{\prime}_{i}+m_{i}z_{1}(A)|\geq|m_{i}|y_{1}(A)\geq C_{0}^{-1}$ and if $m_{i}=0$, one has $|\eta^{\prime}_{i}+m_{i}z_{1}(A)|=|\eta^{\prime}_{i}|=\operatorname{Nm}\eta/|\eta^{\prime}|=p_{i}r/|\eta^{\prime}|\geq p_{i}r/|\eta_{i}|\geq r/C_{5}$.

Therefore,

[TABLE]

Let $f(z)$ be the $\mathcal{O}$-coefficient quadratic polynomial in $z$ given by $(\eta_{1}z+l_{1})(\eta^{\prime}_{2}+m_{2}z)-(\eta_{2}z+l_{2})(\eta^{\prime}_{1}+m_{1}z)$. We first show that the leading coefficient $\eta_{1}m_{2}-\eta_{2}m_{1}\neq 0$. If not, then $f(z)=(m_{2}l_{1}-m_{1}l_{2})z+l_{1}\eta_{2}^{\prime}-l_{2}\eta_{1}^{\prime}$ and $m_{2}l_{1}-m_{1}l_{2}\neq 0$ (otherwise, one would have $p_{1}=p_{2}$). In particular, $|m_{2}l_{1}-m_{1}l_{2}|\geq 1$. Hence $|f(z_{1}(A))|$ is bounded below by its imaginary part $|m_{2}l_{1}-m_{1}l_{2}|y_{1}(A)\geq C_{0}^{-1}$. On the other hand, since $|\eta^{\prime}_{i}+m_{i}z_{1}(A)|\leq C_{5}p_{i}(1+2C_{0})$, we have

[TABLE]

This leads to a contradiction, when $N^{\frac{C_{1}}{2}-2}>2C_{0}(C_{5}+2C_{0}C_{5})^{2}C_{9}$.

Now we show that $f$ has two complex roots. Let $\alpha,\beta$ be the two roots of $f$, then we have $f(z)=(\eta_{1}m_{2}-\eta_{2}m_{1})(z-\alpha)(z-\beta)$. Then by assumption, we have

[TABLE]

Since $m_{i}\in\mathbb{Z}$ and $\eta_{i}\in\mathcal{O}$, one has $|\eta_{1}m_{2}-\eta_{2}m_{1}|\geq|\eta_{1}^{\prime}m_{2}-\eta_{2}^{\prime}m_{1}|^{-1}\geq(2C_{5}^{2}p_{1}p_{2})^{-1}$. Moreover, $|\eta^{\prime}_{i}+m_{i}z_{1}(A)|\leq C_{5}p_{i}(1+2C_{0})$. If $\alpha,\beta$ were real, then $|z_{1}(A)-\alpha|,|z_{1}(A)-\beta|\geq C_{0}^{-1}$ and one gets a contradiction when $N^{\frac{C_{1}}{2}-4}>4C_{0}^{2}C_{5}^{4}C_{9}(1+2C_{0})^{2}$. Then we may assume that $\alpha$ has positive imaginary part.

We define $A_{\mathrm{CM}}$ to be the abelian surface such that $z_{1}(A_{\mathrm{CM}})=\alpha$ and $\displaystyle z_{2}(A_{CM})=-\frac{\eta_{1}z_{1}(A_{CM})+l_{1}}{\eta^{\prime}_{1}+m_{1}z_{1}(A_{CM})}$. Then $A_{{\mathrm{CM}}}$ lies on both $T(p_{1}r)$ and $T(p_{2}r)$ (so $[A_{{\mathrm{CM}}}]$ is a special point). In other words, the integer coefficient binary quadratic form associated to $[A_{\mathrm{CM}}]$ represents $p_{1}r$ and $p_{2}r$.

Since $\beta$ has negative imaginary part, $|z_{1}(A)-\beta|\geq C_{0}^{-1}$. Since $p_{2}\leq\sqrt{p_{1}}$, the above discussion shows that

[TABLE]

Moreover,

[TABLE]

where $C_{10}^{-1}$ is the lower bound of $y_{1}(A_{\mathrm{CM}})$ given by $\min\{C_{0}^{-1}-4C_{0}C_{5}^{4}C_{9}(1+2C_{0})^{2}p_{1}^{-C_{1}+6},r/C_{5}\})$. Hence we take $C_{8}$ large enough and conclude that $z(A)$ is close to $z(A_{\mathrm{CM}})$. ∎

The following lemma shows that the special point $[A_{\mathrm{CM}}]$ constructed above is unique when $N$ is large enough.

Lemma 3.1.6**.**

Assume that $N$ is large enough141414In the proof, we give a constant $N_{4}(C_{1})$ such that being large enough means $N>N_{4}$. and that for $p_{i}\in[N^{1/2},N]$ with $i=1,2,3,4$, $p_{1}\neq p_{2}$, $p_{3}\neq p_{4}$, there exists $[B_{i}]\in T_{{\mathfrak{p}}_{i}}([A])$ such that $|az_{1}(B_{i})z_{2}(B_{i})+\lambda z_{1}(B_{i})+\lambda^{\prime}z_{2}(B_{i})+b|<p_{i}^{-C_{1}}$. Let $[A_{1}],[A_{2}]$ be two special points constructed as in Lemma 3.1.5 by using the assumption on $p_{1},p_{2}$ and $p_{3},p_{4}$. Then $[A_{1}]=[A_{2}]$. More precisely, if $|z(A_{1})-z(A_{2})|\leq 2C_{8}N^{-C_{1}+7}$, then $z(A_{1})=z(A_{2})$.

Proof.

Let $f_{1}(z),f_{2}(z)\in\mathcal{O}[z]$ be the quadratic equations defining $z_{1}(A_{1}),z_{1}(A_{2})$ in the proof of Lemma 3.1.5. Let $\alpha_{1},\alpha_{2}\in F$ be the leading coefficients of $f_{1},f_{2}$ and $-\Delta_{1},-\Delta_{2}$ the discriminants. Since $f_{i}$ has two complex roots for both embeddings of $F$, one has that $\Delta_{i}$ is totally real. Given a real embedding of $F$, we may assume both $\alpha_{1},\alpha_{2}$ are positive with respect to this embedding.

By the definition of $f_{i}$ and Lemma 3.1.3, we have $|\alpha_{i}|\leq 2C_{5}^{2}N^{2}$ and $|\Delta_{i}|\leq C_{11}N^{4}$. Moreover, since $\alpha_{i},\Delta_{i}\in\mathcal{O}$, we have the nonzero $|\operatorname{Nm}(\alpha_{2}^{2}\Delta_{1}-\alpha_{1}^{2}\Delta_{2})|\geq 1$ and hence

[TABLE]

Putting these inequalities together, we obtain

[TABLE]

This contradicts our assumption when $N^{C_{1}-23}>2C_{8}C_{12}^{-1}$. ∎

Corollary 3.1.7**.**

Assume that $N$ is large enough as above and that $A$ satisfies the assumption in Lemma 3.1.5. For any $p_{3}\in[\sqrt{N},N]$ such that there exists $[B^{\prime\prime}]\in T_{\mathfrak{p}_{3}}[A]$ satisfying $|az_{1}(B^{\prime\prime})z_{2}(B^{\prime\prime})+\lambda z_{1}(B^{\prime\prime})+\lambda^{\prime}z_{2}(B^{\prime\prime})+b|<p_{3}^{-C_{1}}$, the quadratic form $Q_{N}$ in Lemma 3.1.5 represents $p_{3}r$.

Proof.

By Lemma 3.1.5, we construct special points $[A_{1}]$ by using $p_{1},p_{2}$ and $[A_{2}]$ by using $p_{1},p_{3}$. By Lemma 3.1.6, we have $[A_{1}]=[A_{2}]$ and hence they have the same quadratic form $Q_{N}$. Since $[A_{2}]$ lies on $T(p_{3}r)$, then $Q_{N}$ represents $p_{3}r$. ∎

Lemma 3.1.8**.**

Fix $C_{1}>23$ and let $\Delta_{N}$ denote the discriminant of $Q_{N}$ in Lemma 3.1.5. As $N\rightarrow\infty$, we have $|\Delta_{N}|\rightarrow\infty$.

Proof.

Fix a bound $X$ of $|\Delta_{N}|$, then there are only finitely many equivalent classes of integral binary quadratic forms of discriminant $\leq X$. For each class, [HZ]*Thm. 1151515Although in [HZ], they assume that $D$ is a prime, their method still works in general. See for example [vdG]*V.6. shows that are only finitely many special points corresponding to the given class of quadratic forms. As $N\rightarrow\infty$, the CM approximation $[A_{{\mathrm{CM}},N}]$ is closer to $\sigma[A]$ and hence $|\Delta_{N}|$ cannot be bounded. ∎

Proof of Proposition 3.1.2.

Let $p_{1},p_{2}$ be the smallest primes in $[\sqrt{N},N]$ such that there exists $B\in T_{\mathfrak{p}_{i}}A$ such that $|az_{1}(B)z_{2}(B)+\lambda z_{1}(B)+\lambda^{\prime}z_{2}(B)+b|<p_{i}^{-C_{1}}$. Then by Lemma 3.1.5 and Corollary 3.1.7, we obtain a quadratic form $Q_{N}$ associated to a special point $[A_{{\mathrm{CM}},N}]$ which represents $p_{3}r$ for any prime $p_{3}$ in $[\sqrt{N},N]$ which satisfies the condition that there exists $[B^{\prime\prime}]\in T_{\mathfrak{p}_{3}}[A]$ satisfying $|az_{1}(B^{\prime\prime})z_{2}(B^{\prime\prime})+\lambda z_{1}(B^{\prime\prime})+\lambda^{\prime}z_{2}(B^{\prime\prime})+b|<p_{3}^{-C_{1}}$. There exists a positive definite integral binary quadratic form $Q_{N}^{\prime}$ such that a prime $p$ is represented by $Q_{N}^{\prime}$ if and only if $pr$ is represented by $Q_{N}$. One also has that the absolute value of the discriminant $\Delta_{N}^{\prime}$ of $Q_{N}^{\prime}$ is at least $|\Delta_{N}|/r^{2}$.

It remains to show that the density $\displaystyle\frac{\{\text{prime }p\in[N^{1/2},N],p\textrm{ is represented by }Q^{\prime}_{N}\}}{\{p\in[N^{1/2},N]\}}\rightarrow 0$ as $N\rightarrow\infty$. When $|\Delta_{N}^{\prime}|\leq(\log N)^{4}$, [TZ, Corollary 1.3] shows that $\displaystyle\{\text{prime }p\in[N^{1/2},N],p\textrm{ is represented by }Q_{N}\}\ll\frac{\operatorname{Li}(N)}{h_{N}}$, where $h_{N}$ is the number of $\operatorname{SL}_{2}(\mathbb{Z})$-equivalence classes of primitive positive definite integral binary quadratic forms of discriminant $\Delta_{N}^{\prime}$. Since $\Delta_{N}^{\prime}\rightarrow-\infty$ by Lemma 3.1.8, one has $h_{N}\rightarrow\infty$. When $|\Delta_{N}^{\prime}|>(\log N)^{4}$, [Ch, Lemma 5.2] shows that

[TABLE]

We get the desired property by putting these two cases together. ∎

3.2. From equidistribution to an upper bound of archimedean contribution

This is the main theorem of this section. We use Proposition 3.1.2 and equidistribution theorem for Hecke orbits to show that for most $p$, the archimedean contribution in the height of $T_{\mathfrak{p}}([\mathcal{A}])$ is $o(p\log p)$.

Theorem 3.2.1**.**

For any $\epsilon_{1},\epsilon_{2}>0$, there is an $N(\epsilon_{1},\epsilon_{2})>0$ such that for every $N>N(\epsilon_{1},\epsilon_{2})$, the number of primes in the interval $[N^{1/2},N]$ for which

[TABLE]

is at most $\epsilon_{2}\#\{\ell\in[N^{1/2},N]\text{ prime}\}$.

Proof.

Notation as in 3.1.1. We first show that a fixed triple $(a,b,\gamma)$,

[TABLE]

For any $\epsilon>0$, by the equidistribution theorem of Hecke orbits (see for example [COU]), there exist constants $N_{1}(\epsilon,C_{1})>0$ and $C_{3}(\epsilon,C_{1})<0$ such that for any $p>N_{1}$,

[TABLE]

Let $\mathbb{I}^{\prime}=\{r\in\mathbb{I}\mid c(r)>0\}$ and $M=\sum_{r\in\mathbb{I}^{\prime}}\#\mathcal{M}_{\Omega,r}$. Taking $\epsilon_{3}=\epsilon_{2}/M$ and applying Proposition 3.1.2 for all triples $(a,b,\gamma)\in\bigcup_{r:c(r)>0}\mathcal{M}_{\Omega,r}$, we have that, for every $N>N_{2}(\epsilon_{2},C_{1})\colonequals\max_{(a,b,\gamma)}\{N_{0}(a,b,\gamma,\epsilon_{3},C_{1}),N_{1}\}$,

[TABLE]

holds for primes $p\in[N^{1/2},N]$ outside a set $\mathbb{B}_{N}$ of density $\epsilon_{2}$ (this set is the union of the exceptional sets for all $(a,b,\gamma)\in\cup_{r:c(r)>0}\mathcal{M}_{\Omega,r}$).

Let $\phi$ be a smooth function which is $1$ in $\Omega$ with compact support in $\overline{\mathcal{F}}$. Then by 3.1.1, the function $f=G+\sum_{r\in\mathbb{I}}c(r)\sum_{(a,b,\gamma)\in\mathcal{M}_{\Omega,r}}\phi(z)\log|az_{1}z_{2}+\gamma z_{1}+\gamma^{\prime}z_{2}+b|$ is smooth on $\overline{\mathcal{F}}$. Since $G$ and hence $f$ go to $-\infty$ as $y_{1}y_{2}$ goes to $\infty$, we see that $f$ is bounded above on $\overline{\mathcal{F}}$. On the other hand, since $\phi(z)$ has compact support, $\phi(z)\log|az_{1}z_{2}+\gamma z_{1}+\gamma^{\prime}z_{2}+b|$ is also bounded above. Therefore, since $\#T_{\mathfrak{p}}[A]=p+1$, we have

[TABLE]

Take $\epsilon=\epsilon_{1}/(2M)$, then we have, for $N>N_{2}$, for $p\in[N^{1/2},N]\backslash\mathbb{B}_{N}$ ,

[TABLE]

Then by taking $N(\epsilon_{1},\epsilon_{2})>N_{2}$ large enough so that $C_{4}N(\epsilon_{1},\epsilon_{2})^{1/2}<(\epsilon_{1}N(\epsilon_{1},\epsilon_{2})^{1/2}\log N(\epsilon_{1},\epsilon_{2}))/4$, the theorem follows. ∎

4. Special endomorphisms and contributions at finite places

In this section, we will bound the local intersection multiplicities of $(T_{\mathfrak{p}}([\mathcal{A}]),\mathcal{T}(r))$ at non-archimedean places for $r\in\mathbb{I}$, where $\mathbb{I}$ is a fixed finite set such that $D|r$ and $\mathcal{T}(r)$ is compact in $\mathcal{H}$ for all $r\in\mathbb{I}$. The set $\mathbb{I}$ will be chosen by Lemma 5.1.1. Throughout, $\ell$ denotes a prime, and $B,B^{\prime},\mathcal{B},\mathcal{B}^{\prime}$ denote abelian surfaces with $\mathcal{O}_{F}$-multiplication and $\mathfrak{a}$-polarization. We fix a finite place $v$ of a number field $K$ over $\ell$. Recall that $e_{v}$ is the ramification degree of $K$ at $v$. The abelian surfaces may be defined over $K,\mathcal{O}_{K}$, $\mathcal{O}_{K_{v}}$, or $\mathcal{O}_{K_{v}}/v^{n}$. Recall that we use $\mathcal{B},\mathcal{B}^{\prime}$ to denote abelian surfaces defined over $\mathcal{O}_{K_{v}}$ and we use $\mathcal{B}_{v,n},\mathcal{B}^{\prime}_{v,n}$ to denote their reduction modulo $v^{n}$. We will use $\mathcal{B}_{\ell,n}$ and $\mathcal{B}^{\prime}_{\ell,n}$ to denote surfaces over $\mathcal{O}_{K_{v}}/\ell^{n}$ (note that the abelian varieties are defined modulo $\ell^{n}$, not $v^{n}$) which do not a priori come with lifts to $\mathcal{O}_{K_{v}}$. Let $M_{v,n}$ and $M_{\ell,n}$ denote the module of special endomorphisms of $\mathcal{B}_{v,n}$ and $\mathcal{B}_{\ell,n}$ respectively. Finally, we let $\Lambda_{v}=\operatorname{End}(\mathcal{B}[\ell^{\infty}])\cap M_{v,1}\otimes\mathbb{Z}_{\ell}$, where the intersection takes place in $\operatorname{End}(\mathcal{B}_{v,1})\otimes\mathbb{Z}_{\ell}=\operatorname{End}(\mathcal{B}_{v,1}[\ell^{\infty}])$. We call the $\mathbb{Z}_{\ell}$-module $\Lambda_{v}$ the set of special endomorphisms of $\mathcal{B}[\ell^{\infty}]$

4.1. Deformation theory

The following result is crucial to bounding the local intersections:

Theorem 4.1.1**.**

Let $m$ be the $\mathbb{Z}$-rank of $M_{v,1}$ and $m^{\prime}$ be the $\mathbb{Z}_{\ell}$-rank of $\Lambda_{v}$. Then there exists a positive integer $n_{0}$ such that $M_{n^{\prime}_{0}+ke_{v},v}=(\Lambda_{v}+\ell^{k}M_{v,n^{\prime}_{0}}\otimes\mathbb{Z}_{\ell})\cap M_{n_{0},v}$ where $n^{\prime}_{0}\geq n_{0}$ and $k$ is allowed to be any positive integer.

In an earlier version of the paper, we had in an earlier version of this paper implicitly assumed that $\Lambda_{v}=0$ (this assumption simplified the geometry-of-numbers arguments), and we are very grateful to Davesh Maulik for pointing this out to us. We will need the following result on homomorphisms between abelian surfaces:

Lemma 4.1.2**.**

Let $\alpha\in\operatorname{Hom}(\mathcal{B}_{\ell,n-1},\mathcal{B}^{\prime}_{\ell,n-1})$ for any $n\geq 3$.

1. (1)

The homomorphism $\ell\alpha$ lifts uniquely to $\operatorname{Hom}(\mathcal{B}_{\ell,n},\mathcal{B}^{\prime}_{\ell,n})$. 2. (2)

If $\ell\alpha$ lifts to $\operatorname{Hom}(\mathcal{B}_{\ell,n+1},\mathcal{B}^{\prime}_{\ell,n+1})$, then $\alpha$ lifts to $\operatorname{Hom}(\mathcal{B}_{\ell,n},\mathcal{B}^{\prime}_{\ell,n})$.

Let $\mathscr{G}_{i}=\mathcal{B}_{\ell,i}[\ell^{\infty}]$, and $\mathscr{G}^{\prime}_{i}=\mathcal{B}^{\prime}_{\ell,i}[\ell^{\infty}]$. By the Serre–Tate lifting theorem, it suffices to prove the analogous result for $\ell$-divisible groups. This is a straightforward application of Grothendieck–Messing theory. Before proceeding to the proof, we recall some facts from Grothendieck–Messing theory ([Messing] contains every result that we need). All the reduction maps between the $\mathcal{O}_{K_{v}}/\ell^{i}$ for $i=n,n\pm 1$ are canonically equipped with nilpotent divided powers (in fact, as $n>2$, all the ideals in play are square-zero). Let $\mathbb{D}$ and $\mathbb{D}^{\prime}$ denote the Dieudonne-crystals associated to $\mathscr{G}_{n-1}$ and $\mathscr{G}^{\prime}_{n-1}$ (see [Messing, §2.5 of Chapter IV]). Any homomorphism between $\mathscr{G}_{n-1}$ and $\mathscr{G}^{\prime}_{n-1}$ canonically induces a map of crystals $\mathbb{D}\rightarrow\mathbb{D}^{\prime}$.

Let $D_{i}$ and $D^{\prime}_{i}$ denote $\mathbb{D}$ and $\mathbb{D}^{\prime}$ evaluated at $\mathcal{O}_{K_{v}}/\ell^{i}$ for $i=n,n\pm 1$ (these are free $\mathcal{O}_{K_{v}}/\ell^{i}$-modules whose ranks equal the heights of $\mathscr{G}$ and $\mathscr{G}^{\prime}$). Grothendieck–Messing theory associates canonical filtrations $F_{i}\subset D_{i}$ and $F^{\prime}_{i}\subset D^{\prime}_{i}$ to the groups $\mathscr{G}_{i}$ and $\mathscr{G}^{\prime}_{i}$ for $i=n,n\pm 1$. Note that the $F_{n+1}$ reduces to $F_{n}$ and $F_{n-1}$ under the canonical quotient maps (the analogous statement holds for $F^{\prime}_{n+1}$). The filtrations are direct summands of the crystals evaluated at the $\mathcal{O}_{K_{v}}/\ell^{i}$. Suppose that $W_{n+1}\subset D^{\prime}_{n+1}$ is some submodule such that $D^{\prime}_{n+1}=F^{\prime}_{n+1}\oplus W_{n+1}$. Let $W_{n}\subset D^{\prime}_{n}$ denote the mod-$\ell^{n}$ reduction of $W_{n+1}$. Clearly, $W_{n}\oplus F^{\prime}_{n}=D^{\prime}_{n}$. By [Messing, Theorem 1.6 of Chapter V], a homomorphism between $\mathscr{G}_{n-1}$ and $\mathscr{G}^{\prime}_{n-1}$ lifts to $\operatorname{Hom}(\mathscr{G}_{i},\mathscr{G}^{\prime}_{i})$ (for $i=n,n+1$) if and only if the associated map of crystals evaluated at $\mathcal{O}_{K_{v}}/\ell^{i}$ maps the filtration $F_{i}$ to $F^{\prime}_{i}$. Let $\alpha_{i}:D_{i}\rightarrow D^{\prime}_{i}$ ($i=n,n\pm 1$) denote the maps induced by $\alpha$. We now proceed to the proofs of the two statements.

Proof of Lemma 4.1.2.

1. (1)

Let $v_{n}\in F_{n}\subset D_{n}$, whose image in $F_{n-1}$ is denoted by $v_{n-1}$. It suffices to prove that $\ell\alpha_{n}(v)\in F^{\prime}_{n}$. Let $\alpha_{n}(v)=v_{n}+w_{n}$, where $v^{\prime}_{n}\in F^{\prime}_{n}$ and $w_{n}\in W_{n}$. As $\alpha_{n-1}(F_{n-1})\subset F^{\prime}_{n-1}$, we have $w_{n}$ modulo $\ell^{n-1}$ is zero. It follows that $\ell w_{n}=0$. Therefore, $\ell\alpha_{n}(v_{n})=\ell v^{\prime}_{n}\in F^{\prime}_{n}$. Thus $\ell\alpha_{n}$ preserves filtrations, as requried. 2. (2)

As above, let $v_{n}\in F_{n}$. Let $v_{n+1}\in F_{n+1}$, whose mod-$\ell^{n}$ reduction is $v_{n}$. Suppose that $\alpha_{n+1}(v_{n+1})=v^{\prime}_{n+1}+w_{n+1}$, where $v^{\prime}_{n+1}\in F^{\prime}_{n+1}$ and $w_{n+1}\in W^{\prime}_{n+1}$. As $\ell\alpha$ lifts to $\operatorname{Hom}(\mathscr{G}_{n+1},\mathscr{G}^{\prime}_{n+1})$, it follows that $\ell w_{n+1}=0$. Therefore, $w_{n+1}=0$ modulo $\ell^{n}$. It follows that $\alpha_{n+1}(v_{n+1})$ modulo $\ell^{n}$ - which equals $\alpha_{n}(v_{n})$ - is an element of $F^{\prime}_{n}$.

∎

We now prove Theorem 4.1.1

Proof of Theorem 4.1.1.

For ease of notation, denote by $\Lambda_{v,i}$ the $\mathbb{Z}_{\ell}$-module $M_{v,i}\otimes\mathbb{Z}_{\ell}$. By the Serre–Tate theorem, it suffices to prove the existence of $n_{0}$ such that $\Lambda_{v,n^{\prime}_{0}+ke_{v}}=\Lambda_{v}+\ell^{k}\Lambda_{v,n^{\prime}_{0}}$. First, note that Lemma 4.1.2 implies that $\Lambda_{v}\subset\Lambda_{v,2e_{v}}$ is co-torsion free. Let $\Lambda^{\prime}\subset\Lambda_{v,2e_{v}}$ denote a direct summand of $\Lambda_{v}$. As the $\mathbb{Z}_{\ell}$-module of special endomorphisms of $\mathcal{B}[\ell^{\infty}]=\Lambda_{v}$, it follows that Therefore, $\bigcap_{n}(\Lambda^{\prime}\cap\Lambda_{v,n})=0$.

The theorem follows directly from the following claim.

Claim*.*

We have that $\Lambda^{\prime}\cap\Lambda_{v,n+e_{v}}\subset\Lambda^{\prime}\cap\ell\Lambda_{v,n}$ for large enough $n$.

To prove the claim, we fix any $n^{\prime}>2e_{v}$. Since $\bigcap_{n}(\Lambda^{\prime}\cap\Lambda_{v,n})=0$, then $\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+ke_{v}}\subset\ell(\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}})$ for large enough $k$. We now prove by contradiction that $\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+(k+1)e_{v}}\subset\ell(\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+ke_{v}})$ for such $k$. Assume that there exists a special endomorphism $\alpha\in(\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+(k+1)e_{v}})\backslash\ell(\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+ke_{v}})$. If $\alpha\in\ell(\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+(k-1)e_{v}})$, we write $\alpha=\ell\beta$, where $\beta\in\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+(k-1)e_{v}}$. By assumption, $\ell\beta\in\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+(k+1)e_{v}}$ and then by Lemma 4.1.2, $\beta\in\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+ke_{v}}$. This contradicts that $\alpha\notin\ell(\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+ke_{v}})$ and hence we have shown that $\alpha\notin\ell(\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}+(k-1)e_{v}})$. By iterating this argument, it follows that $\alpha\notin\ell(\Lambda^{\prime}\cap\Lambda_{v,n^{\prime}})$, which is a contradiction. ∎

4.2. Geometry of numbers and applications to counting special endomorphisms

For an $m$-dimensional lattice $M$ with a positive definite quadratic form $Q$, let $\mu_{1}(M)\leq\mu_{2}(M)\ldots\leq\mu_{m}(M)$ denote the successive minima of $M$ (see [Esk, Definition 2.2] for the definition of the term successive minima). We will need the following lemma due to Schmidt:

Lemma 4.2.1**.**

Then $\#\{s\in M\mid Q(s)\leq N\}=O\left(\displaystyle{\sum_{j=0}^{m}\frac{N^{j/2}}{\mu_{1}(M)\ldots\mu_{j}(M)}}\right)$, where the implied constant depends only on $m$.

Proof.

Equations (5) and (6) on page 518 of [Esk] imply that Lemma 2.4 of loc. cited implies the stated result (the authors refer to [Sch] for a proof of Lemma 2.4). ∎

We now prove a elementary lemma that will allow us to use Lemma 4.2.1 to bound special endomorphisms of $B_{v,n}$. We refer to the beginning of §4 for notation.

Lemma 4.2.2**.**

Let $m$ be the $\mathbb{Z}$-rank of $M_{v,1}$ and $m^{\prime}$ be the $\mathbb{Z}_{\ell}$-rank of $\Lambda_{v}$. Then $\displaystyle{\prod_{i=1}^{j}\mu_{i}(M_{v,n})}\gg\ell^{n(j-m^{\prime})/e_{v}}$.

Proof.

We may assume that $m^{\prime}<j$. It suffices to prove that $\displaystyle{\prod_{i=1}^{j}\mu_{i}(M_{v,n_{0}+(k+1)e_{v}})\geq\ell^{j-m^{\prime}}\prod_{i=1}^{j}\mu_{i}(M_{v,n_{0}+ke_{v}})}$.

For a lattice $M$, let $d(M)$ denote the square root of its discriminant. Theorem 4.1.1 implies that $d(M_{n_{0}+(k+1)e_{v}})\geq\ell^{m-m^{\prime}}d(M_{n_{0}+ke_{v}})$. Thus,

[TABLE]

by [Esk]*Eqn. (5),(6). This is the desired result for $j=m$.

Moreover, if $M\subset M^{\prime}$ are lattices, then $\mu_{i}(M)\geq\mu_{i}(M^{\prime})$. Therefore, Theorem 4.1.1 implies that $\mu_{i}(M_{n_{0}+(k+1)v})\leq\mu_{i}(\ell M_{n_{0}+kv})=\ell\mu_{i}(M_{n_{0}+kv})$. The lemma follows from multiplying (4.2.1) with the inequality $\displaystyle{\prod_{i=j+1}^{m}\mu_{i}(M_{v,n_{0}+(k+1)e_{v}})^{-1}\geq\prod_{i=j+1}^{m}\ell^{-1}\mu_{i}(M_{v,n_{0}+ke_{v}})^{-1}}$. ∎

Lemma 4.2.1 and Lemma 4.2.2 immediately yield the following corollary:

Corollary 4.2.3**.**

Suppose that the $\mathbb{Z}_{\ell}$-rank of $\Lambda_{v}$ is $\leq 1$ and the rank of $M_{v,1}$ is $m$. Then

[TABLE]

.

4.3. Proof of the non-archimidean local results

In what follows, we consider $T_{\mathfrak{p}}$ as in 2.2.1 with $p\neq\ell_{v}$ and recall that we write $(p)=\mathfrak{p}\mathfrak{p}^{\prime}\subset\mathcal{O}_{F}$. We always use $N$ to denote a large enough integer.

Lemma 4.3.1**.**

Over $\mathbb{Z}[1/pr]$, we have $T_{\mathfrak{p}}\mathcal{T}(r)=\mathcal{T}(pr)=T_{\mathfrak{p}^{\prime}}\mathcal{T}(r)$. Moreover, for any $n$, if there exists $[\mathcal{B}]\in T_{\mathfrak{p}}[\mathcal{A}]$ such that $[\mathcal{B}_{v,n}]\in\mathcal{T}(r)$, then $[\mathcal{A}_{v,n}]\in\mathcal{T}(pr)$.

Proof.

By checking on complex points, we have $T_{\mathfrak{p}}T(r)=T(pr)=T_{\mathfrak{p}^{\prime}}\mathcal{T}(r)$. By definition, $\mathcal{T}(m)$ is the Zariski closure of $T(m)$ and hence $T_{\mathfrak{p}}\mathcal{T}(r)=\mathcal{T}(pr)$. The second assertion then follows from the étaleness of Hecke orbits. ∎

For the rest of this section, we assume $\operatorname{End}(A_{\overline{\mathbb{Q}}})=\mathcal{O}_{F}$ and $\mathcal{A}$ has good reduction at $v$. Further, the norm of a special endomorphism $s$ denotes the integer $Q(s)$.

The following lemma is well-known and follows directly from the crystalline realization of the module of special endomorphisms. We record a proof here for completeness.

Lemma 4.3.2**.**

Let $m$ be the $\mathbb{Z}$-rank of $M_{v,1}$ and $m^{\prime}$ be the $\mathbb{Z}_{\ell}$-rank of $\Lambda_{v}$. Then $m\leq 4$ and $m^{\prime}\leq 2$.

Proof.

Since $s^{*}=s$ for any $s\in M_{v,1}$ (resp. $\Lambda_{v}$), we have that $\Lambda_{v}\subset M_{v,1}\otimes\mathbb{Z}_{\ell}\subset H^{2}_{{\mathrm{cris}}}(\mathcal{B}_{v,1}/W(\bar{\mathbb{F}}_{\ell}))$. On the other hand, since $\mathcal{O}_{F}\subset\operatorname{End}(\mathcal{B}_{v,1})$ is stable under Rosati involution, we have a natural embedding $\mathcal{O}_{F}\subset H^{2}_{{\mathrm{cris}}}(\mathcal{B}_{v,1}/W(\bar{\mathbb{F}}_{\ell}))$. Since $H^{2}_{{\mathrm{cris}}}(\mathcal{B}_{v,1}/W(\bar{\mathbb{F}}_{\ell}))[1/\ell]$ is a $W(\bar{\mathbb{F}}_{\ell}))[1/\ell]$-vector space of dimension $6$, then the Frobenius invariant part of $H^{2}_{{\mathrm{cris}}}(\mathcal{B}_{v,1}/W(\bar{\mathbb{F}}_{\ell}))[1/\ell]^{\varphi=1}$ is a $\mathbb{Q}_{\ell}$-vector space of dimension at most $6$.

Since $s\circ f=f^{\prime}\circ s$ for any $s\in M_{v,1}\otimes\mathbb{Q}_{\ell}$ and $f\in F$, then $\mathcal{O}_{F}\otimes\mathbb{Q}_{\ell}\cap M_{v,1}\otimes\mathbb{Q}_{\ell}=0$ in $H^{2}_{{\mathrm{cris}}}(\mathcal{B}_{v,1}/W(\bar{\mathbb{F}}_{\ell}))[1/\ell]^{\varphi=1}$. Since $\mathcal{O}_{F}\otimes\mathbb{Q}_{\ell}$ has dimension $2$, we have that $M_{v,1}\otimes\mathbb{Q}_{\ell}$ is at most of dimension $4$. Since $M_{v,1}\hookrightarrow M_{v,1}\otimes\mathbb{Q}_{\ell}$, we have that $m\leq 4$.

On the other hand, the de Rham cohomology of $\mathcal{B}$ induces a (decreasing) Hodge filtration $\operatorname{Fil}^{\bullet}$ on $H^{2}_{{\mathrm{cris}}}(\mathcal{B}_{v,1}/W(\bar{\mathbb{F}}_{\ell}))\otimes\bar{\mathbb{Q}}_{\ell}$ with $\dim\operatorname{Fil}^{0}=5$ and $\dim\operatorname{Fil}^{1}=1$. Hence $\operatorname{Fil}^{0}\cap H^{2}_{{\mathrm{cris}}}(\mathcal{B}_{v,1}/W(\bar{\mathbb{F}}_{\ell}))[1/\ell]^{\varphi=1}$ is a $\mathbb{Q}_{\ell}$-vector space of dimension at most $5$. By Grothendieck–Messing theory, both $\mathcal{O}_{F}$ and $\Lambda_{v}$ lie in $\operatorname{Fil}^{0}$ and hence $m^{\prime}+2\leq 5$. If $m^{\prime}=3$, then $\operatorname{Span}\{\mathcal{O}_{F},\Lambda_{v}\}=\operatorname{Fil}^{0}$. By Mazur’s weak admissibility theorem, since both $\mathcal{O}_{F}$ and $\Lambda_{v}$ lie in $H^{2}_{{\mathrm{cris}}}(\mathcal{B}_{v,1}/W(\bar{\mathbb{F}}_{\ell}))[1/\ell]^{\varphi=1}$, then $\operatorname{Span}\{\mathcal{O}_{F},\Lambda_{v}\}$ only has trivial filtration. This contradicts that $0\neq\operatorname{Fil}^{1}\subset\operatorname{Fil}^{0}$ and we conclude that $m^{\prime}\leq 2$. ∎

Theorem 4.3.3**.**

Let $M(N,n,\epsilon)$ denote the number of primes $p\in[N^{1/2},N]$ such that $\#\{T_{\mathfrak{p}}([\mathcal{A}_{v,n}])\cap(\bigcup_{r\in\mathbb{I}}\mathcal{T}(r))\}\geq\epsilon p$. Then $M(N,\lceil 3e_{v}\log\log N\rceil,\epsilon)=o(N/(\log N))$.

Proof.

The number of primes in the interval $[N^{1/2},N/\log(N)]$ is $o(N/\log(N)$, so we will restrict ourselves to primes $p\in[N/\log(N),N]$. For each prime $p$, each $[\mathcal{B}_{v,n}]\in T_{\mathfrak{p}}([\mathcal{A}_{v,n}])\cap(\cup_{r\in\mathbb{I}}\mathcal{T}(r))$ induces a special endomorphism of $\mathcal{A}_{v,n}$ whose norm is $pr\operatorname{Nm}\mathfrak{a}/D$. For all $p\in[N/\log N,N]$, the quantity $pr\operatorname{Nm}\mathfrak{a}/D=O(N)$. Notice that distinct $[\mathcal{B}_{v,n}]\in T_{\mathfrak{p}}([\mathcal{A}_{v,n}])\cap(\cup_{r\in\mathbb{I}}\mathcal{T}(r))$ induce distinct special endomorphisms of $\mathcal{A}_{v,n}$. Therefore, $\mathcal{A}_{v,n}$ has at least $M(N,n,\epsilon)\epsilon N/\log N$ special endomorphisms with norm bounded by $N$.

Applying the crudest bounds that Theorem 4.1.1, Lemma 4.2.1 and Lemma 4.3.2 yield, the number of special endomorphisms $\mathcal{A}_{v,n}$ has with norm bounded by $N$ is O\bigg{(}\frac{N^{2}}{\ell^{2n/e_{v}}}+N^{3/2}\bigg{)}. Therefore M(N,n,\epsilon)=O\bigg{(}\frac{N\log N}{\ell^{2n/e_{v}}}+N^{1/2}\log N\bigg{)} ($\epsilon$ gets absorbed in the $O()$). Substituting $n=\lceil 3e_{v}\log\log N\rceil$ yields $M(N,n,\epsilon)=o(N/(\log N))$ as required. ∎

The following theorem shows that one can choose a sequence of $p$ such that the largest $v$-adic intersection multiplicity of a point in $T_{\mathfrak{p}}([\mathcal{A}])$ with $\bigcup_{r\in\mathbb{I}}\mathcal{T}(r)$ is $O(\log p)$.

Theorem 4.3.4**.**

Set $n=\lceil\frac{e_{v}\log N}{\log\ell}\rceil$. Then the number of primes $p\in[N^{1/2},N]$ for which there exists $[\mathcal{B}]\in T_{\mathfrak{p}}([\mathcal{A}])$ and $r\in\mathbb{I}$ with $[\mathcal{B}_{v,n}]\in\mathcal{T}(r)$ is $o(N/\log N)$.

Proof.

Let $p\in[N^{1/2},N]$ such that there exists $[\mathcal{B}]\in T_{\mathfrak{p}}([\mathcal{A}])$ and $r\in\mathbb{I}$ as in the statement. Then $\mathcal{A}_{v,n}$ has a special endomorphism, say $s_{p}$, of norm $pr\operatorname{Nm}\mathfrak{a}/D$. Clearly, $s_{p}\neq s_{p^{\prime}}$ where $p^{\prime}\neq p$ also satisfies the conditions in the statement. Therefore, each such $p$ induces a distinct special endomorphism of $\mathcal{A}_{v,n}$ having norm $O(N)$ and it suffices to bound the number of special endomorphisms of norm $\leq N$.

Let $\Lambda_{v}$ denote the module of special endomorphsims of $\mathcal{A}_{\mathcal{O}_{K_{v}}}[\ell^{\infty}]$. If $\Lambda_{v}$ has $\mathbb{Z}_{\ell}$-rank $\leq 1$, then Corollary 4.2.3 yields the desired result. Therefore, we assume that the rank is at least 2. By Lemma 4.3.2, the rank of $\Lambda_{v}$ is at most two, so we assume that the rank equals two. Let $n=n^{\prime}_{0}+ke_{v}$, where $n^{\prime}_{0}-n_{0}<e_{v}$ with $n_{0}$ as in Theorem 4.1.1. We have $M_{v,n}\subset\ell^{k}\cap M_{v,n_{0}}+P^{\prime}_{n}$, where $P^{\prime}_{n}$ is a rank-two sublattice of $M_{v,n_{0}}$. There is no unique choice of $P^{\prime}_{n}$, so we choose $P_{n}$ to be any one with minimal root-discriminant $d(P_{n})$. As $\mathcal{B}$ has no special endomorphisms generically, it follows that $d(P_{n})\rightarrow\infty$.

We first deal with the case when $d(P_{n})\geq\log(N)^{2}$. Since $\mu_{1}(M_{v,n})\mu_{2}(M_{v,n})$ is of the same order of magnitude as $d(P_{n})$, then $\#\{v\in M_{v,n}:Q(v)\leq N\}=O(\frac{N^{2}}{\ell^{2n/e_{v}}}+\frac{N^{3/2}}{\ell^{n/e_{v}}}+\frac{N}{d(P_{n})}+N^{1/2})$ by Lemma 4.2.1.161616Note that $k$ differs from $n/e_{v}$ by a quantity bounded independent of $n$, so $1/\ell^{k}=O(1/\ell^{\frac{n}{e_{v}}})$. This quantity is $o(N/\log N)$ and so the result follows in this case.

Suppose now that $d(P_{n})\leq\log(N)^{2}$. By Lemma 4.3.5 below, if $v\in M_{v,n}$ has norm bounded by $N$, it follows that $v\in P_{n}$. As $d(P_{n})\rightarrow\infty$, the same argument used to finish the proof of Proposition 3.1.2 applies to prove that the proportion of primes $p$ such that there exists a $\mathcal{B}\in T_{\mathfrak{p}}(\mathcal{A})$ modulo $v^{n}$ goes to zero. ∎

Lemma 4.3.5**.**

Notation as above. Suppose that $d(P_{n})\leq\log(N)^{2}$. If $Q(v)\leq N$, then $v\in P_{n}$.

Proof.

For brevity set $d=d(P_{n})$. Fix a constant $n_{0}$ as in Theorem 4.1.1. Let $P^{\prime}_{n}$ denote the intersection with $M_{v,n_{0}}$ of the orthogonal complement of $P_{n}\otimes\mathbb{Q}$ in $M_{v,n_{0}}\otimes\mathbb{Q}$. We have that $Cd^{4}M_{v,n_{0}}\subset P_{n}+P_{n}^{\prime}$, where $C$ is a positive constant only depending on the discriminant of $Q$ on $M_{v,n_{0}}$. Indeed, let $P_{n}^{\vee}$ (resp. $P_{n}^{\prime\vee}$) denote the dual lattice of $P_{n}$ (resp. $P_{n}^{\prime}$) in $P_{n}\otimes\mathbb{Q}$ (resp. $P_{n}^{\prime}\otimes\mathbb{Q}$) with respect to the restriction of the quadratic form $Q$ to $P_{n}\otimes\mathbb{Q}$ (resp. $P_{n}^{\prime}\otimes\mathbb{Q}$). Then $d^{2}P_{n}^{\vee}\subset P_{n}$. On the other hand, there is a constant $C^{\prime}$ depending only on $\operatorname{disc}Q$ such that $P_{n}^{\prime}$ is spanned by two vectors $x,y$ such that $Q(x),Q(y)\leq C^{\prime}d^{2}$ (since they are given by solving linear equations with coefficients bounded by $O(d)$). Therefore, there exists a constant $C$ such that $Cd^{4}P_{n}^{\prime\vee}\subset P_{n}^{\prime}$. Since $M_{v,n_{0}}\subset(P_{n}+P_{n}^{\prime})^{\vee}=P_{n}^{\vee}+P_{n}^{\prime\vee}$, we have $Cd^{4}M_{v,n_{0}}\subset P_{n}+P_{n}^{\prime}$.

Let $v\in M_{v,n}$ satisfy $Q(v)\leq N$. Suppose that $v=u+\ell^{\lfloor(n-n_{0})/e_{v}\rfloor}w$ with $u\in P_{n}$ and $w\in M_{v,n_{0}}$, and let $Cd^{4}w=w_{1}+w^{\prime}_{1}$, where $w_{1}\in P_{n}$ and $w^{\prime}_{1}\in P^{\prime}_{n}$. Then,

[TABLE]

and thus

[TABLE]

As $\ell^{2\lfloor(n-n_{0})/e_{v}\rfloor}>C^{2}d^{8}N$, it follows $w^{\prime}_{1}=0$ as required. ∎

5. Proof of the main theorem

The goal of this section is to deduce our main theorem from the results in §§3-4 which provide upper bounds of the local intersection numbers. Recall that $A$ is an abelian surface over $K$ with an $\mathfrak{a}$-polarization and $\mathcal{O}_{F}\subseteq\operatorname{End}(A)$. As in 2.1, we will assume the existence of a semi-abelian scheme $\mathcal{A}$ over $\mathcal{O}_{K}$ with semistable reduction everywhere, whose generic fiber is $A$. Recall that $p$ denotes a prime which is totally split in the narrow Hilbert class field of $F$. To prepare for our proof, we first use Borcherds’ theory to choose a suitable Hirzebruch–Zagier divisor in the Hilbert modular surface and then compute the asymptotic of Faltings heights on the Hecke orbits.

5.1. Borcherds’ theory and the Faltings height

We devote this subsection to applying arithmetic Borcherds’ theory to choose a rational section of certain tensor powers of the Hodge line bundle. We then interpret the Faltings height of an abelian surface as a certain Arakelov intersection number.

We use $\mathcal{A}^{\textrm{univ-sa}}$ to denote the universal family of semi-abelian schemes over $\overline{\mathcal{H}}^{\rm{tor}}$ (with suitable level structure). In [BBK]*§6, the authors explain a way to define the arithmetic intersection independently of the choice of a level structure. We recall their definition in 5.1.3. Let $e:\overline{\mathcal{H}}^{\rm{tor}}\rightarrow\mathcal{A}^{\textrm{univ-sa}}$ be the identity section and let $\omega=\det(e^{*}\Omega^{1}_{\mathcal{A}^{\textrm{univ-sa}}/\overline{\mathcal{H}}^{\rm{tor}}})$ over $\overline{\mathcal{H}}^{\rm{tor}}$ be the Hodge line bundle. We endow $\omega$ with a Hermitian metric $||\cdot||_{F}$ (only on $\mathcal{H}(\mathbb{C})$) as in [F85]*sec. 3 and denote by $\overline{\omega}$ the Hermitian line bundle with log singularity along the boundary. By definition, we have

[TABLE]

where $h_{F}$ denotes the stable Faltings height and $ht$ is the height function of subvarieties of an arithmetic variety with respect to certain arithmetic cycles (see, for example, [BBK]*§1.5, eqn. (1.17) and we normalize $h_{F}$ and $ht_{\overline{\omega}}$ to be independent on the choice of $K$; more specifically, $||\ell||_{v}=\ell^{-\frac{[K_{v}:\mathbb{Q}_{\ell}]}{[K:\mathbb{Q}]}}$).

It is well known that the space of global sections of the line bundle $\omega^{\otimes k}$ over $\overline{\mathcal{H}}^{\rm{tor}}_{\mathbb{C}}$ (resp. $\overline{\mathcal{H}}^{\rm{tor}}_{\overline{\mathbb{Q}}}$) is the space of Hilbert modular forms of parallel weight $k$ over $\mathbb{C}$ (resp. $\overline{\mathbb{Q}}$); see, for example, [FC]*Chp. V.1 and [Chai]*sec. 4. Up to a constant, the Hermitian metric171717We also use $||\cdot||_{F}$ to denote the metric on $\overline{\omega}^{\otimes k}$ given by the tensor product of the Hermitian metric $||\cdot||_{F}$ on $\overline{\omega}$. $||\cdot||_{F}$ on $\omega^{\otimes k}$ is defined by $||f(z)||_{\text{Pet}}=|f(z_{1},z_{2})(\Im z_{1})^{k/2}(\Im z_{2})^{k/2}|$, where $f$ is a Hilbert modular form of parallel weight $k$ and $z=(z_{1},z_{2})\in\mathbb{H}^{2}$: indeed, this follows from the $G(\mathbb{R})$-invariance of both metrics.

Lemma 5.1.1**.**

There exist a positive integer $k$ and a meromorphic Hilbert modular form $\Psi$ over $\overline{\mathbb{Q}}$ of parallel weight $k$ such that the divisor $\operatorname{Div}(\Psi)$ defined by $\Psi$ on $\mathcal{H}_{\mathbb{Q}}$ is given by $\sum_{r\in\mathbb{I}}c_{r}T(r)$, where $c_{r}\in\mathbb{Z}$ and $\mathbb{I}$ is a finite subset of

[TABLE]

In particular, $\operatorname{Div}(\Psi)$ is a weighted sum of compact Shimura curves.

Borcherds’ theory lifts weakly holomorphic modular forms on modular curves to meromorphic Hilbert modular forms on $\mathcal{H}_{\mathbb{C}}$. By lifting, it means that the divisor defined the resulting Hilbert modular form is determined by the principal part of the Fourier expansions of the given modular form at cusps of the modular curve. Borcherds and Bruinier showed that the existence such lift of certain modular form can be verified by certain explicit conditions on the Fourier coefficients of its principal part. The Fourier expansions of Borcherds lifts (sometimes also called Borcherds products) have also been studied by many people, which leads to an arithmetic theory of these lifts. One may see [BBK]*§4 for a summary of relevant results when the discriminant of $F$ is a prime.

Proof of Lemma 5.1.1.

By [Br16]*Thm. 1.1,181818[Br16]*Thm. 1.1 is a generalization of [BBK]*Lem. 4.11. The proof of this lemma, which only deals with the case when $D$ is a prime, contains the main idea of the proof for the general case. in which we take the infinite admissible set to be $\mathbb{J}$, there exists a Borcherds product $\Psi^{\prime}$ of non-zero weight $k$ whose divisor is supported on $\cup_{r\in\mathbb{J}}T(r)$. In other words, $\Psi^{\prime}$ is a Hilbert modular form of parallel weight $k$ over $\mathbb{C}$. We may assume $k>0$, since otherwise we just take $\Psi^{\prime-1}$. By [Br16]*Prop. 3.1, the weakly holomorphic modular $f$ whose Borcherds lift is $\Psi$ has integral Fourier coefficients. [Hor]*3.2.14 shows that, after multiplying by a suitable scalar, the Borcherds lift of a modular form with Fourier coefficients in $\mathbb{Q}$ is defined over $\overline{\mathbb{Q}}$. In particular, if we take $\Psi$ to be $\Psi^{\prime}$ multiplied by a suitable scalar, then $\Psi$ is a rational section of $\omega^{\otimes k}$ over $\overline{\mathcal{H}}^{\rm{tor}}_{\overline{\mathbb{Q}}}$. For $r\in\mathbb{J}$, the divisor $T(r)$ is compact by Corollary 2.1.3. ∎

5.1.2*.*

We view $\Psi$ as a rational section of $\omega^{\otimes k}$ over $\overline{\mathcal{H}}^{\rm{tor}}_{\mathcal{O}_{K^{\prime}}}$, where $K^{\prime}$ is a large enough number field such that $\Psi$ is defined. Hence $\operatorname{Div}(\Psi)=\sum_{r}c_{r}\mathcal{T}(r)+\sum_{p}\mathcal{E}_{p}$, where the second sum is over finitely many $p$ and $\mathcal{E}_{p}$ is a finite (weighted) sum of irreducible components of $\overline{\mathcal{H}}^{\rm{tor}}_{\overline{\mathbb{F}}_{p}}$.

5.1.3*.*

Given an arithmetic divisor $\mathcal{D}$ and a horizontal $1$-cycle $\mathcal{Z}$ intersecting properly on $\overline{\mathcal{H}}^{\rm{tor}}_{\mathcal{O}_{K^{\prime}}}$, one defines the (arithmetic) intersection number as follows: (see, for example, [BKK]*Thm. 1.33191919The higher tor group vanishes since we work with a Cartier divisor which intersects the $1$-cycle properly. for regular schemes and [Yang]*eqn. (2.1) for regular Deligne–Mumford stacks.)

[TABLE]

where $v$ ranges over the through finite places of $K^{\prime}$, the intersection $\mathcal{Z}\cap\mathcal{D}=\mathcal{Z}\times_{\overline{\mathcal{H}}^{\rm{tor}}}\mathcal{D}$ is a Deligne–Mumford stack of dimension [math], the ring $\widetilde{\mathcal{O}}_{\mathcal{Z}\cap\mathcal{D},x}$ is the strictly Henselian local ring of $\mathcal{Z}\cap\mathcal{D}$ at $x$, and $k(x)$ is the residue field of $\widetilde{\mathcal{O}}_{\mathcal{Z}\cap\mathcal{D},x}$. By definition, $\displaystyle\frac{\mathcal{D}.\mathcal{Z}}{[K^{\prime}:\mathbb{Q}]}$ is independent of the choice of $K^{\prime}$.

In [BBK]*sec. 6.3, they define the arithmetic intersection number on $\overline{\mathcal{H}}^{\rm{tor}}$ as the arithmetic intersection number of the pull back of arithmetic cycles to $\overline{\mathcal{H}}^{\rm{tor}}(N)$, the Hilbert modular surface with full level $N$-structure with $N\geq 3$, divided by the degree of the map $\overline{\mathcal{H}}^{\rm{tor}}(N)\rightarrow\overline{\mathcal{H}}^{\rm{tor}}$. This is the idea behind the above formula.

Remark 5.1.4*.*

For $\mathcal{D}$ and $\mathcal{Z}$ as above, let $n$ denote the largest integer such that $\mathcal{Z}$ is contained in $\mathcal{D}$ modulo $v^{n}$ (here, we consider $\mathcal{Z}$ and $\mathcal{D}$ as subschemes of the coarse Hilbert modular surface). In our applications, we will only consider the intersection at finitely many places. Therefore, we may pass to a suitable level structure étale at these finitely many places so that $\mathcal{T}(r)$ are regular ([Ca]). Then the length at $v$ referred to in 5.1.3 differs from $n$ by an absolutely bounded factor. As we are only concerned with bounds, we will in the sequel restrict ourselves with controlling the growth of $n$.

Lemma 5.1.5**.**

Assume that $\operatorname{End}(A_{\overline{K}})=\mathcal{O}_{F}$. Following the notation as in Lemma 5.1.1, there exists a constant $C_{1}$ independent of $A$ such that

[TABLE]

Proof.

By Lemma 2.1.6, if $[A]$ lies on $T(r)$, then $\operatorname{End}(A_{\overline{K}})$ is strictly larger than $\mathcal{O}_{F}$. Hence our assumption implies that $[A]$ does not lie on any $T(r)$. It follows that the $1$-cycle $[\mathcal{A}]$ intersects $\sum_{r\in\mathbb{I}}c_{r}T(r)$ properly. Set $\mathcal{E}(A)=\frac{1}{[K:\mathbb{Q}]}[\mathcal{A}].(\sum_{p}\mathcal{E}_{p})$, where $\sum_{p}\mathcal{E}_{p}$ was defined in 5.1.2. By definition, $\mathcal{E}(A)$ is bounded by an absolute constant independent of $A$. Moreover,

[TABLE]

(see, for example, [Yang]*eqn. (2.3)). The lemma then follows from the fact that $||\cdot||_{F}$ and $||\cdot||_{\text{Pet}}$ differ by an absolute constant independent of $A$. ∎

We end this subsection with a formula for the average Faltings height of abelian surfaces corresponding to points in $T_{\mathfrak{p}}[A]$ when $A$ has good reduction at all the primes of $K$ above $p$. The idea of proof builts on use Autissier’s idea in [Au]. From now on, we say that $A$ has good reduction at $p$ (resp. ordinary reduction at $p$) if $A$ has good reduction (resp. ordinary reduction) at all the primes of $K$ above $p$.

Proposition 5.1.6**.**

Let $p$ be a prime as in 2.2.1. If $A$ has good reduction at $p$, then

[TABLE]

Proof.

The proof consists two parts. We first show that $\sum_{[B]\in T_{\mathfrak{p}}[A]}h_{F}(B)-(p+1)h_{F}(A)$ is independent of $A$. Then we compute this quantity in a particular case.

1. (1)

Let $\mathcal{H}(\mathfrak{p})$ denote the Hilbert modular surface over $\mathbb{Z}$ with $\Gamma_{0}(\mathfrak{p})$ level structure. The stack $\mathcal{H}(\mathfrak{p})$ parameterizes degree $p$ isogenies $\phi:A_{1}\rightarrow A_{2}$ between abelian varieties with $\mathcal{O}_{F}$-multiplication such that $\ker(\phi)\subset A_{1}[\mathfrak{p}]$ (see for example [Pa]*sec. 2.2). Let $\pi_{i}:\mathcal{H}(\mathfrak{p})_{\mathbb{Z}_{(p)}}\rightarrow\mathcal{H}_{\mathbb{Z}_{(p)}}$ for $i=1,2$ be the forgetful map that sends $\phi$ to $A_{i}$. We claim that each $\pi_{i}$ is finite flat.

We first show that $\pi_{i}$ is quasi-finite. Let $v$ be any finite place of $K$ over $p$. The group scheme $\mathcal{A}[\mathfrak{p}^{\infty}]$ of $\mathfrak{p}$-power torsions of $\mathcal{A}$ is a $p$-divisible group of height 2, whose mod $v$ reduction has dimension 1. There are two cases: the mod-$v$ reduction of $\mathcal{A}[\mathfrak{p}^{\infty}]$ is either ordinary, or supersingular. If ordinary, [FC]*§VII.4 shows that there are only finitely many degree $p$ subgroups of the mod-$v$ reduction of $\mathcal{A}[\mathfrak{p}]$. Therefore, by the modular interpretation of $\mathcal{H}(\mathfrak{p})$, the map $\pi_{1}$ is quasi-finite. Now we assume that the mod-$v$ reduction of $\mathcal{A}[\mathfrak{p}^{\infty}]$ is supersingular. Since the number of degree $p$ subgroups of the reduction of $\mathcal{A}[\mathfrak{p}]$ only depends on the isomorphism class of the $p$-divisible group, we may assume that $\mathcal{A}$ has supersingular reduction at $v$. Since $p$ is split in $F$, the supersingular locus of $\mathcal{H}_{\mathbb{F}_{v}}$ is [math]-dimensional and hence there are only finitely many mod-$v$ points on $\mathcal{H}$ corresponding to abelian surfaces isogenous to $\mathcal{A}$ mod $v$. In particular, $\pi_{1}$ is quasi-finite. The same argument applies to $\pi_{2}$ when we study the kernel of the Rosati involution of $\phi$.

By [Pa]*2.1.3, Cor. 2.2.3, the stack $\mathcal{H}(\mathfrak{p})$ is Cohen–Macaulay and $\mathcal{H}$ is regular. Since all the fibers of $\pi_{i}$ are [math]-dimensional, then by [EGAIV]*II.6.1.5, each $\pi_{i}$ is flat. On the other hand, the $\pi_{i}$ are proper by [Pa]*the discussion after Def. 2.2.1. Therefore, each $\pi_{i}$ is finite flat.

By the argument in [Au, Theorem 5.1], one observes that the independence of the quantity $\sum_{[B]\in T_{\mathfrak{p}}[A]}h_{F}(B)-(p+1)h_{F}(A)$ on $A$ is a formal consequence of the finite-flatness of $\pi_{i}$, the normality of $\mathcal{H}$, and the irreducibility of $\mathcal{H}_{\mathbb{F}_{v}}$ for every $v$ above $p$. 2. (2)

We now compute $\sum_{[B]\in T_{\mathfrak{p}}[A]}h_{F}(B)-(p+1)h_{F}(A)$ when $A$ has ordinary reduction at $p$.202020One may also choose any CM abelian surface on $\mathcal{H}$ and apply the formula for the Faltings height in [lucia] to compute this difference. Such an $A$ always exists: indeed, for a CM field $K_{2}$ containing $F$ such that $K_{2}$ is Galois over $\mathbb{Q}$ of degree $4$ and $p$ splits completely in $K_{2}$, there exist abelian surfaces with CM by $K_{2}$ that correspond to points on $\mathcal{H}$ and these abelian surfaces are ordinary at $p$.

Now assume that $A$ has good ordinary reduction at $p$ and we prove the result for such $A$. We first enlarge $K$ so that all $[B]\in T_{\mathfrak{p}}[A]$ are defined over $K$. Since $\mathcal{A}[\mathfrak{p}^{\infty}]$ over $\mathcal{O}_{K_{v}}$ is $1$-dimensional, it only contains a unique degree $p$ subgroup which is multiplicative (equivalently, in the ordinary case, not étale) for any $v$ above $p$. Then by Lemma 2.2.2, there exists only one element in $T_{\mathfrak{p}}[\mathcal{A}]$ that corresponds to an isogeny with multiplicative kernel. Now we apply [F85]*Lem. 5. By the standard calculation on $\Omega^{1}$ of finite flat groups of degree $p$, at each $v$ there are $p$ out of $p+1$ elements in $T_{\mathfrak{p}}[A]$ such that the term $\log(\#e^{*}(\Omega^{1}_{\ker\phi/\mathcal{O}_{K_{v}}}))$ in Faltings’ formula is [math], and one element such that $\log(\#e^{*}(\Omega^{1}_{\ker\phi/\mathcal{O}_{K_{v}}}))=-[K_{v}:\mathbb{Q}_{p}]\log p$. We obtain the desired formula by summing up all the local contributions. ∎

5.2. Proof of Theorem 1

5.2.1*.*

We first sketch the proof of Theorem 1. First, we use Theorem 1 to choose a good Hirzebruch–Zagier divisor $\sum_{r\in\mathbb{I}}c_{r}\mathcal{T}(r)=\operatorname{Div}(\Psi)$. By Proposition 5.1.6, we have $\sum_{[B]\in T_{\mathfrak{p}}([A])}h_{F}(B)$ is $O(p\log p)$. On the other hand, the local results in §§4-3 show that each local term in Lemma 5.1.5 is $o(p\log p)$. By local term, we mean either $-\sum_{[B]\in T_{\mathfrak{p}}[A]}\log||\Psi(\sigma([B]))||_{\mathrm{Pet}}$ for all $\sigma:K\hookrightarrow\mathbb{C}$ or the $v$-adic intersection number $(\sum_{[\mathcal{B}]\in T_{\mathfrak{p}}[\mathcal{A}]}[\mathcal{B}],\sum_{r\in\mathbb{I}}c_{r}\mathcal{T}(r))_{v}$ for finite places $v$. This implies that $T_{\mathfrak{p}}([\mathcal{A}])$ intersects $\sum_{r\in\mathbb{I}}c_{r}\mathcal{T}(r)$ at infinitely many places as $p\rightarrow\infty$ and then Theorem 1 follows from Corollary 2.1.7.

5.2.2*.*

If $\mathcal{O}_{F}\subsetneq\operatorname{End}(A_{\overline{K}})$, then by the classification of the endomorphism ring of absolutely simple abelian surfaces over a characteristic zero field, $\operatorname{End}(A_{\overline{K}})\otimes\mathbb{Q}$ is either an indefinite quaternion algebra over $\mathbb{Q}$ or a degree $4$ CM field. In the first case, $\mathcal{A}_{\overline{\mathbb{F}}_{v}}$ is not simple if the quaternion algebra splits at ${\rm{char}}\,\mathbb{F}_{v}$. In the second case, there exists a positive density set of primes $\ell$ so that $\mathcal{A}_{\mathbb{F}_{v}}$ is supersingular for all $v|\ell$ and hence not geometrically simple. Therefore, to prove Theorem 1, we assume that $\operatorname{End}(A_{\overline{\mathbb{Q}}})=\mathcal{O}_{F}$ from now on and hence for any $[B]\in T_{\mathfrak{p}}([A])$, we also have $\operatorname{End}(B_{\overline{K}})=\mathcal{O}_{F}$. Therefore, all $T_{\mathfrak{p}}([\mathcal{A}])$ intersect Hirzebruch–Zagier divisors properly.

Proof of Theorem 1.

We now show that there are infinitely many primes $v$ of $K$ such that $\mathcal{A}_{\overline{\mathbb{F}}_{v}}$ is not simple. Assume, for the sake of contradiction, that there is a finite set of places $\Sigma$ of $K$ such that $\mathcal{A}$ has geometrically simple, or bad reduction modulo $v$ for $v\notin\Sigma$. Corollary 2.1.3 and Lemma 5.1.1 give a meromorphic Hilbert modular form $\Psi$ such that $\operatorname{Div}(\Psi)$ is a compact special divisor $\sum_{r\in\mathbb{I}}c_{r}\mathcal{T}(r)$ with $D|r$ for all $r\in\mathbb{I}$. The intersection $(T_{\mathfrak{p}}([\mathcal{A}]),\sum_{r\in\mathbb{I}}c_{r}\mathcal{T}(r))$ has a nonzero $v$-adic term only when $v\in\Sigma$ by Corollary 2.1.7. Throughout the proof, $p$ will denote a prime which is totally split in the narrow Hilbert class field of $F$ and $v\nmid p$ for all $v\in\Sigma$. We now explain how to choose an increasing sequence of primes $p$ such that we can bound the local terms as described in 5.2.1 by $\epsilon p\log p$ for arbitrary $\epsilon>0$.

By Theorems 4.3.3 and 4.3.4 and 5.1.4, outside a density-zero set of primes $p$, the $v$-adic intersection

[TABLE]

where $C_{1}$ is the absolute constant mentioned in 5.1.4. Notice that $2\log p\geq\log N$.

Let $C_{2}$ be the density of primes splitting completely in the narrow Hilbert class field of $F$. By taking $\epsilon_{1}=\epsilon,\,\epsilon_{2}=\frac{C_{2}}{4[K:\mathbb{Q}]}$ in Theorem 3.2.1, we have that, for $N\gg 0$ and for $p\in[N^{1/2},N]$ in a set of density at least $3C/4$, for all $\sigma:K\hookrightarrow\mathbb{C}$, the archimedean term

[TABLE]

We have shown that for $N\gg 0$, there exists a positive density set of primes $p\in[N^{1/2},N]$ such that all the local terms are $o(p\log p)$. On the other hand, by Proposition 5.1.6, $\displaystyle{\sum_{[B]\in T_{\mathfrak{p}}([A])}h_{F}(B)}$ has order of magnitude $p\log p$. We then obtain the desired contradiction by applying Lemma 5.1.5 to all $[B]\in T_{\mathfrak{p}}([A])$. ∎