Subconvexity for $GL(3)\times GL(2)$ twists (with an appendix by Will Sawin)

TL;DR

This paper establishes new subconvex bounds for certain $L$-functions involving $GL(3)$ and $GL(2)$ automorphic forms twisted by characters, advancing understanding of their growth rates at the critical line.

Contribution

It provides the first subconvexity bounds for $L$-functions of $GL(3) imes GL(2)$ twists, improving previous convexity bounds and employing novel analytic techniques.

Findings

Proves $L(1/2, \pi imes f imes \chi) ext{ bound } p^{3/2 - 1/16 + ext{small }\epsilon}$.

Establishes $L(1/2, \pi imes \chi) ext{ bound } p^{3/4 - 1/32 + ext{small }\epsilon}$.

Advances the analytic theory of automorphic $L$-functions and their subconvexity bounds.

Abstract

Let $\pi$ be a $SL(3,\mathbb{Z})$ Hecke-Maass cusp form, $f$ be a $SL(2,\mathbb{Z})$ holomorphic cusp form or Maass cusp form and $\chi$ be any non-trivial character $\bmod \, p$, where $p$ is prime. We show that the $L$-function associated with this triplet satisfy \begin{equation*} L\left(\frac{1}{2},\pi\times f\times\chi\right)\ll_{\pi,f,\epsilon} p^{\frac{3}{2}-\frac{1}{16}+\epsilon}. \end{equation*} The method also yields the subconvex bound \begin{equation*} L\left(\frac{1}{2},\pi\otimes \chi\right)\ll_{\pi,\epsilon }p^{\frac{3}{4}-\frac{1}{32}+\epsilon }. \end{equation*}

Equations

$$L\left(\frac{1}{2},\pi \times f\times \chi \right){\ll }_{\pi ,f,ϵ}{p}^{\frac{3}{2}-\frac{1}{16}+ϵ}.$$

$$L\left(\frac{1}{2},\pi \otimes \chi \right){\ll }_{\pi ,ϵ}{p}^{\frac{3}{4}-\frac{1}{32}+ϵ}.$$

$$L(s,\pi \times f\times \chi )={\sum \sum }_{r,n=1}^{\infty }\frac{{\lambda }_{\pi }(r,n){\lambda }_f(n)\chi (n{r}^2)}{(n{r}^2{)}^s},$$

$$L(1/2,\pi \times f\times \chi ){\ll }_{\pi ,f,ϵ}{p}^{\frac{3}{2}+ϵ}.$$

$$L\left(\frac{1}{2},\pi \times f\times \chi \right){\ll }_{\pi ,f,ϵ}{p}^{\frac{3}{2}-\frac{1}{16}+ϵ}.$$

$$L^2(s,\pi \otimes \chi )={\left(\sum _{n=1}^{\infty }\frac{{\lambda }_{\pi }(1,n)\chi (n)}{n^s}\right)}^2={\sum \sum }_{r,n=1}^{\infty }\frac{{\lambda }_{\pi }(r,n)d(n)\chi (n{r}^2)}{(n{r}^2{)}^s},$$

$$L^3(s,f\otimes \chi )={\left(\sum _{n=1}^{\infty }\frac{{\lambda }_f(n)\chi (n)}{n^s}\right)}^3={\sum \sum }_{r,n=1}^{\infty }\frac{{\lambda }_{\min }(r,n){\lambda }_f(n)\chi (n{r}^2)}{(n{r}^2{)}^s},$$

$${\lambda }_{\min }(r,n)=\sum _{d|(r,n)}\mu (d)d_3(r/d)d_3(n/d)$$

$$L(1/2,\pi \otimes \chi ){\ll }_{\pi ,ϵ}{p}^{3/4-1/32+ϵ}.$$

$$L\left(\frac{1}{2},\chi \right){\ll }_{ϵ}{q}^{\frac{3}{16}+ϵ}.$$

$$L(f\times \chi ,s){\ll }_f|s{|}^2{q}^{5/11}{d}^2(q)\log q.$$

$$L\left(\frac{1}{2},\pi \times f\times \chi \right)\ll \left|\sum _{m=1}^{\infty }\frac{{\lambda }_{\pi \times f\times \chi }(m)}{m^{1/2}}\mathcal{V}\left(\frac{m}{p^3}\right)\right|,$$

$${\lambda }_{\pi \times f\times \chi }(m)={\sum \sum }_{n{r}^2=m}{\lambda }_{\pi }(r,n){\lambda }_f(n)\chi (n{r}^2)$$

$$x^j{\mathcal{V}}^{(j)}(x)=O_A(1+|x|{)}^{-A}.$$

$$L\left(\frac{1}{2},\pi \times f\times \chi \right)\ll \underset{r\le p^{\theta }}{\sup }\underset{\frac{p^{3-\theta }}{r^2}\le N\le \frac{p^{3+ϵ}}{r^2}}{\sup }\frac{|S_r(N)|}{N^{1/2}}+p^{(3-\theta )/2},$$

$$S_r(N)=\sum _{n=1}^{\infty }{\lambda }_{\pi }(r,n)\lambda (n)\chi (n)V\left(\frac{n}{N}\right)$$

$$L\left(\frac{1}{2},\pi \times f\times \chi \right)$$

$$\ll \left|{\sum \sum }_{n{r}^2\le p^{3+ϵ}}\frac{{\lambda }_{\pi }(r,n){\lambda }_f(n)\chi (n{r}^2)}{(n{r}^2{)}^{1/2}}\mathcal{V}\left(\frac{n{r}^2}{p^3}\right)\right|+p^{-2019}$$

$$=\left|\sum _{r\le p^{(3+ϵ)/2}}\frac{\chi (r^2)}{r}\sum _{n\le p^{3+ϵ}/r^2}\frac{{\lambda }_{\pi }(r,n){\lambda }_f(n)\chi (n)}{n^{1/2}}\mathcal{V}\left(\frac{n{r}^2}{p^3}\right)\right|+p^{-2019}.$$

$$\sum _{r\le p^{(3+ϵ)/2}}\sum _{n\le \frac{p^{3+ϵ}}{r^2}}=\sum _{r\le p^{\theta }}\sum _{\frac{p^{3-\theta }}{r^2}\le n\le \frac{p^{3+ϵ}}{r^2}}+\sum _{r\le p^{\theta }}\sum _{n<\frac{p^{3-\theta }}{r^2}}+\sum _{r>p^{\theta }}\sum _{n\le \frac{p^{3+ϵ}}{r^2}},$$

$${\sum \sum }_{n_1^2{n}_2\le x}|{\lambda }_{\pi }(n_1,n_2){|}^2\ll x,\sum _{n\le x}|{\lambda }_f(n){|}^2\ll x$$

$$L\left(\frac{1}{2},\pi \times f\times \chi \right)\ll \left|\sum _{r\le p^{\theta }}\frac{\chi (r^2)}{r}\sum _{\frac{p^{3-\theta }}{r^2}\le n\le \frac{p^{3+ϵ}}{r^2}}\frac{{\lambda }_{\pi }(r,n){\lambda }_f(n)\chi (n)}{n^{1/2}}\mathcal{V}\left(\frac{n{r}^2}{p^3}\right)\right|+p^{(3-\theta )/2}.$$

$$\sup_{\frac{p^{3-\theta}}{r^{2}}\leq N\leq\frac{p^{3+\epsilon}}{r^{2}}}\Bigg{|}\sum_{n=1}^{\infty}\frac{\lambda_{\pi}(r,n)\lambda_{f}(n)\chi(n)}{n^{1/2}}W\left(\frac{n}{N}\right)\mathcal{V}\left(\frac{nr^{2}}{p^{3}}\right)\Bigg{|},$$

$$\underset{\frac{p^{3-\theta }}{r^2}\le N\le \frac{p^{3+ϵ}}{r^2}}{\sup }\frac{|S_r(N)|}{N^{1/2}},$$

$$S_r(N)=\sum _{n=1}^{\infty }{\lambda }_{\pi }(r,n)\lambda (n)\chi (n)V_{r,N}\left(\frac{n}{N}\right).$$

$$\delta (n)=\{\begin{array}{ll}1& \text{if}n=0\\ 0& \text{otherwise}.\end{array}$$

$$\delta (n)=\frac{1}{Q}\sum _{q\le Q}\frac{1}{q}\sum _{a\mathrm{mod}q}e\left(\frac{na}{q}\right){\int }_{\mathbb{R}}g(q,x)e\left(\frac{nx}{qQ}\right)dx,$$

$$g(q,x)=1+O\left(\frac{Q}{q}{\left(\frac{q}{Q}+|x|\right)}^A\right),g(q,x)\ll |x{|}^{-A}$$

$$x^j\frac{{\partial }^j}{\partial x^j}g(q,x)\ll \log Q\min \{Q/q,1/|x|\}$$

$${\lambda }_{\pi }(1,l){\lambda }_{\pi }(r,n)={\lambda }_{\pi }(r,nl)+{\lambda }_{\pi }(rl,n/l)+{\lambda }_{\pi }(r/l,nl),$$

$$r<L=p^{\eta }$$

$$S_r(N)=\frac{1}{{\sum }_{l\in \mathcal{L}}|{\lambda }_{\pi }(1,l){|}^2}\sum _{l\in \mathcal{L}}\overline{{\lambda }_{\pi }(1,l)}\sum _{n=1}^{\infty }({\lambda }_{\pi }(r,nl)+{\lambda }_{\pi }(rl,n/l))\lambda (n)\chi (n)V\left(\frac{n}{N}\right).$$

Full text

[TABLE]

The method also yields the subconvex bound

[TABLE]

1. Introduction

Let $\pi$ be a $SL(3,\mathbb{Z})$ Hecke-Maass cusp form, $f$ be a $SL(2,\mathbb{Z})$ holomorphic Hecke cusp form and $\chi$ be any non-trivial Dirichlet character $\bmod\,p$. Throughout this paper, $p$ always denotes a prime number. The Rankin-Selberg convolution of $\pi$ with $f\times\chi$ is given by

[TABLE]

which converges absolutely for $\Re(s)>1$. The series extends to an entire function and satisfies a functional equation of the Riemann type and has arithmetic conductor $p^{6}$ (see [3], Lemma 3.8 and Theorem 3.9). Consequently, the Phragmen-Lindelöf principle yields the convexity bound

[TABLE]

The Lindelöf hypothesis asserts that the exponent $3/2+\epsilon$ can be replaced by any positive number. But even breaking the convexity barrier is hard and has remained open so far. In this paper we prove the following subconvex bound.

Theorem 1.1**.**

Let $\pi$ be a $SL(3,\mathbb{Z})$ Hecke-Maass cusp form, $f$ be a $SL(2,\mathbb{Z})$ holomorphic Hecke cusp form or Hecke-Maass cusp form and $\chi$ be any non-trivial character $\bmod\,p$, then

[TABLE]

We note that Lin-Michel-Sawin [16] generalised Theorem 1.1 by replacing $\chi$ by a generic trace function.

Note that one has

[TABLE]

and

[TABLE]

where

[TABLE]

are the coefficients of the minimal Eisenstein series for $SL(3,\mathbb{Z})$. These are same as (1.1) when $f=E(z,1/2)$ or when $\pi$ is the minimal Eisenstein series for $SL(3,\mathbb{Z})$. In fact, our proof goes through for these variations of $L$-functions as well. All we need is the fact that (1.3) and (1.4) satisfies a functional equation with arithmetic conductor $p^{6}$ and the corresponding Voronoi summation formulas for $d(n)$ and $d_{3}(n)$. Hence, we also obtain the following, which is an improvement over the subconvex bound obtained in Holowinsky-Nelson [10] and in [26] :

Theorem 1.2**.**

Let $\pi$ be a $SL(3,\mathbb{Z})$ Hecke-Maass cusp form and $\chi$ be any non-trivial character $\bmod\,p$, then

[TABLE]

Let us briefly recall the history of subconvexity of $L$-functions. The first subconvex bound ($t$-aspect) for the Riemann Zeta function was obtained by G.H. Hardy and J.E. Littlewood, based on the work of Weyl [33]. The $q$-aspect subconvexity was first proved by D.A. Burgess. Using cancellation in character sums in short intervals, he proved that

[TABLE]

D.R. Heath-Brown [8] proved the hybrid subconvexity, in both $(q,t)$-aspect, for Dirichlet $L$-functions. Since then, several improvements have been made of the above subconvex bounds.

For $GL(2)$ $L$-functions, $t$-aspect subconvexity was first proved by A. Good [7] for holomorphic forms using the spectral theory of automorphic forms. T. Meurman [19] then proved the result for Maass cusp forms. The $q$-aspect subconvexity for $GL(2)$ $L$-functions was first obtained by Duke-Friedlander-Iwaniec using a new form of circle method. Assuming $\chi$ to be primitive modulo $q$ and $\Re(s)=1/2$, they obtained

[TABLE]

V. Blomer and G. Harcos [1] obtained the Burgess exponent $3/8$ for a more general holomorphic or a Maass cusp form.

For degree three $L$-functions, it was initially solved for several special cases in [2],[21], [30], [22] (see [15] for $t$-aspect). However those only dealt with forms which are lifts of $GL(2)$ forms. In his series of papers [24],[23],[25],[26],[29], Munshi introduced a different approach to subconvexity through which he obtained subconvexity for more general degree three $L$-functions. Subconvexity bounds for Rankin–Selberg $L$-functions on $GL(2)\times GL(2)$ were known due to Sarnak [31], Kowalski–- Michel-–Vanderkam [13], and Lau–-Liu–-Ye [14]. The $t$-aspect subconvexity for genuine $GL(4)$ $L$-functions remains an important open problem.

Subconvexity bound for Rankin–Selberg $GL(3)\times GL(2)$ $L$-functions was first obtained by Li [15], where she considered the $L$-function of a fixed self-dual Hecke-Maass form for $SL(3,\mathbb{Z})$ twisted by a Hecke-Maass form for $SL(2,\mathbb{Z})$, or by a holomorphic Hecke cusp form for $SL(2,\mathbb{Z})$, in the eigenvalue aspect, or respectively in the weight aspect, of the $GL(2)$ form. Blomer [2] considered this problem in the level aspect where the twist is by special Hecke-Maass forms of prime square level.

Recently, using his separation of oscillation technique (as introduced in [23]), $t$-aspect subconvexity for $GL(3)\times GL(2)$ $L$- functions was obtained by Munshi [28], where the $GL(3)$ form is any Hecke-Maass cusp form for $SL(3,\mathbb{Z})$. Following similar strategy, we use the ‘conductor lowering’ trick as a device for separation of oscillations in the circle method as introduced in [24]. The key feature here is that the $GL(2),GL(3)$ Voronoi summations together transform the Ramanujan sum arising in the delta method to an additive character with respect to the $GL(3)$ variable. This was the crucial observation in [28]. As such, we save more after applying Poisson summation after Cauchy-Schwarz inequality. After a final application of the Cauchy-Schwarz inequality followed by Poisson summation, we observe that there is not enough saving in the ‘zero frequency’ and extra saving in the non-zero frequencies. To optimize this, we use the ‘Mass transfer’ trick introduced in [27] which essentially decreases the contribution of the zero frequency at the cost of an increase in the length of non-zero frequencies. This is worth comparing to the trick used by James Maynard in [18] (Section 3.4), where he is faced with the same problem of reducing the size of diagonal terms. He handles this by artificially introducing congruence conditions at the beginning, whereas we achieve it by increasing the size of the equation that we detect using the delta symbol (see (2.15) below). We proceed to prove the theorem for $f$ a Maass cusp form. Note that the same proof goes through for other cases as well with mild alterations (depending on their respective Voronoi summation formulas to be precise).

2. The set up

Let $\pi$ be a Hecke-Maass cusp form of type $(\nu_{1},\nu_{2})$ for $SL(3,\mathbb{Z})$, $f$ be a Maass cusp form of type $1/2+i\nu$ for $SL(2,\mathbb{Z})$ and $\chi$ be any non-trivial character $\bmod\,p$. The associated $L$-function $L(s,\pi\times f\times\chi)$ has arithmetic conductor $p^{6}$. Hence, from the approximate functional equation (see Th. 5.3 and Prop. 5.4 in [11], Theorem 3.9 in [3]) we get

[TABLE]

where

[TABLE]

and the smooth function $\mathcal{V}$ satisfies

[TABLE]

Remark 1** (Notation).**

In this paper the notation $\alpha\ll A$ will mean that for any $\epsilon>0$, there is a constant $c$ such that $|\alpha|\leq cAp^{\epsilon}$. The dependence of the constant on $\pi,f$ and $\epsilon$, when occurring, will be ignored. We will also use the phrase “negligible error” by which we mean an error term $O_{B}(p^{-B})$ for an arbitrary $B>0$.

We reduce our problem to estimating the following partial sums :

Lemma 2.1**.**

For any $0\leq\theta\leq 3$, we have

[TABLE]

where

[TABLE]

where $V$ is a smooth function supported in $[1,2]$ and satisfies $V^{(j)}(x)\ll_{j}1$.

Proof.

From (2.1) and (2.2) we get

[TABLE]

We divide the range in the last summation as

[TABLE]

where an optimal $\theta>0$ will be chosen later. Using the Ramanujan bound on average

[TABLE]

one sees that the last two ranges contributes at most $p^{(3-\theta)/2}$ to (2.4). Hence we have

[TABLE]

Using a smooth dyadic partition of unity $W$, we see that the second sum inside the absolute value above is at most

[TABLE]

which can we written as

[TABLE]

where

[TABLE]

where $V_{r,N}(x)=x^{-1/2}W(x)\mathcal{V}(Nr^{2}x/p^{3})$. Note that $V_{r,N}(x)$ is supported on $[1,2]$ and satisfies $V_{r,N}^{(j)}\ll_{j}1$ (bounds independent of $r,N$). Henceforth we ignore the dependence on $r,N$ and assume $V_{r,N}$ is same function for all $r,N$ and call it $V(x)$ (abusing notation). The claim follows. ∎

2.1. The delta method.

We now separate oscillations from $\lambda_{\pi}(r,n)$ and $\lambda(n)\chi(n)$ using a version of the delta method due to Duke, Friedlander and Iwaniec. More specifically, we will use the expansion $(20.157)$ given in Chapter 20 of [11]. Let $\delta:\mathbb{Z}\to\{0,1\}$ be defined by

[TABLE]

Then for $n\in\mathbb{Z}\cap[-2M,2M]$, we have

[TABLE]

where $Q=2M^{1/2}$. The function $g$ satisfies the following property (see $(20.158)$ and $(20.159)$ of [11], Lemma 15 of [9]).

[TABLE]

for any $A>1$. In particular, the second property implies that the effective range of the integral in (2.7) is $[-M^{\epsilon},M^{\epsilon}]$. It also follows from first the property that if $\min\{Q/q,1/|x|\}>Q^{\epsilon}$, we can replace $g$ by $1$ up to a negligible error term. And if $\min\{Q/q,1/|x|\}\leq Q^{\epsilon}$, then the third property implies $x^{j}\frac{\partial^{j}}{\partial x^{j}}g(q,x)\ll Q^{\epsilon}$. In any case, we can view $g(q,x)$ as a ‘nice’ weight function.

2.2. The mass transfer

A direct application of the delta method fails to beat the convexity bound for the diagonal contribution. So, we need employ the trick of ‘Mass transfer’ (amplification) as introduced by Munshi in [27].

From above discussion, it suffices study to the sums $S_{r}(N)$. Let $\mathscr{L}$ be the set of primes in $[L,2L]$. Recall the Hecke relation (see Theorem 6.4.11 of [6])

[TABLE]

where the second term occurs only if $l|n$ and the third term only if $l|r$. The latter does not happen since the size of $l\in\mathcal{L}$ and $r$ will be chosen such that

[TABLE]

for some $0<\eta<1$. Using this we have

[TABLE]

Using the fact that (see Lemma 5.1 of [17])

[TABLE]

we get

[TABLE]

The contribution coming from the second term is

[TABLE]

Note that from Hecke relations one has

[TABLE]

where $\theta_{0}=5/14$ (see [12]). By Cauchy-Schwarz, (2.11) and Ramanujan bound on average for the $GL(2)$ coefficient, we see that (2.10) is bounded by

[TABLE]

Hence we get

[TABLE]

Write

[TABLE]

The rest of the paper is devoted to estimation of the sum $\tilde{S}_{r}(N)$. Plugging in the $\delta$ function, we get

[TABLE]

where $U$ is any smooth function supported in $[1/2,5/2]$ and equals 1 on $[1,2]$. To complete the separation, we use the delta expansion in (2.7) with

[TABLE]

and detect $p|(m-nl)$ using additive characters to see that

[TABLE]

We first work out the details assuming

[TABLE]

The remaining cases provide us with smaller bounds and are dealt with later towards the end of the paper. In the case (2.18), we replace $a\to pa$ and re-write (2.17) as

[TABLE]

3. Sketch of the proof

For simplicity we assume the generic case $N\sim p^{3},q\sim Q=p\sqrt{L}$ and $r=1$. After applying the circle method and the conductor lowering trick by Munshi , our main object of study becomes of the form

[TABLE]

Trivially estimating at this stage gives $S(N)\ll N^{2}L$. So we want to save $NL$ plus a little more in the above sum. We apply Voronoi summation formulae to both $n$ and $m$ sums. In the $GL(2)$ Voronoi formula we save $N/pq\sim p/\sqrt{L}$ with the dual length $\sim pL$ and in the $GL(3)$ Voronoi formula we save $NL/(pq)^{3/2}\sim L^{1/4}$ with the dual length $\sim p^{3}\sqrt{L}$. We also save $\sqrt{Q}$ in the $a$ sum and $\sqrt{p}$ in the $u$ sum. Hence in total we have saved $p^{2}$ and at this stage trivial estimation gives

[TABLE]

So we need to save $pL$ and a little more in the tranformed sum

[TABLE]

where $\mathcal{J}$ is a integral transform which does not oscillate and

[TABLE]

where

[TABLE]

Note that we have already assumed square root cancellation in all the variables in $\mathcal{C}$.

We next apply Cauchy-Schwarz inequality to arrive at

[TABLE]

Opening the absolute value square, we apply Poisson summation formula on the $m$ sum. In the diagonal (zero frequency) we save $(LQpL)^{1/2}\sim pL^{5/4}$ and the contribution of the diagonal becomes $p^{4}L\times 1/pl^{5/4}=p^{3}/L^{1/4}$. Recall that the square-root saving in $\alpha\bmod p$ have been already considered. By linearity, the off-diagonal produces a factor $pqq^{\prime}$ together with a congruence relation of the form

[TABLE]

We save $q$ and $q^{\prime}$ from $n$ and $n^{\prime}$ respectively. Given $l,l^{\prime}q,q^{\prime},n$ and $n^{\prime}$, $\alpha^{\prime}$ is then determined by $\alpha$, say $f(\alpha)$. Finally, by viewing as a hypergeometric function (as defined by Katz), square-root cancellation in the sum

[TABLE]

is proved by Will Sawin in the appendix. Before Cauchy-Schwarz, the $\alpha$ and $m$ sum were contributing $p^{3}\sqrt{L}\times\sqrt{p}$. After Cauchy-Schwarz and Poisson summation, they are contributing $p^{3/2}L^{1/4}p^{1/4}p^{1/2}$. Hence in the off-diagonal, we effectively save $p^{3/2}L^{1/4}/p^{1/4}$ and its contribution to $S(N)$ is bounded by $p^{4}L\times 1/p^{5/4}L^{1/4}=p^{(3-1/4)}L^{3/4}$. With the optimal choice $L=p^{1/4}$, we get the claimed bound.

4. Voronoi summation formulae

We now proceed to transform the $GL(3)$ and $GL(2)$ sums in (2.17) using their respective Voronoi formulas.

4.1. The $GL(3)$ sum.

Let $\{\alpha_{i}:i=1,2,3\}$ be the Langlands parameters for $\pi$. Let $g$ be a compactly supported smooth function on $(0,\infty)$. For $l=0,1$, we define

[TABLE]

Set $\gamma_{\pm}(s)=\gamma_{0}\pm i\gamma_{1}(s)$ and let

[TABLE]

where $\sigma>-1+\max\{-\Re(\alpha_{1}),-\Re(\alpha_{2}),-\Re(\alpha_{3})\}$ and $\tilde{g}$ is the Mellin transform of $g$.

Lemma 4.1** ($GL(3)$ Voronoi summation formula**).

Let $c,d,r\geq 1$ be integers such that $(c,d)=1$. Then, with the above notation we have,

[TABLE]

Proof.

See [20] ∎

We have the following oscillatory behaviour of transforms $G_{\pm}$ due to X. Li [15].

Lemma 4.2**.**

Suppose $g(y)$ is a smooth function compactly supported on $[Y,2Y]$. Then for any fixed integer $K\geq 1$ and $yY\gg 1$,

[TABLE]

where $c_{j}$ and $d_{j}$ are constants depending on $\alpha_{i}$’s.

The asymptotic behaviour $G_{1}(y)$ is similar, with changes only in the constants $c_{j}$ and $d_{j}$. In $G_{0}(y)$, note that the oscillatory part is same for all $j$, whereas the magnitude of weight functions $g(z)/(yz)^{j/3}$ decreases as $j$ increases (this uses the assumption $yY\gg 1$). Hence the main term in $G_{0}(y)$ is well represented by the $j=1$ contribution, and we will be working with this term only.

Note that $(pq,ap+uq)=1$ by the assumption in (2.18). In our context we have $c=pq,d=ap+uq$ and $g(n)=e(nx/pqQ)V(n/lN)$. By the assumption in Lemma 4.2, the above asymptotic expansion can be used in the range

[TABLE]

i.e,

[TABLE]

where are able to replace $m$ sum in (2.17) essentially by

[TABLE]

By repeated integration by parts, we see that the integral is negligibly small unless $n_{1}^{2}n_{2}\ll M_{0}$, where

[TABLE]

Note that if $q\leq p^{-\epsilon}Q$, then

[TABLE]

in which case the integral above is seen to be negligibly small unless

[TABLE]

On other hand if $q\geq p^{-\epsilon}Q$, then (4.4) gives

[TABLE]

Hence we use expression (4.5) in the range

[TABLE]

Remark 2**.**

In the complementary range $xX\ll 1$, i.e.

[TABLE]

by shifting the contour in (4.1) to $\sigma=-1/3$ (say) without crossing the poles of the gamma factors we get

[TABLE]

So, in this case, we put the integral transform $G_{\pm}\left(\frac{n_{1}^{2}n_{2}}{c^{3}r}\right)$ inside the weight functions and proceed same as below. The only difference is now we will be working with a smaller range for the $n_{2}$ sum given by (4.11), and a smaller range for the Poisson variable at a later stage (see Remark 4 below).

4.2. The $GL(2)$ sum

Lemma 4.3** (GL(2) Voronoi summation formula.).**

Let $f$ be a Maass cusp form with Laplacian eigenvalue $1/4+\nu^{2},\nu\geq 0$, and $\varepsilon_{f}$ the eigenvalue of the involution operator. Let $h$ be a compactly supported smooth function on the interval $(0,\infty)$. Let $a>0$ an integer and $c\in\mathbb{Z}$ be such that $(a,c)=1$. Then we have

[TABLE]

where for $\nu>0$,

[TABLE]

and for $\nu=0$,

[TABLE]

Proof.

See appendix A.4 of [13]. ∎

As with the $GL(3)$ case, we need the following asymptotics for the Bessel functions to extract the oscillation (see [32], p. 206).

Lemma 4.4**.**

For $y>0$, the Bessel functions $J_{2i\nu}(y)$ and $K_{2i\nu}(y)$, ($\nu\in\mathbb{R}$) satisfy the following oscillatory behaviour

[TABLE]

where the function $P_{2i\nu}(y)$ (and similarly $Q_{2i\nu}(y)$) satisfies

[TABLE]

Now, the $n$ sum in (2.17) equals

[TABLE]

Recall that $(pl,q)=1$ by assumption (2.18) and $(p,l)=1$ since $l$ is prime and $l\sim L<p$. Note that if $b=ul\bmod p$, then the conductor of $n$ sum reduces to $(NL)^{2}/(pQ)^{2}\ll NL/p$, whereas the initial length is of size $N$. Hence the dual length is $\ll L/p\ll p^{-\delta}$, for some $\delta>0$ to be chosen later. So, the dual sum becomes negligible in the case $b=ul\bmod p$. For $b\neq ul$, applying the $GL(2)$ Voronoi formula to (4.16) with $c=pq,a=-l(ap+(u-b\bar{l})q)$ and $h(n)=e(-nlx/pqQ)U(n/N)$, we arrive at

[TABLE]

We proceed with the calculation for the first part involving $H^{-}$ (see Remark 3 below). Using (4.14) we see that the part with $H^{-}$ is essentially a sum of four sums of the type

[TABLE]

By (4.15), the weight function $(\sqrt{nN}/(pq))^{1/2}U_{\pm 2i\nu}\left(\frac{4\pi\sqrt{nNy}}{pq}\right)U(y)$ has bounded derivatives (bounds not depending on $n,N,p,q$). Like earlier, we assume that this function is the same for all $n,N,p,q$ and call it $U(y)$.

By repeated integration by parts, it is clear that the integral is negligibly small unless $n\ll N_{0}$, where

[TABLE]

Remark 3**.**

For the estimation of the part involving $H^{+}$, note that by the second property of (4.14), the function

[TABLE]

has bounded derivatives. Hence, in this case we can put (4.20) inside the weight functions and proceed same as below. The only difference is the potentially smaller range for the dual sum given by

[TABLE]

which again follows from the second property of (4.14).

5. Cauchy and Poisson

5.1. Cauchy inequality

Rearranging (4.18) we get,

[TABLE]

where

[TABLE]

and $J(n,q,l)$ is the integral in (4.18).

Combining (2.17), (4.5) and (5.1), we arrive at

[TABLE]

where

[TABLE]

where

[TABLE]

and

[TABLE]

Note that the inverse $\overline{ap+uq}$ is taken modulo $pq$. Splitting $q$ in dyadic blocks $q\sim C$ with

[TABLE]

the $C$ block in (5.3) is

[TABLE]

Applying the Cauchy-Schwarz inequality yields that (5.7) is at most

[TABLE]

where

[TABLE]

and

[TABLE]

5.2. Poisson summation

Smoothing out the $n_{2}$ sum in (5.10) with an appropriate bump function $W$, and opening the absolute value square, we arrive at

[TABLE]

where $q=q_{1}q_{2}$ and $q^{\prime}=q_{1}q^{\prime}_{2}$. We apply Poisson summation formula on the $n_{2}$ sum with modulus $pq_{1}q_{2}q_{2}^{\prime}r/n_{1}$. After a change of variable in the Fourier transform, we arrive at

[TABLE]

where the character sum $\mathcal{C}$ is given by

[TABLE]

and the integral transform $\mathcal{J}$ is given by

[TABLE]

where $\mathcal{I}$ as in (5.5). Note that the oscillation of $w$ in $\mathcal{I}(p^{-3\epsilon}M_{0}w,n,q)\overline{\mathcal{I}(p^{-3\epsilon}M_{0}w,n^{\prime},q^{\prime})}$ is of size

[TABLE]

It follows by repeated integration by parts in the $w$ integral that $\mathcal{J}$ is negligibly small unless

[TABLE]

that is,

[TABLE]

Remark 4**.**

In the case (4.11), the extra oscillation (5.15) is not present and we get a smaller dual range

[TABLE]

where $\tilde{M}_{0}$ as in (4.11).

For smaller values of $q$, there is oscillation in the integrand of $\mathcal{I}$ and hence we have the following bound

Lemma 5.1**.**

For $w\asymp 1$ we have,

[TABLE]

and consequently,

[TABLE]

Proof.

Note that if $q>p^{-\epsilon/3}Q$, then we obtain (5.19) trivially by taking absolute values inside $\mathcal{I}$. Hence we assume $Q/q\geq p^{\epsilon/3}$.

Changing variable $u=z-y$, we get

[TABLE]

Suppose first $\min\{Q/q,1/|x|\}\leq p^{\epsilon/8}$, that is $x\geq p^{-\epsilon/8}$. Then by the third property in (2.8), we have $x^{j}g^{(j)}(q,x)\ll p^{\epsilon/7}$, and hence from the $x$ integral, it follows by repeated integration by parts (after introducing a dyadic partition of unity to the $x$ integral) that

[TABLE]

for non-negligible contribution. Now consider the $y$ integral. Changing variable $y\mapsto y^{2}$, the $y$ integral becomes

[TABLE]

where

[TABLE]

We have

[TABLE]

where the last inequality occurs since $u\ll p^{-\epsilon/6}$. Hence the Lemma follows by (5.22) and the second derivative bound for the $y$ integral. On the other hand, if $\min\{Q/q,1/|x|\}>p^{\epsilon/8}$, then by the first property of (2.8), we can replace $g$ by $1$. The $x$ integral then saves a factor $pqQ/Nl$ and we are left with the $(u,y)$ integral

[TABLE]

Now if $u\leq p^{-\epsilon}$, then following the same argument from (5.22)-(5.25) we get the required square-root factor from the $y$ integral. If $u>p^{-\epsilon}$, then the Lemma follows after applying the second derivative bound for the $u$ integral in (5.26). ∎

It remains to estimate the character sum $\mathcal{C}$. As usual, the cases $n_{2}=0\bmod p$ and $n_{2}\neq 0\bmod p$ have to be dealt separately.

6. The non-zero frequencies modulo $p$ $(n_{2}\neq 0\bmod p)$

Note that it follows from (2.9), (2.18) and (5.6) that $(p,rq_{2}q_{2}^{\prime}q_{1}/n_{1})=(q_{2}q_{2}^{\prime},prq_{1}/n_{1})=1$, so that

[TABLE]

where

[TABLE]

[TABLE]

and

[TABLE]

We proceed for the estimation of $\mathcal{C}_{1},\mathcal{C}_{2}$ and $\mathcal{C}_{3}$ according as $n_{1}=0\bmod p$ or not.

6.1. ($n_{1}\neq 0\bmod p$)

We begin with the following estimate of $\mathcal{C}_{1}$, which is proved by Will Sawin in the appendix to this paper.

Lemma 6.1**.**

For $n_{1},n_{2}\neq 0\bmod p$, we have

[TABLE]

Proof.

We begin with some elementary transformations. Changing $b$ to $blu$, the $(b,u)$ sum in (6.2) becomes

[TABLE]

Similarly, changing $b^{\prime}$ to $b^{\prime}l^{\prime}u^{\prime}$, the $(b^{\prime},u^{\prime})$ sum in (6.2) becomes

[TABLE]

Substituting the above two simplifications into (6.2), and changing $(\alpha,\alpha^{\prime})\mapsto(\overline{\alpha},\overline{\alpha^{\prime}})$ and using the congruence condition between $\alpha,\alpha^{\prime}$, we get

[TABLE]

with $c_{1}=-n\overline{n_{1}l},c_{2}=-n^{\prime}\overline{n_{1}l^{\prime}},k_{1}=\overline{q_{2}}q_{2}^{\prime}$ and $k_{2}=-\overline{q_{2}}n_{2}$. Since $k_{2}\neq 0\bmod p$ by our assumption, the lemma follows from Theorem 9.1 in the appendix. ∎

Lemma 6.2**.**

We have

[TABLE]

Symmetrically, the same bound holds when $(q_{2}^{\prime},l^{\prime},n^{\prime},\alpha^{\prime})$ is replaced by $(q_{2},l,n,\alpha)$.

Proof.

Recall

[TABLE]

Also recall the assumptions $(q_{2},n_{1}r)=(q_{2}^{\prime},n_{1}r)=1$ and $q_{1}|(n_{1}r)^{\infty}$ from (5.6). Hence, given $\alpha^{\prime}$, there can be at most one $\alpha$. Hence we get

[TABLE]

∎

Lemma 6.3**.**

We have

[TABLE]

Proof.

We have

[TABLE]

Recall $(n_{1},q_{2}q_{2}^{\prime})=1$ from (5.6). Hence we get $\alpha=-n\overline{ln_{1}}\bmod d_{2}$. Note that it is implicit in congruence conditions that $(n,d_{2})=(n^{\prime},d_{2}^{\prime})=1$. Then using the congruence relation modulo $q_{2}q_{2}^{\prime}$, we obtain $-q_{2}^{\prime}n_{1}l\bar{n}=n_{2}\bmod d_{2}$. Hence

[TABLE]

∎

Combining Lemma 6.1, Lemma 6.2 and Lemma 6.3 we obtain,

Lemma 6.4**.**

For $n_{1},n_{2}\neq 0\bmod p$, we have

[TABLE]

Note that similar bounds holds when the roles of $(q_{2}^{\prime},l^{\prime},n^{\prime},\alpha^{\prime})$ and $(q_{2},l,n,\alpha)$ are interchanged.

6.2. $n_{1}=0\,(\bmod\,p)$

In this case, the last exponential factor in (6.2) is not present and consequently the character sum $\mathcal{C}_{1}$ is much simpler. Indeed, we now have

[TABLE]

The estimation of $\mathcal{C}_{2}$ and $\mathcal{C}_{3}$ remains the same as (6.10) and (6.13) respectively. Compared to the earlier case, we save an extra $p^{1/2}$ in the $\mathcal{C}_{1}$ sum, and consequently, we save more in this case.

Let $\Omega_{\neq 0\bmod p}$ denote the contribution of $n_{2}\neq 0\bmod p$ to $\Omega$ in (5.12), and let $\sum_{\neq 0\bmod p}$ be its contribution to (5.8).

Lemma 6.5**.**

We have

[TABLE]

and

[TABLE]

Proof.

We work out the bounds for $n_{1}\neq 0\bmod p$, since the other case contributes less as pointed out earlier. We substitute the bound from Lemma 6.4 in (5.12). We also use the inequality

[TABLE]

and consider the contribution of only the first term $|\lambda(n)|^{2}$, since the calculation for the second term is absolutely similar. Executing the above steps, we see that the contribution of the case under consideration towards $\Omega_{\neq 0\bmod p}$ is

[TABLE]

We substitute the bound for $\mathcal{J}\ll(q/Q)^{3}$ from Lemma (5.1). We next execute the sum over $n^{\prime}$ to get

[TABLE]

The sum over $\alpha,\alpha^{\prime}$ and $\beta^{\prime}$ in (6.19) is now bounded by $(q_{2},n_{2})rq_{1}/n_{1}$. Substituting, we get that (6.19) is

[TABLE]

The over $q_{2}$ is then bounded by $(n_{2},q_{2}^{\prime}n_{1}l)C/q_{1}$, which summed over $n_{2}$ is bounded by $N_{2}C/q_{1}$. Hence we are left with

[TABLE]

Executing the remaining sum we get

[TABLE]

where we have used the Ramanujan bound on average (2.5) for $l,l^{\prime}$ and $n$ sum. This is the first line of the lemma. Note that after substituting $N_{1}\ll N_{0}$, the contribution from $N_{1}$ in the parenthesis dominates the other. Substituting $N_{2}=(p^{\epsilon}n_{1}L^{1/2}N^{1/2})/(q_{1}p^{3/2})$, $N_{1}\ll N_{0}=p^{1+\epsilon}L,M_{0}=rp^{3/2+\epsilon}N^{1/2}L^{1/2}$ and $Q=\sqrt{(NL)/p}$, we get the second line of the first part of the lemma.

Using the bound from the second line of (6.16) in (5.8), we get

[TABLE]

where

[TABLE]

An application of Cauchy-Schwarz gives

[TABLE]

where in the last sum we applied the Ramanujan bound on average (2.5) and partial summation. The second part of the lemma follows after substituting (6.26) in (6.24). ∎

7. The zero frequencies modulo $p$

We proceed to obtain bounds for the sum character sum $\mathcal{C}$ in (5.13) for $n_{2}=0\bmod p$. First, we note that from the congruence relation

[TABLE]

in $\mathcal{C}$ (5.13), $n_{2}=0\bmod p$ implies

[TABLE]

Substituting (7.2) in $\mathcal{C}$, and executing the sum over $\alpha\bmod p$ part, we get

[TABLE]

Rearranging we get

[TABLE]

where

[TABLE]

and

[TABLE]

We obtain the bounds for the character sums $\mathcal{D}_{0},\mathcal{D}_{1}$ in the following two lemmas. For $\mathcal{D}_{0}$, we consider the following general sum which we will frequently encounter in this section.

Lemma 7.1**.**

For $c_{1},c_{2},\beta\in\mathbb{Z}$, define

[TABLE]

We have

[TABLE]

Proof.

Changing variables $\overline{u-b}\mapsto b_{1}$ and $\overline{u^{\prime}-b^{\prime}}\mapsto b_{2}$, we obtain

[TABLE]

First suppose $c_{1}=c_{2}=0\bmod p$. Then the $b_{1}$ and $b_{2}$ sum above is $-\chi(\beta u)$ and $-\bar{\chi}(u)$ respectively. Multiplying and summing over $u$, we get the desired bound in this case.

Next suppose $c_{1}=0\bmod p$ and $c_{2}\neq 0\bmod p$. Then the $b_{1}$ sum above evaluates to $-\chi(\beta u)$ and the $b_{2}$ sum is $\chi(c_{2})g_{\bar{\chi}}e(-c_{2}u/p)-\bar{\chi}(u)$. Hence

[TABLE]

in this case as well. Similar arguments holds for the $c_{2}=0\bmod p$ and $c_{1}\neq 0\bmod p$ case.

Now suppose $c_{1},c_{2}\neq 0\bmod p$, but $\beta=0\bmod p$. Then the $b_{1}$ sum is the Gauss sum $\bar{\chi}(-c_{1})g_{\chi}$ and the $b_{2}$ sum is $\chi(c_{2})g_{\bar{\chi}}e(-c_{2}u/p)-\bar{\chi}(u)$, and we get

[TABLE]

Hence for the rest of the argument, we assume $c_{1},c_{2},\beta\neq 0\bmod p$. Evaluating the last $b_{1},b_{2}$ sum in (7.9), we obtain,

[TABLE]

and the lemma follows. ∎

Lemma 7.2**.**

With $\mathcal{D}_{1}$ as in (7.5), we have

[TABLE]

Proof.

Consider the $(b,u)$ sum in $\mathcal{D}_{1}$. Executing the $u$ sum first we obtain

[TABLE]

Substituting, the $(b,u)$ sum in $\mathcal{D}_{1}$ (7.5) becomes

[TABLE]

One similarly obtains that the $(b^{\prime},u^{\prime})$ sum in $\mathcal{D}_{1}$ is $\ll p^{1/2}$, and the lemma follows. ∎

We combine the above results to get

Lemma 7.3**.**

We have

[TABLE]

where

[TABLE]

[TABLE]

and $\mathcal{C}_{2}$ and $\mathcal{C}_{3}$ as in (6.3) and (6.4) respectively.

Note that the $p|n_{1}$ case is already being considered in (6.15), which was independent of whether $n_{2}=0\bmod p$ or not.

Proof.

First note that, from the congruence condition

[TABLE]

in $\mathcal{D}_{2}$, $n_{2}=0$ implies $q_{2}=q_{2}^{\prime}$ and $\alpha=\alpha^{\prime}$.

Hence, in the first of case of (7.11), $\mathcal{D}_{0}$ is the sum $\tilde{\mathcal{C}}$ in (7.7) with $c_{1}=c_{2}=n\overline{q^{2}l},\beta=1$, multiplied by a factor of $\chi(\bar{l}l^{\prime})$. In this case, the Lemma 7.1 gives

[TABLE]

Also we have

[TABLE]

Hence the first case of the Lemma follows from (7.4), (7.10), (7.13) and (7.14).

In the second case, $\mathcal{D}_{0}$ is the sum $\tilde{\mathcal{C}}$ in (7.7) with $c_{1}=n\overline{q^{2}l},c_{2}=n^{\prime}\overline{q^{2}l^{\prime}}$ and $\beta=1$, multiplied by a factor of $\chi(\bar{l}l^{\prime})$. In this case we have $\beta c_{1}\neq c_{2}\bmod p$ and hence Lemma 7.1 gives

[TABLE]

Also, we have

[TABLE]

and the second part of the Lemma follows after combining (7.4), (7.10), (7.15) and (7.16).

In the last case, the number of such $n_{2}$ is $\ll N_{2}/p$. The bound in the Lemma in this case follows after using

[TABLE]

from Lemma 7.1 and observing that

[TABLE]

since $(q_{2}q_{2}^{\prime},rq_{1}/n_{1})=1$, where $\mathcal{C}_{2}$ and $\mathcal{C}_{3}$ as in (6.3) and (6.4) respectively. ∎

We now estimate the contribution of each of the cases of Lemma 7.3 to (5.8). Let $\sum_{0},\sum_{\mathcal{A}}$ and $\sum_{\mathcal{B}}$ denote the corresponding contribution of the cases in the above lemma towards (5.8).

Lemma 7.4**.**

We have

[TABLE]

Proof.

Recall that $n_{2}=0$ implies $q_{2}=q^{\prime}_{2}$, and hence $q^{\prime}=q$. Substituting the first bound from Lemma 7.3 and rearranging, we see that the contribution of this case towards $\Omega$ in (5.12) is dominated by

[TABLE]

As earlier in (6.18), we use the inequality

[TABLE]

and consider the contribution of the first term only. From the last congruence condition in (7.20), $\alpha$ is determined modulo the l.c.m of $d/(n_{1},d)$ and $d^{\prime}/(n_{1},d^{\prime})$, and note that

[TABLE]

where $[a,b]$ denotes the l.c.m. of $a$ and $b$. Since $d,d^{\prime}|q$ and $(n_{1},q_{2})=1$, we get $[(n_{1},d),(n_{1},d^{\prime})]\leq(n_{1},q_{1})$, so that the first term in the above inequality is greater than $[d,d^{\prime}]/(n_{1},q_{1})$. Hence we get

[TABLE]

We substitute this and the bound $(q/Q)^{3}$ for $\mathcal{J}$ in the last sum of (7.20).

We next execute the $n^{\prime}$ sum according to the cases $p(d,d^{\prime})\ll N_{1}$, $N_{1}\ll p(d,d^{\prime})\ll N_{1}L$ and $p(d,d)\gg N_{1}L$. In the first case, $p(d,d^{\prime})\ll N_{1}$ , we count $n^{\prime}$ satisfying the second last congruence in (7.20) to get

[TABLE]

For the $n$ sum, we can now use the Ramanujan bound on average and execute the remaining sum to get

[TABLE]

where we have used the Cauchy-Schwarz inequality and (2.5) for the $l,l^{\prime}$ sum. Substituting $N_{1}\ll pL$ and $Q=\sqrt{(NL)/p}$, we see that the above is bounded by

[TABLE]

Substituting, we see that the contribution of this case in (5.8) is

[TABLE]

where we used (6.26) in the last line. Substituting $M_{0}=rp^{3/2+\epsilon}N^{1/2}L^{1/2}$ gives the first term of the lemma.

In the second case ,$N_{1}\ll p(d,d^{\prime})\ll N_{1}L$, for each $n$, there is at most one $n^{\prime}\sim N_{1}$ satisfying the congruence relation $p(d,d^{\prime})|(nl^{\prime}-n^{\prime}l)$ (depends on $l,l^{\prime}$ as well). So, (7.20) in this case (recall that we are considering the contribution only the first term in the right hand side of (7.21)) is bounded by

[TABLE]

Using the Ramanujan bound on average on the $n$ sum, we bound the above sum by

[TABLE]

We use the upper bound $(d,d^{\prime})\ll N_{0}L/p$ and execute the remaining sum trivially. It is then clear that when $N_{1}$ is replaced by $N_{0}$, (7.29) differs from (7.25) by only a factor of $L$. Hence multiplying its square root, $L^{1/2}$, with the first term of the lemma, gives the second term of the lemma,

[TABLE]

For the last case $p(d,d)\gg N_{1}L$, we must have $nl^{\prime}-n^{\prime}l=0$ . In this case we keep the term $|\lambda_{f}(n)\lambda_{f}(n^{\prime})|$ as it is. Since $l$ and $l^{\prime}$ are primes, there are two possible cases, either $n=n^{\prime},l=l^{\prime}$ or $n=\tilde{n}l,n^{\prime}=\tilde{n}l^{\prime}$ for some $\tilde{n}\ll N_{1}/L$. In the latter case, (7.20) is bounded by

[TABLE]

Using the Hecke relation, one has

[TABLE]

where the second term exists only if $l|\tilde{n}$. Hence

[TABLE]

Using these bounds after an application of Cauchy-Schwarz inequality in (7.31), we get

[TABLE]

Executing the remaining sum, we see that the above is dominated by

[TABLE]

where we have used the Cauchy-Schwarz inequality and (2.5) for the $l,l^{\prime}$ sum.

For the case with $n=n^{\prime},l=l^{\prime}$, using the bound from (7.23), (7.20) reduces to

[TABLE]

Using the Ramanujan bound on average for $l$ and $n$ sum, and executing the remaining sum trivially, we see that this case produces the same bound as (7.34).

Note that when $N_{1}$ is replaced with $N_{0}=p^{1+\epsilon}L$, and $C$ with $Q=(NL/p)^{1/2}$, (7.34) differs from (7.25) by a factor of $(Qp)/(N_{0}L)\sim N^{1/2}/(p^{1/2}L^{3/2})$. Hence multiplying its square root with the first term of the lemma, we see that this case contributes

[TABLE]

towards (5.8), which is the third term of the lemma. This completes the proof the lemma. ∎

Lemma 7.5**.**

We have

[TABLE]

Proof.

For $\sum_{\mathcal{A}}$, substituting the second bound from Lemma 7.3, we see that the contribution of this case to $\Omega$ is bounded by

[TABLE]

We again appeal to the inequality

[TABLE]

and work out the details only for the first term. We obtain that (7.36) is bounded by

[TABLE]

Counting the number of $n^{\prime}$ with given constraint and executing the $n$ sum using the Ramanujan bound on average, we get

[TABLE]

which is smaller than the first bound in (6.16) by a factor of $p^{1/2}N_{2}q_{1}$. Consequently, $\sum_{\mathcal{A}}$ gets absorbed by the bound in the second line of (6.16).

Finally, for $\sum_{\mathcal{B}}$, recall that $p|n_{2},n_{2}\neq 0$. The number of such $n_{2}$ is $\ll N_{2}/p$. Using the earlier obtained bounds for $\mathcal{C}_{2},\mathcal{C}_{3}$ from Lemma 6.2 and Lemma 6.3, and doing the same calculation as in (6.19)-(6.23), with the only changes $p^{3}$ instead of $p^{5/2}$, and $N_{2}/p$ instead of $N_{2}$, we arrive at a bound for $\sum_{\mathcal{B}}$ which is smaller than (6.16) by factor of $p^{1/2}$. ∎

8. The special cases

In this section we discuss the cases complementary to (2.18). Treatment of these cases is similar to the general case above and yields smaller contributions.

8.1. $u=0$ and $(pl,q)=1$

In this case, the conductor of the GL(3) variable drops and we get huge saving from the Voronoi summation formulas itself. Hence, trivially estimating the dual sum which arises on applying the Voronoi formulae is enough for this case. The $m$ sum in (2.17) becomes

[TABLE]

Hence, after the application of the $GL(3)$ Voronoi formula, the $m$ sum (4.5) changes to

[TABLE]

where now the dual variables restrict up to $n_{1}^{2}n_{2}\ll M_{0}$, where $M_{0}=(rN^{2}L^{2})/(p^{3}Q^{3})=(rN^{1/2}L^{1/2})/p^{3/2}$. The treatment for the $GL(2)$ sum remains the same (with $u=0$). So the contribution of this case to $S_{r}(N)$ sum in (5.3) is

[TABLE]

where $N_{0}=pL$ as earlier, and

[TABLE]

and

[TABLE]

where

[TABLE]

so that

[TABLE]

Hence we get

[TABLE]

Following the arguments used in the proof of Lemma 5.1, (5.22) and the comment before (5.26) in particular, we get

[TABLE]

Substituting these two bounds, we see that the second line of (8.3) is bounded by

[TABLE]

Consequently, (8.3) is bounded by

[TABLE]

The last $q$-sum is bounded by $Q^{1/2}$. By partial summation and the Ramanujan bound on average (2.5), the $n$-sum is then bounded by $N_{0}^{3/4}$. Substituting these two bounds, we get

[TABLE]

Using the Ramanujan bound on average (2.5) and partial summation, we see that the last sum in (8.12) is bounded by $M_{0}^{2/3}$. Also, the sum over $\mathscr{L}$ is bounded by $L$ by (2.5). Substituting $M_{0}=p^{-3/2}N^{1/2}L^{1/2}r,N_{0}=pL$ and $Q=\sqrt{(NL)/p}$, we see that the contribution of this case towards $S_{r}(N)$ in (5.3) is bounded by

[TABLE]

where the last inequality holds since $N\ll p^{3}/r^{2}$ and $L<p$ (see (2.6), (2.9)). Recall that the term on the right hand side is the earlier bound from (6.17).

8.2. $(pl,q)>1$

In this subsection, we will focus only on the character sum changes and pass over the various technical points that have been already presented in full detail in the above sections.

Note that since $p,l$ are primes and $q\leq Q=(NL/p)^{1/2}\leq pL^{1/2}$, either $l|q$ or $p|q$. In the former case, we replace $q$ with $q^{\prime}=q/l\sim C/l$ and proceed the same as the general case above to obtain a smaller contribution. Except for a minor change in the $GL(2)$ Voronoi summation, the treatment for everything else is pretty much the same, and we omit the details for the sake of simplicity. We proceed with the details for the latter case.

8.2.1. $p|q$ and $(l,q)=1$

Writing $q=pq^{\prime}$, where

[TABLE]

(2.17) is now of the form

[TABLE]

Replacing $a+upq^{\prime}\mapsto a\,\,(\bmod p^{2}q^{\prime})$, (8.15) becomes

[TABLE]

The $GL(3)$ and $GL(2)$ Voronoi formulae transforms the $m$ and $n$ sum essentially into

[TABLE]

and

[TABLE]

respectively. Hence, (8.16) essentially becomes

[TABLE]

where the character sum $\mathfrak{C}$ is given by

[TABLE]

We have the following reduction of $\mathfrak{C}$ into additive character with respect to the $\tilde{m}$ variable :

Lemma 8.1**.**

We have

[TABLE]

where

[TABLE]

Proof.

Firstly, note that

[TABLE]

Substituting and executing the $b$-sum in (8.20), which is a gauss sum, we obtain

[TABLE]

Opening up the Kloosterman sum we obtain

[TABLE]

where

[TABLE]

and

[TABLE]

Note that by definition, $\mathcal{C}_{1}(x_{1})=f(x_{1},\tilde{n}\bar{l};q^{\prime})$, where $f$ defined as in (8.22). To evaluate $\mathcal{C}_{2}$, we write $\bar{a_{2}}=p\alpha+\beta\bmod p^{2}$, where $\alpha=0,1,\cdots,p-1$ and $\beta=1,2,\cdots,p-1$, to see that the $a_{2}$ sum in (8.25) becomes

[TABLE]

Writing $x_{2}=p\gamma+\tilde{n}\bar{l},\gamma=0,1,\cdots,p-1$, and substituting the last expression into (8.25), we obtain

[TABLE]

The lemma follows after substituting (8.26) and (8.24) in (8.23). ∎

Like earlier, we now apply the Cauchy-Schwarz inequality to (8.19) to get rid of the $GL(3)$ coefficient, but this time we keep the $q^{\prime}$-sum outside the square and $\tilde{n},l$ sum inside. We get that (8.19) is bounded by

[TABLE]

where

[TABLE]

Opening the absolute value square and substituting (8.21) we arrive at

[TABLE]

We now apply Poisson summation in the $\tilde{m}$-sum in (8.29) with modulus $p^{2}q^{\prime}r$. We observe that

[TABLE]

where we have used $N\leq p^{3+\epsilon}/r^{2}$ and $L=p^{\eta},\eta<1$ for the last two inequality in (8.30). Hence, it follows that only the zero-frequency contributes non-negligibly in dual side of the Poisson summation of the $\tilde{m}$ sum in (8.29). In other words,

[TABLE]

The above congruence condition implies

[TABLE]

Since $|nl^{\prime}-nl|\ll(p^{2}q^{\prime})^{2}/N\leq pL^{2}<p^{2}$ (see line after (9.4)), the second congruence in (8.32) implies

[TABLE]

Hence

[TABLE]

But we have

[TABLE]

Substituting we get

[TABLE]

From the earlier calculations (7.31)-(7.33) we have

[TABLE]

Substituting and executing the remaining $l,l^{\prime}$ sum in (8.36) using the Cauchy-Schwarz inequality and the Ramanujan bound on average (2.5) we get

[TABLE]

Substituting in (8.27), we see that the contribution of this case to $S_{r}(N)$ is bounded by

[TABLE]

Finally, substituting $Q=((NL)/p)^{1/2}$, we get the concluding bound for this subsection to be

[TABLE]

which gets absorbed into the error term of (2.13) since $L<p$.

9. Optimal choice for $L$ and $r$

Combining (2.12), Lemma 6.5, Lemma 7.4 and Lemma 7.5, we conclude

[TABLE]

Hence

[TABLE]

Using the upper bound $N\ll p^{3+\epsilon}/r^{2}$ and $r\leq p^{\theta}$ we get

[TABLE]

where we have assumed $L=p^{\eta}$, for some $0<\eta<1$. Equating the second and the third term, we choose $\eta=1/4$. We then choose $\theta=1/8$, to see that the second and the third term dominates the first and the last and we finally obtain

[TABLE]

Note that the choice $\eta=1/4,\theta=1/8$ is consistent with (2.9).

Acknowledgements

The author would like to thank Prof. Ritabrata Munshi for suggesting the problem, sharing his ideas and explaining his methods. He would also like to thank Jyoti Sengupta and Philippe Michel for their helpful comments, Will Sawin for his invaluable help and writing the appendix, and Indian Statistical Institute, Kolkata, for the excellent research environment. He is indebted to the anonymous reviewer for careful reading and many useful comments and suggestions. This is a part of the author’s Ph.D. thesis written under the supervision of Prof. R. Munshi and Prof. R. Sridharan.

**APPENDIX : A CHARACTER SUM BOUND VIA KATZ’S HYPERGEOMETRIC FUNCTIONS

** Will Sawin

Let $p$ be a prime. Let $\chi$ be a non-trivial Dirichlet character mod $p$. For $c_{1},c_{2},k_{1},k_{2}\in\mathbb{Z}$, define

[TABLE]

The goal of this appendix will be to prove:

Theorem 9.1**.**

We have $\mathcal{C}\ll p^{3/2}$ if $k_{2}\neq 0\mod p$.

To that end, define a sum $F_{c}(a)$ by

[TABLE]

so that

[TABLE]

Lemma 9.2**.**

If $c=0$ then $F_{c}(a)=O(p^{-1/2})$.

Proof.

We have

[TABLE]

This estimate will suffice to handle the cases when $c_{1}=0$ or $c_{2}=0$. The $c\neq 0$ case is more difficult.

Lemma 9.3**.**

If $c\neq 0$, then $F_{c}$ is the trace function of a sheaf $\mathcal{F}_{c}$ on $\mathbb{A}^{1}$. The sheaf $\mathcal{F}_{c}$ is lisse away from $0,c,\infty$, is pure of weight [math], is geometrically irreducible, and has conductor (in the sense of [FKMS19, Definition 4.3]) uniformly bounded.

Furthermore, the local monodromy representations at [math] and $c$ have nonzero invariants, while the local monodromy representation at $\infty$ does not.

Proof.

We have $\chi(c+ax)=\chi(c)\chi(1+(a/c)x)$ so $F_{c}(a)=\chi(c)F_{1}(a/c)$.

We thus restrict attention to the case $c=1$. In this case, the substitution $x\mapsto-x$ gives

[TABLE]

where $f(x)=\chi(x-1)$ and $*$ denotes multiplicative convolution

[TABLE]

We now relate $f$, and thus $F$, to finite field hypergeometric functions defined by Katz [Ka88], and hence to trace functions.

Let $\psi\colon\mathbb{F}_{p}\to\mathbb{C}^{\times}$ be an additive character. Katz defines [Ka88, (8.2.7)] a hypergeometric sum $\operatorname{Hyp}(\psi;\chi\textrm{'s};\psi\textrm{'s})(E,t)$, which, in the special case $\operatorname{Hyp}(\psi;1;\chi)(\mathbb{F}_{p},t)$, is given by

[TABLE]

[TABLE]

where $G(\overline{\chi})$ is a Gauss sum.

By the convolution property of hypergeometric functions [Ka88, (8.2.3)], this gives

[TABLE]

[TABLE]

Hence

[TABLE]

By definition [Ka88, (8.2.7)], $\operatorname{Hyp}(\psi;1,1;\chi,{\chi})(\mathbb{F}_{p},a)$ is the trace function of the sheaf Katz calls $\operatorname{Hyp}(!,\psi;1,1;\chi,{\chi})$. Thus $\operatorname{Hyp}(\psi;1,1;\chi,{\chi})(\mathbb{F}_{p},a/c)$ is the trace function of the pullback of that sheaf along multiplication by $1/c$, which Katz calls $\operatorname{Hyp}_{c}(!,\psi;1,1;\chi,{\chi})$ [Ka88, (8.2.13)] (he defines it as the pushforward along multiplication by $c$, but this is equivalent as pushforward along an invertible morphism is equivalent to pullback along the inverse map).

Hence $F_{c}(a)$ is the trace function of the $\alpha^{\operatorname{degree}}$ twist of $\operatorname{Hyp}_{c}(!,\psi;1,1;\chi,{\chi})(\mathbb{F}_{p},a)$, where $\alpha=\frac{\chi(c)}{p^{1/2}G(\overline{\chi},\psi)^{2}}$ satisfies $|\alpha|=p^{-3/2}$.

By [Ka88, Theorem 8.4.2(4)], $\operatorname{Hyp}_{c}(!,\psi;1,1;\chi,{\chi})$ is pure of weight $2+2-1=3$, so its $\alpha^{\operatorname{degree}}$ twist is pure of weight $3-3=0$.

The sheaf is geometrically irreducible by [Ka88, Theorem 8.4.2(1)].

The singularities and local monodromy representations are described by [Ka88, Theorem 8.4.2(8)]. In particular, this implies: $\mathcal{F}_{c}$ is lisse away from $0,c$. The local monodromy at $c$ is a transvection, meaning the codimension of its invariant subspace is $1$, so the dimension is $2-1=1>0$. The local monodromy at [math] is the tensor product of a trivial representation with a unipotent representation and thus has nontrivial invariants. The local monodromy representation at $\infty$ is the tensor product of a nontrivial representation of dimension $1$ with a unipotent representation and thus does not have nontrivial invariants.

Since the rank is $2$, the number of singularities is $3$, and the local monodromy at each singularity is tame, the conductor in the sense of [FKMS19, Definition 4.3] is bounded by $2+3+0=5$. ∎

Proof of Theorem 9.1.

If $c_{1}=c_{2}=0$ then by (9.5) and Lemma 9.2,

[TABLE]

When $c\neq 0$, a sheaf $\mathcal{F}_{c}$ as in Lemma 9.3 is pure of weight [math] and thus satisfies the standing assumption of [FKMS19, Remark 3.11]. A basic property of trace functions of sheaves that are pure of weight [math] they are $\ll 1$ is that they are bounded by the rank of the sheaf (and thus by the conductor). Thus, when $c_{1}=0$ but $c_{2}\neq 0$, or vice versa, we have

[TABLE]

This handles the cases when $c_{1}=0$ or $c_{2}=0$. For the remainder, we assume $c_{1},c_{2}\neq 0$.

We first handle the case $k_{1}\neq 0\mod p$. In this case, the pullback of $F_{c}$ along the rational linear transformations $\alpha\mapsto\overline{\alpha}$ and $\alpha\mapsto\overline{k_{1}\alpha+k_{2}}$ are thus also trace functions (of sheaves obtained by pulling back $\mathcal{F}_{c}$ along the same rational linear transformations [FKMS19, §8]). Since pullback by rational linear transformations preserves geometric irreducibility, these are also trace functions of geometrically irreducible sheaves. (We need $k_{1}\neq 0\mod p$ for $\alpha\mapsto\overline{k_{1}\alpha+k_{2}}$ to be a rational linear transformation and not a constant function.)

By the Riemann hypothesis in the form [FKMS19, Equation (5.4)], the sum

[TABLE]

unless the sheaves $[\alpha\mapsto\alpha^{-1}]^{*}\mathcal{F}_{c_{1}}$ and $[\alpha\mapsto(k_{1}\alpha+k_{2})^{-1}]^{*}\mathcal{F}_{c_{2}}$ are geometrically isomorphic.

If they were geometrically isomorphic, they would have to have the same local monodromy at each point. Since $\mathcal{F}_{c_{1}}$ has nontrivial local monodromy invariants at every finite point but not at $\infty$, its pullback $[\alpha\mapsto\alpha^{-1}]^{*}\mathcal{F}_{c_{1}}$ has nontrivial local monodromy at every point except the inverse image [math] of $\infty$.

Similarly, $[\alpha\mapsto(k_{1}\alpha+k_{2})^{-1}]^{*}\mathcal{F}_{c_{2}}$ has nontrivial local monodromy invariants at every point except the inverse image $-k_{2}/k_{1}$ of $\infty$.

Thus, the sheaves can only be isomorphic if $0=-k_{2}/k_{1}$, which happens only if $k_{2}=0$ (mod $p$), as desired.

Now in the case $k_{1}=0$, the sum $\frac{\mathcal{C}}{p}$ simplifies as

[TABLE]

and since $\overline{F_{c_{2}}}(\overline{k_{2}})=O(1)$, it suffices to prove that $\sum^{*}_{\begin{subarray}{c}\alpha(p)\\ \alpha\neq-k_{2}/k_{1}\end{subarray}}F_{c_{1}}(\overline{\alpha})=O(\sqrt{p})$ which, since $[\alpha\mapsto\alpha^{-1}]^{*}\mathcal{F}_{c_{1}}$ is geometrically irreducible and nontrivial, follows from another application of the Riemann hypothesis [FKMS19, Corollary 4.7] (or from a more elementary estimate by opening up the sum).

∎

I would like to thank Emmanuel Kowalski, Yongxiao Lin, and Philippe Michel for helpful conversations. This work was supported by NSF grant DMS-2101491.