Test vectors for trilinear forms, when two representations are unramified and one is special

TL;DR

This paper investigates explicit test vectors for trilinear forms on tensor products of representations of GL(2,F), focusing on cases where two are unramified and one is special, completing the understanding for low conductor cases.

Contribution

It extends the explicit construction of test vectors to new cases where two representations are unramified and one is special, especially for low conductor representations.

Findings

Explicit test vectors are identified for cases with low conductors.

Almost complete classification for when each representation has conductor at most 1.

Results generalize previous work by Gross and Prasad.

Abstract

Let F be a finite extension of Qp and G be GL(2,F). When V is the tensor product of three admissible, irreducible, finite dimensional representations of G, the space of G-invariant linear forms has dimension at most one. When a non zero linear form exists, one wants to find an element of V which is not in its kernel: this is a test vector. Gross and Prasad found explicit test vectors for some triple of representations. In this paper, others are found, and they almost complete the case when the conductor of each representation is at most 1.

Equations

$$\{\begin{array}{ccc}{V}_3& ⟶& \mathbb{C}\text{hfill}\\ v& ⟼& \ell (v_1\otimes v_2\otimes v)\end{array}$$

$${\Gamma }_0({\pi }_F^n)=\{\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\in Kc\equiv 0mod{{\pi }_F}^n\}$$

$$\{\begin{array}{ccc}{V}_3& ⟶& \mathbb{C}\text{hfill}\\ v& ⟼& \ell (v_1\otimes v_2^{\ast }\otimes v)\end{array}$$

$$\forall k\in K_f,\forall g\in G,f(gk)=f(g)$$

$$\forall g,g_0\in G⟨\pi (g),f⟩(g_0)=f(g_{0}g).$$

$$\gamma =\left(\begin{array}{cc}{\pi }_F^n& 0\\ 0& 1\end{array}\right)\mathrm{and}R={\gamma }^{-1}{\mathrm{M}}_2({\mathcal{O}}_F)\gamma .$$

$$R^{\ast }={\gamma }^{-1}K\gamma \mathrm{and}{R}^{\ast }\cap K={\Gamma }_0({\pi }_F^n).$$

$$v_2^{\ast }={\pi }_2({\gamma }^{-1})\cdot v_2$$

$$v_1\in {V_1}^K{v}_2^{\ast }\in {V_1}^{R^{\ast }}{v}_3\in {V_3}^{R^{\ast }\cap K}$$

$$\chi_{1}\Bigl{(}\begin{matrix}{\pi}_{F}&0\cr 0&{{\pi}_{F}}^{-1}\cr\end{matrix}\Bigr{)}\not=-1\quad{\rm or}\quad\chi_{2}\Bigl{(}\begin{matrix}{\pi}_{F}&0\cr 0&{{\pi}_{F}}^{-1}\cr\end{matrix}\Bigr{)}\not=-1$$

$$V_{1}\otimes V_{2}\otimes V_{3}\simeq\bigl{(}V_{1}\otimes\eta_{1}^{-1}\bigr{)}\otimes\bigl{(}V_{2}\otimes\eta_{2}^{-1}\bigr{)}\otimes\bigl{(}V_{3}\otimes\eta_{3}^{-1}\bigr{)}$$

$$\pi_{i}={\rm Ind}_{B}^{G}\chi_{i}\qquad{\rm with}\qquad\chi_{i}\Bigl{(}\begin{pmatrix}a&b\cr 0&d\cr\end{pmatrix}\Bigl{)}=\mu_{i}\Bigl{(}\frac{a}{d}\Bigr{)}$$

$$V_{1}\otimes V_{2}={\rm Res}_{G}\,{\rm Ind}_{B\times B}^{G\times G}\Bigl{(}\chi_{1}\times\chi_{2}\Bigr{)}$$

$$0\rightarrow{\rm ind}_{T}^{G}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)}\xrightarrow{{\rm\bf ext}}V_{1}\otimes V_{2}\xrightarrow{{\rm\bf res}}{\rm Ind}_{B}^{G}\Bigl{(}\chi_{1}\chi_{2}\delta^{\frac{1}{2}}\Bigr{)}\rightarrow 0$$

$$\Delta_{B\backslash G}=\Bigl{\{}(g,bg)\quad|\quad b\in B,\quad g\in G\Bigr{\}}.$$

$$F\Bigl{(}g,\begin{pmatrix}0&1\cr 1&0\cr\end{pmatrix}g\Bigr{)}=f(g).$$

$$0\rightarrow{\rm Hom}_{G}\Bigl{(}{\rm Ind}_{B}^{G}\Bigl{(}\chi_{1}\chi_{2}\delta^{\frac{1}{2}}\Bigr{)},\widetilde{V_{3}}\Bigr{)}\rightarrow{\rm Hom}_{G}\Bigl{(}V_{1}\otimes V_{2},\widetilde{V_{3}}\Bigr{)}\rightarrow{\rm Hom}_{G}\Bigl{(}{\rm ind}_{T}^{G}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)},\widetilde{V_{3}}\Bigr{)}\\ \hfill\downarrow\hskip 56.9055pt\\ \hfill\cdots\leftarrow{\rm Ext}_{G}^{1}\Bigl{(}{\rm Ind}_{B}^{G}\Bigl{(}\chi_{1}\chi_{2}\delta^{\frac{1}{2}}\Bigr{)},\widetilde{V_{3}}\Bigr{)}$$

$${\rm Ind}_{B}^{G}\Bigl{(}\chi_{1}\chi_{2}\delta^{\frac{1}{2}}\Bigr{)}\longrightarrow\widetilde{V_{3}}$$

$$\Psi :V_1\otimes V_2⟶\widetilde{V_3}$$

$$\ell \{\begin{array}{ccc}{V}_1\otimes V_2\otimes V_3& ⟶& \mathbb{C}\text{hfill}\\ v\otimes v^{\prime }\otimes v^{\prime \prime }& ⟼& \Psi (v\otimes v^{\prime }).v^{\prime \prime }\end{array}$$

$$\forall i\in \{1,2\},\forall b\in B,\forall k\in K,v_i(bk)={\chi }_i(b)\cdot \delta (b{)}^{\frac{1}{2}}$$

$$\forall g\in G,v_2^{\ast }(g)=v_2(g{\gamma }^{-1}).$$

$$(v_{1}\otimes v_{2}^{*})\Bigl{(}\begin{pmatrix}1&0\cr 0&1\cr\end{pmatrix}\Bigr{)}=v_{1}\Bigl{(}\begin{pmatrix}1&0\cr 0&1\cr\end{pmatrix}\Bigr{)}v_{2}\Bigl{(}{\gamma}^{-1}\Bigr{)}=v_{2}\Bigl{(}\begin{pmatrix}{\pi}_{F}^{-1}&0\cr 0&1\cr\end{pmatrix}\Bigr{)}=\mu_{2}({\pi}_{F})^{-1}|{\pi}_{F}|^{-\frac{1}{2}}=\frac{\sqrt{q}}{\mu_{2}({\pi}_{F})}$$

$$(v_{1}\otimes v_{2}^{*})\Bigl{(}\begin{pmatrix}0&1\cr 1&0\cr\end{pmatrix}\Bigr{)}=v_{2}\Bigl{(}\begin{pmatrix}0&1\cr 1&0\cr\end{pmatrix}\begin{pmatrix}{\pi}_{F}^{-1}&0\cr 0&1\cr\end{pmatrix}\Bigr{)}=v_{2}\Bigl{(}\begin{pmatrix}1&0\cr 0&{\pi}_{F}^{-1}\cr\end{pmatrix}\begin{pmatrix}0&1\cr 1&0\cr\end{pmatrix}\Bigr{)}=\frac{\mu_{2}({\pi}_{F})}{\sqrt{q}}.$$

$$(v_{1}\otimes v_{2}^{*})\Bigl{(}\begin{pmatrix}1&0\cr 0&1\cr\end{pmatrix}\Bigr{)}\not=(v_{1}\otimes v_{2}^{*})\Bigl{(}\begin{pmatrix}0&1\cr 1&0\cr\end{pmatrix}\Bigr{)}.$$

$$\ell (v_1\otimes v_2^{\ast }\otimes v_3)=\Psi (v_1\otimes v_2^{\ast }).v_3\ne 0$$

$${\rm Ext}_{G}^{1}\Bigl{(}{\rm Ind}_{B}^{G}\Bigl{(}\chi_{1}\chi_{2}\delta^{\frac{1}{2}}\Bigr{)},\widetilde{V_{3}}\Bigr{)}=0$$

$${\rm Hom}_{G}\Bigl{(}V_{1}\otimes V_{2},\widetilde{V_{3}}\Bigr{)}\simeq{\rm Hom}_{G}\Bigl{(}{\rm ind}_{T}^{G}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)},\widetilde{V_{3}}\Bigr{)}$$

$${\rm Hom}_{G}\Bigl{(}{\rm ind}_{T}^{G}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)},\widetilde{V_{3}}\Bigr{)}\simeq{\rm Hom}_{T}\Bigl{(}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)},\widetilde{{V_{3}}_{|T}}\Bigr{)}$$

$$\begin{matrix}\ell&\in&{\rm Hom}_{G}\Bigl{(}V_{1}\otimes V_{2}\otimes V_{3},\mathbb{C}\Bigr{)}\\ &&\downarrow\wr\\ \Psi&\in&{\rm Hom}_{G}\Bigl{(}V_{1}\otimes V_{2},\widetilde{V_{3}}\Bigr{)}\\ &&\downarrow\wr\\ \Phi&\in&{\rm Hom}_{G}\Bigl{(}{\rm ind}_{T}^{G}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)},\widetilde{V_{3}}\Bigr{)}\\ &&\downarrow\wr\\ \varphi&\in&{\rm Hom}_{T}\Bigl{(}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)},\widetilde{{V_{3}}_{|T}}\Bigr{)}\\ \end{matrix}$$

Full text

Test vectors for trilinear forms, when two representations are unramified and one is special

Louise Nyssen

1 Introduction

Let $F$ be a finite extension of $\mathbb{Q}_{p}$, with ring of integers ${\mathcal{O}}_{F}$, and uniformizing parameter ${\pi}_{F}$, whose residual field has $q$ elements. For $G={\rm GL}_{2}(F)$, let $(\pi_{1},V_{1})$, $(\pi_{2},V_{2})$ and $(\pi_{3},V_{3})$ be three irreducible, admissible, infinite dimensional representations of $G$. Using the theory of Gelfand pairs, Diprenda Prasad proves in [P] that that the space of $G$-invariant linear forms on $V_{1}\otimes V_{2}\otimes V_{3}$ has dimension at most one. He gives a precise criterion for this dimension to be one, that we will explain now.

Let $D_{F}^{*}$ be the group of invertible elements of the quaternion division algebra $D_{F}$ over $F$. When $(\pi_{i},V_{i})$ is a discrete serie representation of $G$, denote by $(\pi^{\prime}_{i},V^{\prime}_{i})$ the irreducible representation of $D^{*}_{F}$ associated to $(\pi_{i},V_{i})$ by the Jacquet-Langlands correspondance. Again, by the theory of Gelfand pairs, the space of $D^{*}_{F}$-invariant linear forms on $V^{\prime}_{1}\otimes V^{\prime}_{2}\otimes V^{\prime}_{3}$ has dimension at most one.

Let $\sigma_{i}$ be the two dimensional representations of the Weil-Deligne group of $F$ associated to the irreducible representations $\pi_{i}$. The triple tensor product $\sigma_{1}\otimes\sigma_{2}\otimes\sigma_{3}$ is an eight dimensional symplectic representation of the Weil-Deligne group, and has local root number $\varepsilon(\sigma_{1}\otimes\sigma_{2}\otimes\sigma_{3})=\pm 1$. When $\varepsilon(\sigma_{1}\otimes\sigma_{2}\otimes\sigma_{3})=-1$, one can prove that the representations $\pi_{i}$’s are all discrete serie representations of $G$.

Theorem 1**.**

(Prasad, theorem 1.4 of [P] ) Let $(\pi_{1},V_{1})$, $(\pi_{2},V_{2})$, $(\pi_{3},V_{3})$ be three irreducible, admissible, infinite dimensional representations of $G$ such that the product of their central characters is trivial. If all the representations $V_{i}$’s are cuspidal, assume that the residue characteristic of $F$ is not 2. Then

$\centerdot$* $\varepsilon(\sigma_{1}\otimes\sigma_{2}\otimes\sigma_{3})=1$ if and only if there exist a non zero $G$-invariant linear form on $V_{1}\otimes V_{2}\otimes V_{3}$*

$\centerdot$* $\varepsilon(\sigma_{1}\otimes\sigma_{2}\otimes\sigma_{3})=-1$ if and only if there exist a non zero $D_{k}^{*}$ invariant linear form on $V^{\prime}_{1}\otimes V^{\prime}_{2}\otimes V^{\prime}_{3}$.*

Once you got a non zero $G$-invariant linear form $\ell$ on $V_{1}\otimes V_{2}\otimes V_{3}$, or a non zero $D_{k}^{*}$-invariant linear form $\ell^{\prime}$ on $V^{\prime}_{1}\otimes V^{\prime}_{2}\otimes V^{\prime}_{3}$, you want to find a vector in $V_{1}\otimes V_{2}\otimes V_{3}$ which is not in the kernel of $\ell$, or a vector in $V^{\prime}_{1}\otimes V^{\prime}_{2}\otimes V^{\prime}_{3}$ which is not in the kernel of $\ell^{\prime}$. Such a vector is called a test vector. At first sight, it appears to have strong connections with the new vectors $v_{1}$, $v_{2}$ and $v_{3}$ of the representations $\pi_{1}$, $\pi_{2}$ et $\pi_{3}$.

Theorem 2**.**

(Prasad, theorem 1.3 of [P]) When all the $\pi_{i}$’s are unramified principal series representations of $G$, $v_{1}\otimes v_{2}\otimes v_{3}$ is a test vector for $\ell$.

Theorem 3**.**

(Gross and Prasad, proposition 6.3 of [G-P]) When all the $\pi_{i}$’s are unramified twists of the special representation of $G$ :

$\centerdot$* if $\varepsilon(\sigma_{1}\otimes\sigma_{2}\otimes\sigma_{3})=1$, then $v_{1}\otimes v_{2}\otimes v_{3}$ is a test vector for $\ell$,*

$\centerdot$* if $\varepsilon(\sigma_{1}\otimes\sigma_{2}\otimes\sigma_{3})=-1$, let $R^{\prime}$ be the unique maximal order in $D_{F}$. Then the open compact subgroup $R^{\prime*}\times R^{\prime*}\times R^{\prime*}$ fixes a unique line in $V^{\prime}_{1}\otimes V^{\prime}_{2}\otimes V^{\prime}_{3}$. Any vector on this line is a test vector for $\ell^{\prime}$.*

The proof by Gross and Prasad of the first statement of this theorem, actually contains another result:

Theorem 4**.**

When two of the $\pi_{i}$’s are unramified twists of the special representation of $G$ and the third one belongs to the unramified principal serie of $G$, $v_{1}\otimes v_{2}\otimes v_{3}$ is a test vector for $\ell$.

But the paper [G-P] ends up with an evidence that $v_{1}\otimes v_{2}\otimes v_{3}$ is not always a test vector for $\ell$. Let $K={\rm GL}({\mathcal{O}}_{F})$ be the maximal compact subgroup of $G$. If $\pi_{1}$ and $\pi_{2}$ are unramified and if $\pi_{3}$ has conductor $n\geq 1$, $\ell$ being $G$-invariant, $v_{1}$ and $v_{2}$ being $K$-invariant, one gets a $K$-invariant linear form

[TABLE]

which must be 0 since $\pi_{3}$ is ramified. Then $\ell(v_{1}\otimes v_{2}\otimes v_{3})=0.$

Now Gross and Prasad make the following suggestion. Let ${\Gamma}_{0}({\pi}_{F}^{n})$ be the congruence subgroup

[TABLE]

and $R$ be a maximal order ${\rm M}_{2}(F)$ such that $R^{*}\cap K={\Gamma}_{0}({\pi}_{F}^{n})$. If $v_{2}^{*}$ is a $R^{*}$-invariant vector in$V_{2}$, the linear form

[TABLE]

is invariant under the action of $R^{*}\cap K={\Gamma}_{0}({\pi}_{F}^{n})$, and one can still hope that $v_{1}\otimes v_{2}^{*}\otimes v_{3}$ is a test vector for $\ell$. In theorem 5 we will focus on the case $n=1$, and prove that $v_{1}\otimes v_{2}^{*}\otimes v_{3}$ is a test vector for $\ell$, up to a condition on $\pi_{1}$ and $\pi_{2}$. This will almost complete the study of test vectors when the $\pi_{i}^{\prime}s$ have ramification 0 or 1.

In the long term, the search for test vectors is motivated by the subconvexity problem for $L$-functions. Roughly speaking, one wants to bound some $L$-functions along the critical line $\Re(z)=\frac{1}{2}$. A recent and successful idea in this direction has been to relate triple products of automorphic forms to special values of $L$-functions on the critical line. In [B-R 1] and [B-R 2] Joseph Bernstein and Andre Reznikov did this in the eigenvalue aspect , and in [V] Akshay Venkatesh did it in the level aspect. More details about subconvexity and those related techniques will be found in [M-V]. Test vectors are key ingredients. Bernstein and Reznikov use an explicit test vector. Venkatesh uses a theoretical one, but explains that the bounds would be better with an explicit one (see paragraph 5 of [V]). Unfortunately, the difficulty of finding them increases with the ramification of the representations involved.

There is an extension of Prasad’s result in [H-S], where Harris and Scholl prove that the dimension of the space of $G$-invariant linear forms on $V_{1}\otimes V_{2}\otimes V_{3}$ is one when $\pi_{1}$, $\pi_{2}$ and $\pi_{3}$ are principal series representations, either irreducible or reducible with their unique irreducible subspace, infinite dimensional. They apply the global setting of this to the construction of elements in the motivic cohomology of the product of two modular curves constructed by Beilinson.

I would like to thank Philippe Michel for suggesting this problem, and Wen-Ching Winnie Li who invited me to spend one semester at PennState University where I could write the first draft of this paper.

2 Strategy

2.1 Notations

Let $(\rho,W)$ be a smooth representation of a closed subgroup $H$ of $G$. Let $\Delta_{H}$ be the modular function on $H$. The induction of $\rho$ from $H$ to $G$ is a representation $\pi$ whose space is the space {\rm Ind}_{H}^{G}\bigl{(}\rho\bigr{)} of functions $f$ from $G$ to $W$ satisfying the two following conditions :

(1) $\forall h\in H\quad\forall g\in G\quad f(hg)={\Delta_{H}}^{-\frac{1}{2}}(h)\rho(h)f(g)$,

(2) there exist an open compact subgroup $K_{f}$ of $G$ such that

[TABLE]

where $G$ acts by right translation. The resulting function will be denoted $\langle\pi(g),f\rangle$ that is

[TABLE]

With the additional condition that $f$ must be compactly supported modulo $H$, one gets the compact induction denoted by ${\rm ind}_{H}^{G}$. When $G/H$ is compact, there is no difference between ${\rm Ind}_{H}^{G}$ and ${\rm ind}_{H}^{G}$.

Let $B$ the Borel subgroup of upper triangular matrices in $G$ and $T$ be the diagonal torus. Then we will use $\delta={\Delta_{B}}^{-1}$ with \delta\Bigl{(}\begin{pmatrix}a&b\cr 0&d\cr\end{pmatrix}\Bigl{)}=|\frac{a}{d}| and $\Delta_{T}$ is trivial. The quotient $B\backslash G$ is compact and can be identified with ${\mathbb{P}}^{1}(F)$.

For a smooth representation $V$ of $G$, $V^{*}$ is the space of linear forms on $V$. The contragredient representation $\widetilde{\pi}$ is given by the action of $G$ on $\widetilde{V}$, the subspace of smooth vectors in $V^{*}$. If $H$ is a subgroup of $G$, $\widetilde{V}\subset\widetilde{V_{|H}}\subset V^{*}$.

More information about induced and contragredient representations will be found in [B-Z].

Let $(\pi_{1},V_{1})$, $(\pi_{2},V_{2})$ and $(\pi_{3},V_{3})$ be three irreducible, admissible, infinite dimensional representations of $G$ such that the product of their central characters is trivial. Assume that $\pi_{1}$ and $\pi_{2}$ are unramified principal series, and that $\pi_{3}$ has conductor $n\geq 1$. Then, according to theorem 1, there exist a non-zero, $G$-invariant linear form $\ell$ on $V_{1}\otimes V_{2}\otimes V_{3}$, and we are looking for a vector $v$ in $V_{1}\otimes V_{2}\otimes V_{3}$ which is not in the kernel of $\ell$. In order to follow Gross and Prasad suggestion, we will consider

[TABLE]

One can easily check that

[TABLE]

If $v_{1}$, $v_{2}$ and $v_{3}$ denote the new vectors of $\pi_{1}$, $\pi_{2}$ and $\pi_{3}$, the vector

[TABLE]

is invariant under the action of $R^{*}$. Hence we can write

[TABLE]

According to Gross and Prasad $v_{1}\otimes v_{2}^{*}\otimes v_{3}$ should be a test vector for $\ell$ , for any $n\geq 1$. In this paper, we will focus on the case where $n=1$. We will need the following condition regarding $\pi_{1}$ and $\pi_{2}$: since they are unramified principal series, they are induced from characters $\chi_{1}$ and $\chi_{2}$ of $B$, that are required to satisfy

[TABLE]

We will prove

Theorem 5**.**

If $n=1$, and (1) is satisfied, $v_{1}\otimes v_{2}^{*}\otimes v_{3}$ is a test vector for $\ell$.

The proof will follow the same pattern as Prasad’s proof of theorem 2 in [P], with the necessary changes.

2.2 Central characters

Let $\omega_{1}$, $\omega_{2}$ and $\omega_{3}$ be the central caracters of $\pi_{1}$, $\pi_{2}$ and $\pi_{3}$. Notice that the condition $\omega_{1}\omega_{2}\omega_{3}=1$ derives from the $G$-invariance of $\ell$. Since $\pi_{1}$ and $\pi_{2}$ are unramified, $\omega_{1}$ and $\omega_{2}$ are unramified too, and so is $\omega_{3}$ because $\omega_{1}\omega_{2}\omega_{3}=1$. Let $\eta_{i}$, for $i\in\{1,2,3\}$ be unramified quasi-characters of $F^{*}$ with $\eta_{i}^{2}=\omega_{i}$ and $\eta_{1}\eta_{2}\eta_{3}=1$. Then

[TABLE]

as a representation of $G$. Hence it is enough to prove theorem 4 when the central characters of the representations are trivial.

When $n=1$, it is also enough to prove theorem 5 when $V_{3}$ is the special representation $\rm Sp$ of $G$ : take $\eta_{3}$ to be the unramified character such that $V_{3}=\eta_{3}\otimes\rm Sp$.

2.3 Prasad’s exact sequences

Let us now explain how Prasad finds $\ell$. It is equivalent to search $\ell$ or to search a non zero element in {\rm Hom}_{G}\Bigl{(}V_{1}\otimes V_{2},\widetilde{V_{3}}\Bigr{)}. Since the central characters of $\pi_{1}$ and $\pi_{2}$ are trivial, there are unramified characters $\mu_{1}$ and $\mu_{2}$ such that for $i=1$ and $i=2$

[TABLE]

Hence

[TABLE]

where $G$ is diagonally embedded in $G\times G$ for the restriction. The action of $G$ on $B\times B\backslash G\times G={\mathbb{P}}^{1}(F)\times{\mathbb{P}}^{1}(F)$ has precisely two orbits : the first one is $\{(u,v)\in{\mathbb{P}}^{1}(F)\times{\mathbb{P}}^{1}(F)\quad|\quad u\not=v\}$, it is open and can be identified with $T\backslash G$, the second one is the diagonal embedding of ${\mathbb{P}}^{1}(F)$ in ${\mathbb{P}}^{1}(F)\times{\mathbb{P}}^{1}(F)$, it is closed and it can be identified with $B\backslash G$. Then, we have a short exact sequence of $G$-modules

[TABLE]

The surjection ${\rm\bf res}$ is the restriction of functions from $G\times G$ to the diagonal part of $B\backslash G\times B\backslash G$, that is

[TABLE]

The injection ${\rm\bf ext}$ takes a function f\in{\rm ind}_{T}^{G}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)} to a function F\in{\rm Ind}_{B\times B}^{G\times G}\Bigl{(}\chi_{1}\times\chi_{2}\Bigr{)} given by the relation

[TABLE]

Applying the functor {\rm Hom}_{G}\Bigl{(}\cdot\,,\widetilde{V_{3}}\Bigr{)}, one gets a long exact sequence

[TABLE]

2.4 The simple case

The situation is easier when $n=1$ and $\mu_{1}\mu_{2}|\cdot|^{\frac{1}{2}}=|\cdot|^{-\frac{1}{2}}$. Then $\pi_{3}$ is special and there is a natural surjection

[TABLE]

whose kernel is the one dimensional subspace of constant functions. Thanks to the exact sequence (2) one gets a surjection

[TABLE]

which corresponds to

[TABLE]

The surjection $\Psi$ vanishes on $v_{1}\otimes v_{2}^{*}$ if and only if ${\rm\bf res}(v_{1}\otimes v_{2}^{*})$ has constant value on ${\mathbb{P}}^{1}(F)\simeq B\backslash G$. Easy computation proves that it is not constant : the new vectors $v_{1}$ and $v_{2}$ are functions from $G$ to $\mathbb{C}$ such that

[TABLE]

and

[TABLE]

Then

[TABLE]

and

[TABLE]

The representation $\pi_{2}$ is principal so $\frac{\sqrt{q}}{\mu_{2}({\pi}_{F})}\not=\frac{\mu_{2}({\pi}_{F})}{\sqrt{q}}$ and

[TABLE]

Hence, $\Psi$ does not vanish on $v_{1}\otimes v_{2}^{*}$. Then, $v_{1}$ being $K$-invariant and $v_{2}^{*}$ being $R^{*}$-invariant, $\Psi(v_{1}\otimes v_{2}^{*})$ is a non zero ${\Gamma}_{0}({\pi}_{F}^{n})$-invariant element of $\widetilde{V_{3}}$, that is, a new vector for $\widetilde{\pi_{3}}$, and it does not vanish on $v_{3}$ :

[TABLE]

Then $v_{1}\otimes v_{2}^{*}\otimes v_{3}$ is a test vector for $\ell$.

2.5 The other case

If $n\geq 2$ or $\mu_{1}\mu_{2}|\cdot|^{\frac{1}{2}}\not=|\cdot|^{-\frac{1}{2}}$ then {\rm Hom}_{G}\Bigl{(}{\rm Ind}_{B}^{G}\Bigl{(}\chi_{1}\chi_{2}\delta^{\frac{1}{2}}\Bigr{)},\widetilde{V_{3}}\Bigr{)}=0 and by corollary 5.9 of [P]

[TABLE]

Through the long exact sequence (3) we get an isomorphism

[TABLE]

and by Frobenius reciprocity

[TABLE]

By lemmas 8 and 9 of [W], this latter space is one dimensional. Thus, we have a chain of isomorphic one dimensional vector spaces

[TABLE]

with generators $\ell$, $\Psi$, $\Phi$ and $\varphi$ corresponding via the isomorphisms. Notice that $\varphi$ is a linear form on $V_{3}$ such that

[TABLE]

Lemma 1**.**

$\varphi(v_{3})\not=0$.

Proof : this is proposition 2.6 of [G-P] with the following translation :

• the local field $F$ is the same,

• the quadratic extension $K/F$ of Gross and Prasad is $F\times F$ (this case in included in their proof) and their group $K^{*}$ is our torus $T$,

• the infinite dimensional representation $V_{1}$ of Gross and Prasad is our $\pi_{3}$,

• the one dimensional, unramified representation $V_{2}$ of Gross and Prasad is $\frac{\chi_{1}}{\chi_{2}}$.

Then the representation that Gross and Prasad call $V$ is $\frac{\chi_{1}}{\chi_{2}}\otimes\pi_{3}$ and their condition (1.3) is exactly our condition (4). The character $\omega$ of Gross and Prasad, which is the central character of their $V_{1}$, is trivial for us. Let $\alpha_{K/F}$ be the quadratic character of $F^{*}$ associated to the extension $K/F$ by local class-field theory, and let $\sigma$ and $\sigma_{3}$ be the representations of the Weil-Deligne group of $F$ associated to $\frac{\chi_{1}}{\chi_{2}}$ and $\pi_{3}$. Thanks to [T] we know that $\varepsilon(\sigma\otimes\sigma_{3})=\alpha_{K/F}(-1)$ because $K$ is not a field, and we are in the first case of proposition 2.6.

The restriction of $\frac{\chi_{1}}{\chi_{2}}\otimes\pi_{3}$ to the group

[TABLE]

fixes a unique line in $V_{3}$ : it is the line generated by the new vector $v_{3}$. According to Gross and Prasad, a non-zero linear form on $V_{3}$ which satisfies (4) cannot vanish on $v_{3}$. $\hfill\Box$

We still need to prove that $\ell(v_{1}\otimes v_{2}^{*}\otimes v_{3})\not=0$. For the reason described at the end of section 2.4, it is enough to prove that

[TABLE]

is non zero in $\widetilde{V_{3}}$.

In order to do that we want to build a function $F$ in $V_{1}\otimes V_{2}$, of the form

[TABLE]

which vanishes on the closed orbit of $G$ in ${\mathbb{P}}^{1}(F)\times{\mathbb{P}}^{1}(F)$. Then, $F$ is in the kernel of ${\rm\bf res}$ so it is the image by ${\rm\bf ext}$ of a function f\in{\rm ind}_{T}^{G}\Bigl{(}\frac{\chi_{1}}{\chi_{2}}\Bigr{)}. The important point is that $f$ must be the characteristic function of the orbit of the unit in the decomposition of $T\backslash G$ under the action of ${\Gamma}_{0}({\pi}_{F}^{n})$, which means :

[TABLE]

Then, the function

[TABLE]

is invariant by the action of $T$ by left translation and we can do the following computation: on the one hand

[TABLE]

where $\lambda$ is a non zero constant. Thanks to lemma 1 we know that $\varphi(v_{3})\not=0$ then

[TABLE]

On the other hand, it comes from (5) that

[TABLE]

then $\Psi(v_{1}\otimes v_{2}^{*})\not=0$ and $v_{1}\otimes v_{2}^{*}\otimes v_{3}$ is a test vector for $\ell$.

3 Calculations

3.1 The big function $F$ and the little function $f$

The function $F$ has to be ${\rm\bf ext}(f)$, where $f$ is the function described by formula (6). Since $F$ is in V_{1}\otimes V_{2}={\rm Res}_{G}\,{\rm Ind}_{B\times B}^{G\times G}\Bigl{(}\chi_{1}\times\chi_{2}\Bigr{)} and $G=BK$, it is enough to know the values of $F$ on $K\times K$.

Lemma 2**.**

$\forall(k,k^{\prime})\in K\times K$,

[TABLE]

Proof : $F$ must vanish on

[TABLE]

The other part of $B\backslash G\times B\backslash G$ can be identified with $T\backslash G$ via the bijection

[TABLE]

through which, the orbit of the unit in $T\backslash G$ under the action of ${\Gamma}_{0}({\pi}_{F}^{n})$ corresponds to

[TABLE]

Pick any $(k,k^{\prime})\in K\times K$. If $k^{\prime}\in Bk$, then $k^{\prime}\in{\Gamma}_{0}({\pi}_{F}^{n})$ if and only if $k\in{\Gamma}_{0}({\pi}_{F}^{n})$, and $k^{\prime}\in{\Gamma}_{0}({\pi}_{F})$ if and only if $k\in{\Gamma}_{0}({\pi}_{F})$. Else, put

[TABLE]

There exist $(b_{1},b_{2})\in B\times B$ such that

[TABLE]

Then

[TABLE]

It follows that $(k,k^{\prime})$ corresponds to an element of the orbit of the unit in the decomposition of $T\backslash G$ under the action of ${\Gamma}_{0}({\pi}_{F}^{n})$ if and only if $k\in{\Gamma}_{0}({\pi}_{F}^{n})$ and $k^{\prime}\notin{\Gamma}_{0}({\pi}_{F})$. $\hfill\Box$

3.2 The big function $F$ when $n=1$

Now we have to find the coefficients $a_{i}$ and elements $g_{i}$ of (5) to get the right $F$. This can be done for $n=1$. For the sake of simplicity, for any family $(g_{i})$ of elements of $G$, and $(a_{i})$ some complex numbers, denote

[TABLE]

Let $\{\,\tau_{0},\dots\tau_{q-1}\,\}$ be a set of representatives of ${\mathcal{O}}_{F}/{\pi}_{F}{\mathcal{O}}_{F}$ in ${\mathcal{O}}_{F}$, and $A$ be the number

[TABLE]

which can be defined because the representations $\pi_{1}$ and $\pi_{2}$ are principal so $\mu_{1}({\pi}_{F})^{2}-q\not=0$ and $\mu_{2}({\pi}_{F})^{2}-q\not=0$.

Lemma 3**.**

When $n=1$ and $1+\mu_{1}({\pi}_{F})^{2}\not=0$ the function $F$ is given by

[TABLE]

When $n=1$ and $1+\mu_{2}({\pi}_{F})^{2}\not=0$ the function $F$ is given by

[TABLE]

Proof : for $g\in G$ and $k=\begin{pmatrix}a&b\cr c&d\cr\end{pmatrix}\in K$ in order to compute

[TABLE]

write

[TABLE]

with $k_{1}$ and $k_{2}$ in $K$. Then

[TABLE]

The following tables give the pairs \Bigl{(}\Bigl{<}\pi_{1}(g),v_{1}\Bigr{>}(k),\Bigl{<}\pi_{2}(g),v_{2}^{*}\Bigr{>}(k)\Bigr{)}. The entries, are : an element $g$ in $G$, $val(c)$ and $val(d)$ where $(c,d)$ is the second line of $k$.

The first table is inspired by the formula

[TABLE]

[TABLE]

Fix

[TABLE]

It comes out that $\forall(k,k^{\prime})\in K\times K$

[TABLE]

Now consider ${\gamma}^{-1}F_{1}(k,k^{\prime})$. On the one hand

[TABLE]

On the other hand, for any $(k,k^{\prime})$ in $K\times K$,

[TABLE]

Take $k=\begin{pmatrix}a&b\cr c&d\cr\end{pmatrix}$, and $k{\gamma}^{-1}=\begin{pmatrix}\frac{a}{{\pi}_{F}}&b\cr\frac{c}{{\pi}_{F}}&d\cr\end{pmatrix}$. If ${\rm val}\,c=0$, then ${\rm val}\,d+1\geq{\rm val}\,c$ and

[TABLE]

with

[TABLE]

If ${\rm val}\,c\geq 1$, then ${\rm val}\,d=0$ and

[TABLE]

with

[TABLE]

The same calculation with $k^{\prime}$ leads to the following:

[TABLE]

Now, with the simple table

[TABLE]

and

[TABLE]

one gets the first formula of lemma 3.

The second formula of lemma 3 is obtained by considering the decomposition

[TABLE]

Then, using the table

[TABLE]

one gets a function

[TABLE]

which satisfies $\forall(k,k^{\prime})\in K\times K$

[TABLE]

This is the same situation as the previous one : by computing ${\gamma}^{-1}F_{2}$ and choosing the right coefficients, one gets the second formula of lemma 3. $\hfill\Box$

Conclusion : Thus, we could write the function $F$ for $n=1$ and $1+\mu_{1}({\pi}_{F})^{2}\not=0$ or $1+\mu_{2}({\pi}_{F})^{2}\not=0$. The latter condition is precisely condition 1 of theorem 5, which is now proved. Of course, it would be interesting to remove this condition and then to find $F$ for any $n$.