Some results on reducibility of parabolic induction for classical groups

TL;DR

This paper investigates conditions under which the parabolic induction of a representation of a classical group is reducible, providing criteria based on the supercuspidal support of the inducing representation.

Contribution

It establishes a reducibility criterion for parabolic induction in classical groups based on the supercuspidal support, extending previous understanding of representation theory.

Findings

Reducibility of $ ho\rtimes\sigma$ implies reducibility of $\pi\rtimes\sigma$ when $\rho$ is in the support of $\pi$.

Provides irreducibility criteria in special cases where the support condition is not met.

Enhances understanding of the structure of induced representations in classical groups.

Abstract

Given a (complex, smooth) irreducible representation $\pi$ of the general linear group over a non-archimedean local field and an irreducible supercuspidal representation $\sigma$ of a classical group, we show that the (normalized) parabolic induction $\pi\rtimes\sigma$ is reducible if there exists $\rho$ in the supercuspidal support of $\pi$ such that $\rho\rtimes\sigma$ is reducible. In special cases we also give irreducibility criteria for $\pi\rtimes\sigma$ when the above condition is not satisfied.

Equations

$$S_{\sigma }=\{\rho \in {\mathrm{Cusp}}^{\mathrm{GL}}:\rho ⋊\sigma \text{ is reducible}\}.$$

$$\pi$$

$$\mathrm{supp}\pi \subset \rho [\mathbb{Z}]\text{ where }\rho \in S_{\sigma }\text{ and }\rho \in \{\tilde{\rho },\tilde{\rho }[\pm 1],\tilde{\rho }[\pm 2]\}.$$

$$\text{if π is SI then so is any non-zero subobject σ of π, and soc(σ)=soc(π)}.$$

$$\text{π is simple if and only if π or π∨ is SI and soc(π)∨≃soc(π∨)}.$$

$$\mathcal{C}(G)={\oplus }_{\mathfrak{d}\in {\mathcal{D}}_{\mathrm{cusp}}(G)}\mathcal{C}(G{)}_{\mathfrak{d}}$$

$$\mathcal{R}(G)={\oplus }_{\mathfrak{d}\in {\mathcal{D}}_{\mathrm{cusp}}(G)}\mathcal{R}(G{)}_{\mathfrak{d}}\text{where }\mathcal{R}(G{)}_{\mathfrak{d}}=\mathcal{R}(\mathcal{C}(G{)}_{\mathfrak{d}}).$$

$${\mathcal{C}}^{\mathrm{GL}}={\oplus }_{n\ge 0}\mathcal{C}({\mathrm{GL}}_n),\mathrm{Irr}\mathrm{GL}=\mathrm{Irr}{\mathcal{C}}^{\mathrm{GL}}={\bigsqcup }_{n\ge 0}\mathrm{Irr}{\mathrm{GL}}_n,{\mathcal{R}}^{\mathrm{GL}}=\mathcal{R}({\mathcal{C}}^{\mathrm{GL}})={\oplus }_{n\ge 0}\mathcal{R}({\mathrm{GL}}_n).$$

$$\mathfrak{c}({\pi }_1\times {\pi }_2)=(\mathfrak{c}({\pi }_1)\mathrm{deg}{\pi }_1+\mathfrak{c}({\pi }_2)\mathrm{deg}{\pi }_2)/(\mathrm{deg}{\pi }_1+\mathrm{deg}{\pi }_2).$$

$$\times :\mathcal{C}({\mathrm{GL}}_n)\times \mathcal{C}({\mathrm{GL}}_m)\to \mathcal{C}({\mathrm{GL}}_{n+m})$$

$$\times :{\mathcal{C}}^{\mathrm{GL}}\times {\mathcal{C}}^{\mathrm{GL}}\to {\mathcal{C}}^{\mathrm{GL}}.$$

$$J_{\max }^{\mathrm{GL}}:{\mathcal{C}}^{\mathrm{GL}}\to {\mathcal{C}}^{\mathrm{GL}}\otimes {\mathcal{C}}^{\mathrm{GL}}={\oplus }_{n,m}\mathcal{C}({\mathrm{GL}}_n\times {\mathrm{GL}}_m),$$

$$J_{(n,m)}^{\mathrm{GL}}:\mathcal{C}({\mathrm{GL}}_{n+m})\to \mathcal{C}({\mathrm{GL}}_n)\otimes \mathcal{C}({\mathrm{GL}}_m)=\mathcal{C}({\mathrm{GL}}_n\times {\mathrm{GL}}_m),n,m\ge 0$$

$$m^{\ast }:{\mathcal{R}}^{\mathrm{GL}}\to {\mathcal{R}}^{\mathrm{GL}}\otimes {\mathcal{R}}^{\mathrm{GL}}.$$

$$J_{\mathrm{submax}}^{\mathrm{GL}}=(J_{\max }^{\mathrm{GL}}\otimes \mathrm{Id})\circ J_{\max }^{\mathrm{GL}}=(\mathrm{Id}\otimes J_{\max }^{\mathrm{GL}})\circ J_{\max }^{\mathrm{GL}}:{\mathcal{C}}^{\mathrm{GL}}\to {\mathcal{C}}^{\mathrm{GL}}\otimes {\mathcal{C}}^{\mathrm{GL}}\otimes {\mathcal{C}}^{\mathrm{GL}}$$

$$m_{\mathrm{it}}^{\ast }=(m^{\ast }\otimes \mathrm{Id})\circ m^{\ast }=(\mathrm{Id}\otimes m^{\ast })\circ m^{\ast }:{\mathcal{R}}^{\mathrm{GL}}\to {\mathcal{R}}^{\mathrm{GL}}\otimes {\mathcal{R}}^{\mathrm{GL}}\otimes {\mathcal{R}}^{\mathrm{GL}}.$$

$${\mathcal{C}}^{\mathrm{GL}}={\oplus }_{\mathfrak{d}\in \mathbb{N}({\mathrm{Cusp}}^{\mathrm{GL}})}{\mathcal{C}}_{\mathfrak{d}}^{\mathrm{GL}}$$

$$p_X:{\mathcal{C}}^{\mathrm{GL}}\to {\mathcal{C}}_X^{\mathrm{GL}}:={\oplus }_{\mathfrak{d}\in X}{\mathcal{C}}_{\mathfrak{d}}^{\mathrm{GL}}.$$

$${\mathcal{C}}^{\mathrm{GL}}\otimes {\mathcal{C}}^{\mathrm{GL}}\stackrel{p_X\otimes \mathrm{Id}}{\to }{\mathcal{C}}_X^{\mathrm{GL}}\otimes {\mathcal{C}}^{\mathrm{GL}}.$$

$${\mathcal{R}}^{\mathrm{GL}}\to {\mathcal{R}}_X^{\mathrm{GL}}\otimes {\mathcal{R}}^{\mathrm{GL}},{\mathcal{R}}^{\mathrm{GL}}\to {\mathcal{R}}^{\mathrm{GL}}\otimes {\mathcal{R}}_X^{\mathrm{GL}}.$$

$${\mathcal{C}}^{\mathrm{GL}}\stackrel{J_{\mathrm{submax}}^{\mathrm{GL}}}{\to }{\mathcal{C}}^{\mathrm{GL}}\otimes {\mathcal{C}}^{\mathrm{GL}}\otimes {\mathcal{C}}^{\mathrm{GL}}\stackrel{p_X\otimes \mathrm{Id}\otimes p_Y}{\to }{\mathcal{C}}_X^{\mathrm{GL}}\otimes {\mathcal{C}}^{\mathrm{GL}}\otimes {\mathcal{C}}_Y^{\mathrm{GL}}$$

$$m_{X;\ast ;Y}^{\ast }:{\mathcal{R}}^{\mathrm{GL}}\to {\mathcal{R}}_X^{\mathrm{GL}}\otimes {\mathcal{R}}^{\mathrm{GL}}\otimes {\mathcal{R}}_Y^{\mathrm{GL}}$$

$${\mathfrak{S}}^l(\pi )=\{\rho \in {\mathrm{Cusp}}^{\mathrm{GL}}:J_{\mathbb{N}(\rho );\ast }^{\mathrm{GL}}(\pi )\ne 1\otimes \pi \}=\{\rho \in {\mathrm{Cusp}}^{\mathrm{GL}}:J_{\{\rho \};\ast }^{\mathrm{GL}}(\pi )\ne 0\}.$$

$${\mathfrak{S}}^l({\pi }_1\times {\pi }_2)={\mathfrak{S}}^l({\pi }_1)\cup {\mathfrak{S}}^l({\pi }_2)$$

$$\text{if π↪π1×π2 with π1,π2∈IrrGL then }{\mathfrak{S}}^l(\pi )\supset {\mathfrak{S}}^l({\pi }_1).$$

$${\mathfrak{S}}^r(\pi )=\{\rho \in {\mathrm{Cusp}}^{\mathrm{GL}}:J_{\ast ;\mathbb{N}(\rho )}^{\mathrm{GL}}(\pi )\ne \pi \otimes 1\}=\{\rho \in {\mathrm{Cusp}}^{\mathrm{GL}}:J_{\ast ;\{\rho \}}^{\mathrm{GL}}(\pi )\ne 0\}.$$

$${\mathfrak{S}}^l(\pi {)}^{\vee }={\mathfrak{S}}^r({\pi }^{\vee }).$$

$$m_{\mathbb{N}(A);\ast }^{\ast }(\pi )={\pi }_A^l\otimes L_A(\pi )+\sum _i{\alpha }_i\otimes {\beta }_i$$

$$m^{\ast }({\pi }_A^l\times L_A(\pi ))={\pi }_A^l\otimes L_A(\pi )+\sum _i{\alpha }_i\otimes {\beta }_i$$

$$m_{\ast ;\mathbb{N}(A)}^{\ast }(\pi )=R_A(\pi )\otimes {\pi }_A^r+\sum _i{\alpha }_i\otimes {\beta }_i,{\alpha }_i,{\beta }_i\in \mathrm{Irr}\mathrm{GL}$$

$$D_{A;B}(\pi )=R_B(L_A(\pi ))=L_A(R_B(\pi ))$$

Full text

1. 1 Introduction
2. 2 The general linear group
3. 3 Classical groups
4. 4 A reducibility result
5. 5 Case of two segments
6. 6 An irreducibility criterion

1. Introduction

In this paper we study reducibility of parabolic induction of representations of classical groups over $p$-adic fields. (All representations considered are complex, smooth and of finite length, hence admissible.) Of course, the problem goes back to the early days of representation theory. It is important not only in its own right but also for studying other classes of representations such as discrete series [MT02] and unitary representations [Tad93].

We fix a classical group $G$ over a non-archimedean local field $F$ of characteristic [math] and a supercuspidal irreducible representation $\sigma$ of $G(F)$. In the unitary case let $E/F$ be the quadratic extension over which $G$ splits and let $\varrho$ be the Galois involution. In the other cases let $E=F$ and $\varrho=\operatorname{Id}$.

Denote by $\tilde{}$ the composition of $\varrho$ with the involution $g\mapsto\tensor[^{t}]g{{}^{-1}}$ of $\operatorname{GL}_{n}(E)$. (We continue to denote by $\tilde{}$ the induced involution on the various objects pertaining to $\operatorname{GL}_{n}(E)$.) Denote by $\operatorname{Irr}\operatorname{GL}$ the set of irreducible representations (up to equivalence) of any of the groups $\operatorname{GL}_{n}(E)$, $n\geq 0$. Let $\operatorname{Cusp}^{\operatorname{GL}}\subset\operatorname{Irr}\operatorname{GL}$ be the set of (not necessarily unitarizable) irreducible supercuspidal representations (up to equivalence) of any of the groups $\operatorname{GL}_{n}(E)$, $n\geq 1$. For any $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$ and $x\in\mathbb{R}$ let $\rho[x]\in\operatorname{Cusp}^{\operatorname{GL}}$ be the twist of $\rho$ by the character $\left|{\det\cdot}\right|^{x}$. Let $\rho[\mathbb{Z}]=\{\rho[m]:m\in\mathbb{Z}\}$ and $\rho[\mathbb{R}]=\{\rho[x]:x\in\mathbb{R}\}$. For any $\pi\in\operatorname{Irr}\operatorname{GL}$ we denote by $\operatorname{supp}\pi\subset\operatorname{Cusp}^{\operatorname{GL}}$ the supercuspidal support of $\pi$.

A major role is played by the set

[TABLE]

(As usual, we denote by $\rtimes$ (resp., $\times$) normalized parabolic induction for classical groups (resp., the general linear group).) By standard results, $\tilde{S}_{\sigma}=S_{\sigma}$ and for any $\rho\in S_{\sigma}$ we have $\rho[\mathbb{R}]\cap S_{\sigma}=\{\rho,\tilde{\rho}\}$. A deeper result due to Mœglin states that in fact $\tilde{\rho}\in\rho[\mathbb{Z}]$.

Let $\pi\in\operatorname{Irr}\operatorname{GL}$. The question of reducibility of $\pi\rtimes\sigma$ naturally divides into two cases, according to whether or not $\operatorname{supp}\pi$ intersects $S_{\sigma}$. In the first case there is a clear-cut answer, which is our first main result.

Theorem 1.1**.**

Let $\pi\in\operatorname{Irr}\operatorname{GL}$ be such that $\operatorname{supp}\pi\cap S_{\sigma}\neq\emptyset$. Then $\pi\rtimes\sigma$ is reducible.

In the case where $\operatorname{supp}\pi\cap S_{\sigma}=\emptyset$ things are more complicated and we do not have a general simple characterization of the irreducibility of $\pi\rtimes\sigma$. We need to impose a condition on either $\pi$ or $\sigma$. Let us describe our partial results in this direction. We first remark that it is easy to reduce the question (assuming irreducibility of parabolic induction for the general linear group is understood) to the case where $\operatorname{supp}\pi\subset\rho[\mathbb{Z}]$ for some $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$ such that $\tilde{\rho}\in\rho[\mathbb{Z}]$. Assume further that $\rho\in S_{\sigma}$. (We remark that the set $\cup_{\rho\in S_{\sigma}}\rho[\mathbb{Z}]$ is independent of $\sigma$, and in fact depends only on the type of $G$.)

For any $\pi\in\operatorname{Irr}\operatorname{GL}$ we construct an auxiliary representation $\pi_{+}\in\operatorname{Irr}\operatorname{GL}$. (See Definition 3.12.) Namely, if $\pi$ corresponds to a multisegment $\mathfrak{m}$ under the Zelevinsky classification, then $\pi_{+}$ corresponds to the submultisegment of $\mathfrak{m}$ consisting of the segments with positive exponents.111By applying the Zelevinsky–Aubert involution, we can work with the Langlands classification instead.

We also recall the notion of ladder representations [LM14], which are generalization of the Speh representations.

Our second main result is the following.

Theorem 1.2**.**

Let $\pi\in\operatorname{Irr}\operatorname{GL}$ be such that $\operatorname{supp}\pi\cap S_{\sigma}=\emptyset$. Assume that

[TABLE]

or that

[TABLE]

Then $\pi\rtimes\sigma$ is irreducible if and only if $\pi_{+}\times\tilde{\pi}_{+}$ is irreducible (where $\tilde{\pi}_{+}=(\tilde{\pi})_{+}$).

As a supplement to the theorem, we remark that in the case where $\pi_{1},\pi_{2}\in\operatorname{Irr}\operatorname{GL}$ are two ladder representations, the irreducibility of $\pi_{1}\times\pi_{2}$ can be characterized explicitly [LM16, Proposition 6.20]. We also remark that by the work of Shahidi [Sha90], the condition $\rho\in\{\tilde{\rho},\tilde{\rho}[\pm 1],\tilde{\rho}[\pm 2]\}$ is satisfied for any $\rho\in S_{\sigma}$ if $G_{n}$ is quasi-split and $\sigma$ admits a Whittaker model with respect to a non-degenerate character of a maximal unipotent subgroup of $G_{n}$. (Without restriction on $G_{n}$ or $\sigma$, the condition $\rho\in\{\tilde{\rho},\tilde{\rho}[\pm 1]\}$ is satisfied for all but finitely many $\rho\in S_{\sigma}$.)

On the other hand, if $\rho=\tilde{\rho}[m]\in S_{\sigma}$ with $m>2$ then it is easy to construct $\pi\in\operatorname{Irr}\operatorname{GL}$ such that $\pi=\pi_{+}$, $\operatorname{supp}\pi\subset\rho[\mathbb{Z}]\setminus S_{\sigma}$ and $\pi\rtimes\sigma$ is reducible. Thus, the additional conditions on $\pi$ or $\sigma$ in Theorem 1.2 are not superfluous. We do not know whether the irreducibility of $\pi_{+}\times\tilde{\pi}_{+}$ is a necessary condition for the irreducibility of $\pi\rtimes\sigma$ in general. At any rate we expect the following to hold.

Conjecture 1.3**.**

Assume that $\pi\in\operatorname{Irr}\operatorname{GL}$ is such that $\operatorname{supp}\pi\cap S_{\sigma}=\emptyset$. Then the irreducibility of $\pi\rtimes\sigma$ depends only on $\pi$ and not on $\sigma$.

Granted this conjecture, an interesting natural follow-up question would be to characterize those $\pi$’s such that $\pi\rtimes\sigma$ is irreducible for any supercuspidal $\sigma$ such that $\operatorname{supp}\pi\cap S_{\sigma}=\emptyset$. This is already non-trivial if $\pi=\pi_{+}$.

The structure of the paper is as follows. In §2 we introduce the notation and recollect some basic results pertaining to the representation theory of the general linear group, in the spirit of Bernstein–Zelevinsky. In §3 we do the same for classical groups. Theorem 1.1 is proved in §4 by reducing it to a simple case which can be treated by hand. In §5 we treat the case of two segments. Although it is not logically necessary for the general case, we include it in order to illustrate the idea. Finally in §6 we prove Theorem 1.2 as well as some other basic facts about irreducibility.

Our results generalize known results such as some of the results in [Gus81, Jan93b, Jan93a, Jan96a, Jan96b, Jan98, BJ03, Tad98, Tad09]. However, our approach is somewhat different.

A natural next step would be to extend our results beyond the case where $\sigma$ is supercuspidal.

Part of this work was done while the authors were hosted by the Mathematisches Forschungsinstitut Oberwolfach for a “Research in Pairs” project. We are very grateful to the MFO for providing ideal working conditions. Finally, it is a pleasure to thank David Goldberg, Max Gurevich, Marcela Hanzer, Chris Jantzen, Alberto Mínguez, Colette Mœglin and Goran Muic for useful correspondence.

1.1. General notation

Throughout we fix a non-archimedean local field $F$ with normalized absolute value $\left|{\cdot}\right|$.

From section 3 onward we assume that $F$ is of characteristic [math]. (Hopefully this assumption can be lifted.)

For any set $X$, $\mathbb{Z}({X})$ (resp., $\mathbb{N}({X})$) denotes the free abelian group (resp., monoid) generated by $X$. We denote the resulting order on $\mathbb{Z}({X})$ by $\leq$. We can think of an element of $\mathbb{N}({X})$ as a finite multiset consisting of elements of $X$. If $a=x_{1}+\dots+x_{k}\in\mathbb{N}({X})$ with $x_{1},\dots,x_{k}\in X$ we call the underlying set $\{x_{1},\dots,x_{k}\}$ the support of $a$.

All abelian categories considered in this paper are essentially small, $\mathbb{C}$-linear and locally finite. (See [EGNO15, Ch. 1] for basic terminology and more details.) For any such category $\mathcal{C}$ let $\operatorname{Irr}\mathcal{C}$ be the set of isomorphism classes of simple objects of $\mathcal{C}$ and $\mathcal{R}(\mathcal{C})$ the Grothendieck group of $\mathcal{C}$, isomorphic to $\mathbb{Z}({\operatorname{Irr}\mathcal{C}})$, and thus an ordered group. For any object $\pi$ of $\mathcal{C}$ we denote by $\operatorname{JH}(\pi)$ the Jordan–Hölder sequence of $\pi$ considered as an element of $\mathbb{N}({\operatorname{Irr}\mathcal{C}})$ and by $\ell(\pi)$ the length of $\pi$ (i.e., the number of elements of $\operatorname{JH}(\pi)$, counted with multiplicities). For simplicity, we often write $\pi_{1}\leq\pi_{2}$ if $\operatorname{JH}(\pi_{1})\leq\operatorname{JH}(\pi_{2})$. Denote by $\operatorname{soc}(\pi)$ (resp., $\operatorname{cos}(\pi)$) the socle (resp., cosocle) of an object $\pi$ of $\mathcal{C}$, i.e., the largest semisimple subobject (resp., quotient) of $\pi$. We say that $\pi$ is SI (socle irreducible) (resp., CSI; cosocle irreducible) if $\operatorname{soc}(\pi)$ (resp., $\operatorname{cos}(\pi)$) is simple and occurs with multiplicity one in $\operatorname{JH}(\sigma)$. Obviously,

[TABLE]

Clearly, $\pi$ is simple if and only if it is SI and $\operatorname{soc}(\pi)\simeq\operatorname{cos}(\pi)$. If $\mathcal{C}$ admits a duality functor ∨ (as will be the case throughout the paper) then $\operatorname{cos}(\pi)^{\vee}=\operatorname{soc}(\pi^{\vee})$. Thus,

[TABLE]

Now let $G$ be a connected reductive group over $F$.222Later on we will also consider orthogonal groups, which are not connected We denote by $\mathcal{C}(G)$ the category of admissible, finitely generated representations of $G(F)$ over $\mathbb{C}$. We refer to [Ren10] for standard facts about representation theory of $G(F)$, some of which we will freely use below. We denote the contragredient duality functor by ∨. For simplicity we write $\operatorname{Irr}G=\operatorname{Irr}\mathcal{C}(G)$ and $\mathcal{R}(G)=\mathcal{R}(\mathcal{C}(G))$. In particular, for the trivial group, $\operatorname{Irr}1$ consists of a single element which we denote by $\mathbf{1}$. Note that if $G_{1}$ and $G_{2}$ are reductive groups over $F$ then $\mathcal{C}(G_{1}\times G_{2})$ is the tensor product of $\mathcal{C}(G_{1})\otimes\mathcal{C}(G_{2})$ in the sense of [Del90, §5]. (This follows from [ibid., Proposition 5.3] and the fact that the tensor product of categories commutes with inductive limits [ibid., p. 143].)

Denote by $\operatorname{Cusp}G\subset\operatorname{Irr}G$ the set of irreducible supercuspidal representations of $G(F)$ (up to equivalence). Let $\mathcal{D}_{\operatorname{cusp}}(G)$ be the set of cuspidal data of $G$, i.e., $G(F)$-conjugacy classes of pairs $(M,\sigma)$ consisting of a Levi subgroup $M$ of $G$ defined over $F$ and $\sigma\in\operatorname{Cusp}M$. For any $\mathfrak{d}\in\mathcal{D}_{\operatorname{cusp}}(G)$ let $\mathcal{C}(G)_{\mathfrak{d}}$ be the Serre subcategory333See [EGNO15, Definition 4.14.1] of $\mathcal{C}(G)$ generated by the (normalized) parabolic induction $\operatorname{Ind}_{P(F)}^{G(F)}\sigma$ where $(M,\sigma)\in\mathfrak{d}$ and $P$ is a parabolic subgroup of $G$ defined over $F$ with Levi subgroup $M$. (The definition is independent of the choice of $(M,\sigma)$ and $P$. We also recall that parabolic induction commutes with ∨.) The category $\mathcal{C}(G)$ splits as

[TABLE]

where for each $\mathfrak{d}$, $\operatorname{Irr}\mathcal{C}(G)_{\mathfrak{d}}$ is finite. Accordingly,

[TABLE]

We denote by $p_{\mathfrak{d}}$ the projection $\mathcal{C}(G)\rightarrow\mathcal{C}(G)_{\mathfrak{d}}$. More generally, if $A\subset\mathcal{D}_{\operatorname{cusp}}(G)$ then we denote the projection $\mathcal{C}(G)\rightarrow\mathcal{C}(G)_{A}:=\oplus_{\mathfrak{d}\in A}\mathcal{C}(G)_{\mathfrak{d}}$ by $p_{A}$. We write $\bar{p}_{A}:\mathcal{R}(G)\rightarrow\mathcal{R}(G)_{A}=\oplus_{\mathfrak{d}\in A}\mathcal{R}(G)_{\mathfrak{d}}$ for the corresponding projection in the Grotehndieck group.

We denote by t the Zelevinsky–Aubert involution on $\mathcal{R}(G)$. It respects the decomposition (1.3) and induces an involution, also denoted by t on $\operatorname{Irr}G$ [Aub95, Aub96] (cf. [Jan05, §6] for the even orthogonal case).444See [BBK17] for a recent approach which highlights the functorial properties of t

2. The general linear group

In this section we recall some facts about representation theory of the general linear group. Most of the results are standard and go back to the seminal work of Bernstein–Zelevinsky.

2.1. Notation

Let

[TABLE]

If $\pi\in\mathcal{C}(\operatorname{GL}_{n})$, we write $\deg\pi=n$, and for any $x\in\mathbb{R}$ we denote by $\pi[x]$ the representation obtained from $\pi$ by twisting by the character $\left|{\det}\right|^{x}$. In particular set $\overset{\rightarrow}{\pi}=\pi[1]$ and $\overset{\leftarrow}{\pi}=\pi[-1]$. If $\pi\in\operatorname{Irr}(\operatorname{GL}_{n})$, $n>0$ (or more generally, if $\pi$ has a central character $\omega_{\pi}$) let $\mathfrak{c}(\pi)$ be the real number such that $\pi[-\mathfrak{c}(\pi)]$ has a unitary central character. (That is, the character $\omega_{\pi}\left|{\cdot}\right|^{-\deg\pi\cdot\mathfrak{c}(\pi)}$ is unitary.) We also set $\mathfrak{c}(\mathbf{1})=0$. Note that $\mathfrak{c}(\pi^{\vee})=-\mathfrak{c}(\pi)$ and

[TABLE]

For any $n,m\geq 0$ let

[TABLE]

be the bilinear biexact bifunctor of (normalized) parabolic induction with respect to the parabolic subgroup of block upper triangular matrices. We also denote by $\times$ the resulting bilinear biexact bifunctor

[TABLE]

Together with the unit element $\mathbf{1}$ and the isomorphism of induction by stages, this endows $\mathcal{C}^{\operatorname{GL}}$ with the structure of a monoidal category (and hence, a ring category over $\mathbb{C}$ in the sense of [EGNO15, Definition 4.2.3]). This structure also induces a $\mathbb{Z}_{\geq 0}$-graded ring structure on $\mathcal{R}^{\operatorname{GL}}$. Although $\times$ is not symmetric, $\mathcal{R}^{\operatorname{GL}}$ is a commutative ring.

Let

[TABLE]

be the left-adjoint functors to $\times$. Thus, $J^{\operatorname{GL}}_{\max}=\oplus_{n}(\oplus_{n_{1}+n_{2}=n}J^{\operatorname{GL}}_{(n_{1},n_{2})})$ and $J^{\operatorname{GL}}_{(n,m)}$ is the (normalized) Jacquet functor. We denote by $\operatorname{m}^{*}$ the resulting ring homomorphism555The fact that $\operatorname{m}^{*}$ is a homomorphism follows from the geometric lemma of Bernstein–Zelevinsky.

[TABLE]

We also write

[TABLE]

and

[TABLE]

Let $\operatorname{Cusp}^{\operatorname{GL}}=\sqcup_{n>0}\operatorname{Cusp}\operatorname{GL}_{n}$. (Note that we exclude $\mathbf{1}$ from $\operatorname{Cusp}^{\operatorname{GL}}$.) We identify $\sqcup_{n\geq 0}\mathcal{D}_{\operatorname{cusp}}(\operatorname{GL}_{n})$ with $\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$. Thus, we get a decomposition

[TABLE]

and for any subset $X\subset\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$ a projection

[TABLE]

We write $\mathcal{R}^{\operatorname{GL}}_{X}=\mathcal{R}(\mathcal{C}^{\operatorname{GL}}_{X})=\oplus_{\mathfrak{d}\in X}\mathcal{R}(\mathcal{C}^{\operatorname{GL}}_{\mathfrak{d}})$ and $\bar{p}_{X}:\mathcal{R}^{\operatorname{GL}}\rightarrow\mathcal{R}^{\operatorname{GL}}_{X}$. For any $\mathfrak{d}\in\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$ we have $\mathcal{C}^{\operatorname{GL}}_{\mathfrak{d}}\subset\mathcal{C}(\operatorname{GL}_{\deg\mathfrak{d}})$ where we extend $\deg$ to $\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$ by linearity. For any $\mathfrak{d}_{1},\mathfrak{d}_{2}\in\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$ we have $\mathcal{C}^{\operatorname{GL}}_{\mathfrak{d}_{1}}\times\mathcal{C}^{\operatorname{GL}}_{\mathfrak{d}_{2}}\subset\mathcal{C}^{\operatorname{GL}}_{\mathfrak{d}_{1}+\mathfrak{d}_{2}}$. If $0\neq\pi\in\mathcal{C}^{\operatorname{GL}}_{\mathfrak{d}}$ we write $\mathfrak{d}_{\pi}=\mathfrak{d}\in\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$ and denote by $\operatorname{supp}\pi\subset\operatorname{Cusp}^{\operatorname{GL}}$ the support of $\mathfrak{d}_{\pi}$.

For any $X\subset\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$ we write $J^{\operatorname{GL}}_{X;*}$ for the composition of $J^{\operatorname{GL}}_{\max}$ with

[TABLE]

Analogously for $J^{\operatorname{GL}}_{*;X}$. We also write $\operatorname{m}^{*}_{X;*}$ and $\operatorname{m}^{*}_{*;X}$ for the corresponding homomorphisms

[TABLE]

Similarly, given $X,Y\subset\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$ we write $J^{\operatorname{GL}}_{X;*;Y}$ for the composition

[TABLE]

and

[TABLE]

for the corresponding map of Grothendieck groups.

2.2. Derivatives

666This notion should not be confused with Zelevinsky’s notion of derivative.

Definition 2.1**.**

Let $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$. We say that $\pi\in\operatorname{Irr}\operatorname{GL}$ is left $\rho$-reduced if the following equivalent conditions are satisfied.

1. (1)

There does not exist $\pi^{\prime}\in\operatorname{Irr}\operatorname{GL}$ such that $\pi\hookrightarrow\rho\times\pi^{\prime}$. 2. (2)

$J^{\operatorname{GL}}_{\{\rho\};*}(\pi)=0$, i.e., there does not exist $\pi^{\prime}\in\operatorname{Irr}\operatorname{GL}$ such that $\rho\otimes\pi^{\prime}\leq J^{\operatorname{GL}}_{\max}(\pi)$. 3. (3)

$J^{\operatorname{GL}}_{\mathbb{N}({\{\rho\}});*}(\pi)=\mathbf{1}\otimes\pi$, i.e., there do not exist $\pi_{1},\pi_{2}\in\operatorname{Irr}\operatorname{GL}$ such that $\pi_{1}\otimes\pi_{2}\leq J^{\operatorname{GL}}_{\max}(\pi)$ and $\operatorname{supp}\pi_{1}=\{\rho\}$.

Given $A\subset\operatorname{Cusp}^{\operatorname{GL}}$ we say that $\pi\in\operatorname{Irr}\operatorname{GL}$ is left $A$-reduced if $\pi$ is left $\rho$-reduced for any $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$. Equivalently, $J^{\operatorname{GL}}_{\mathbb{N}({A});*}(\pi)=\mathbf{1}\otimes\pi$. Similarly for right $\rho$-reduced and right $A$-reduced representations.

We note that $\pi$ is left $\rho$-reduced if and only if $\pi^{\vee}$ is right $\rho^{\vee}$-reduced.

For any $\pi\in\mathcal{C}(\operatorname{GL})$ (not necessarily irreducible) define

[TABLE]

This is a finite subset of $\operatorname{Cusp}^{\operatorname{GL}}$ which is nonempty if $\pi\neq 0$ unless $\deg\pi=0$. It follows from the geometric lemma that

[TABLE]

and by Frobenius reciprocity

[TABLE]

Similarly, define

[TABLE]

Note that

[TABLE]

Lemma 2.2**.**

([Jan07]) For any $\pi\in\operatorname{Irr}\operatorname{GL}$ and $A\subset\operatorname{Cusp}^{\operatorname{GL}}$ there exist $\pi_{A}^{\operatorname{l}}\in\operatorname{Irr}\operatorname{GL}$ and $L_{A}(\pi)\in\operatorname{Irr}\operatorname{GL}$ satisfying the following conditions:

1. (1)

$\pi\hookrightarrow\pi_{A}^{\operatorname{l}}\times L_{A}(\pi)$, i.e., $J^{\operatorname{GL}}_{\max}(\pi)\twoheadrightarrow\pi_{A}^{\operatorname{l}}\otimes L_{A}(\pi)$. 2. (2)

$\operatorname{supp}\pi_{A}^{\operatorname{l}}\subset A$. 3. (3)

$L_{A}(\pi)$* is left $A$-reduced.*

Moreover, $\pi_{A}^{\operatorname{l}}$ and $L_{A}(\pi)$ are uniquely determined by $\pi$ and we have

[TABLE]

where $\deg\beta_{i}>\deg L_{A}(\pi)$ for all $i$.

Proof.

We recall the simple argument since we will use it repeatedly. For the existence part, we take $\pi_{A}^{\operatorname{l}}$ supported in $A$ of maximal degree with respect to the property that $\pi\hookrightarrow\pi_{A}^{\operatorname{l}}\times\pi^{\prime}$ for some $\pi^{\prime}$. The last part (and the uniqueness) follow from the fact that $J^{\operatorname{GL}}_{\max}(\pi)\leq J^{\operatorname{GL}}_{\max}(\pi_{A}^{\operatorname{l}}\times L_{A}(\pi))$ and that by the geometric lemma we have

[TABLE]

where $\deg\beta_{i}>\deg L_{A}(\pi)$ for all $i$ such that $\operatorname{supp}\alpha_{i}\subset A$. ∎

We call $L_{A}(\pi)$ the left partial derivative of $\pi$ with respect to $A$. Analogously, there exist unique $\pi_{A}^{\operatorname{r}},R_{A}(\pi)\in\operatorname{Irr}\operatorname{GL}$ such that $\pi\hookrightarrow R_{A}(\pi)\times\pi_{A}^{\operatorname{r}}$, $\operatorname{supp}\pi_{A}^{\operatorname{r}}\subset A$ and $R_{A}(\pi)$ is right $A$-reduced. We have

[TABLE]

where $\deg\alpha_{i}>\deg R_{A}(\pi)$ for all $i$.

Lemma 2.3**.**

For any $\pi\in\operatorname{Irr}\operatorname{GL}$ and $A,B\subset\operatorname{Cusp}^{\operatorname{GL}}$ there exist $\pi_{1},\pi_{2},\pi_{3}\in\operatorname{Irr}\operatorname{GL}$ such that

1. (1)

$\pi\hookrightarrow\pi_{1}\times\pi_{2}\times\pi_{3}$, i.e., $J^{\operatorname{GL}}_{\operatorname{submax}}(\pi)\twoheadrightarrow\pi_{1}\otimes\pi_{2}\otimes\pi_{3}$. 2. (2)

$\operatorname{supp}\pi_{1}\subset A$; $\operatorname{supp}\pi_{3}\subset B$. 3. (3)

$\pi_{2}$* is left $A$-reduced and right $B$-reduced.*

Moreover, $\pi_{2}$ is uniquely determined and is characterized by the following conditions:

1. (A)

There exist $\alpha,\gamma\in\operatorname{Irr}\operatorname{GL}$ such that $\alpha\otimes\pi_{2}\otimes\gamma\leq\operatorname{m}^{*}_{\mathbb{N}({A});*;\mathbb{N}({B})}(\pi)$. 2. (B)

If $\alpha\otimes\beta\otimes\gamma\leq\operatorname{m}^{*}_{\mathbb{N}({A});*;\mathbb{N}({B})}(\pi)$ with $\alpha,\beta,\gamma\in\operatorname{Irr}\operatorname{GL}$ then either $\beta=\pi_{2}$ or $\deg\beta>\deg\pi_{2}$.

Proof.

For the existence we can take $\pi_{1}=\pi_{A}^{\operatorname{l}}$, $\pi_{2}=R_{B}(L_{A}(\pi))$, $\pi_{3}=(L_{A}(\pi))_{B}^{\operatorname{r}}$. The last statement follows once again from the geometric lemma and the fact that $J^{\operatorname{GL}}_{\operatorname{submax}}(\pi)\leq J^{\operatorname{GL}}_{\operatorname{submax}}(\pi_{1}\times\pi_{2}\times\pi_{3})$. ∎

We will write $D_{A;B}(\pi)=\pi_{2}$. It is clear that

[TABLE]

and

[TABLE]

We also remark that

[TABLE]

Note that $\pi_{1}$ and $\pi_{3}$ (or even their degrees) are not necessarily uniquely determined by $\pi$ unless $A$ and $B$ are disjoint in which case $\pi_{1}=\pi_{A}^{\operatorname{l}}$ and $\pi_{3}=\pi_{B}^{\operatorname{r}}$.

Remark 2.4*.*

Let $A$ be a subset of $\operatorname{Cusp}^{\operatorname{GL}}$. Denote by $\mathcal{C}^{\operatorname{GL}}_{A\operatorname{-red}}$ the Serre ring subcategory of $\mathcal{C}^{\operatorname{GL}}$ consisting of left $A$-reduced representations. Let $(\operatorname{Irr}\operatorname{GL})_{A\operatorname{-red}}=\operatorname{Irr}\mathcal{C}^{\operatorname{GL}}_{A\operatorname{-red}}\subset\operatorname{Irr}\operatorname{GL}$ and $\mathcal{R}^{\operatorname{GL}}_{A\operatorname{-red}}=\mathcal{R}(\mathcal{C}^{\operatorname{GL}}_{A\operatorname{-red}})\subset\mathcal{R}^{\operatorname{GL}}$. Assume that $\overset{\rightarrow}{A}=A$. Then there is no difference between left $A$-reduced and right $A$-reduced. Denote the complement of $A$ by $A^{c}$. The map $\times$ defines a bijection

[TABLE]

which gives rise to an isomorphism of rings

[TABLE]

In fact $\times$ induces an equivalence of ring categories

[TABLE]

2.3. Zelevinsky classification

([Zel80]) A segment is a non-empty finite subset $\Delta$ of $\operatorname{Cusp}^{\operatorname{GL}}$ of the form $\Delta=\{\rho_{1},\dots,\rho_{k}\}$ where $\rho_{i+1}=\overset{\rightarrow}{\rho}_{i}$, $i=1,\dots,k-1$. Denote the set of segments by $\mathcal{SEG}$. Given $\Delta=\{\rho_{1},\dots,\rho_{k}\}\in\mathcal{SEG}$ as before, the representations $\rho_{1}\times\dots\times\rho_{k}$ and $\rho_{k}\times\dots\times\rho_{1}$ are SI. We write

[TABLE]

For compatibility we also write $Z(\emptyset)=L(\emptyset)=\mathbf{1}$. If $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$ and $n\geq-1$ we write

[TABLE]

We have

[TABLE]

If $\Delta_{1},\Delta_{2}\in\mathcal{SEG}$ we write $\Delta_{1}\prec\Delta_{2}$ if $b(\Delta_{1})\notin\Delta_{2}$, $b(\overset{\leftarrow}{\Delta}_{2})\in\Delta_{1}$ and $e(\Delta_{2})\notin\Delta_{1}$. In this case $\operatorname{soc}(Z(\Delta_{1})\times Z(\Delta_{2}))=Z(\Delta_{1}^{\prime})\times Z(\Delta^{\prime}_{2})$ where $\Delta_{1}^{\prime}=\Delta_{1}\cup\Delta_{2}$, $\Delta_{2}^{\prime}=\Delta_{1}\cap\Delta_{2}$ (the latter is possibly empty). Note that

[TABLE]

If either $\Delta_{1}\prec\Delta_{2}$ or $\Delta_{2}\prec\Delta_{1}$ (we cannot have both) then we say that $\Delta_{1}$ and $\Delta_{2}$ are linked. The representation $Z(\Delta_{1})\times Z(\Delta_{2})$ is reducible if and only if $\Delta_{1}$ and $\Delta_{2}$ are linked, in which case it is of length two.

A multisegment is an element $\mathfrak{m}$ of $\mathbb{N}({\mathcal{SEG}})$. Thus, $\mathfrak{m}=\Delta_{1}+\dots+\Delta_{k}$ for some $\Delta_{1},\dots,\Delta_{k}\in\mathcal{SEG}$. We write

[TABLE]

We may enumerate the $\Delta_{i}$’s so that $\Delta_{i}\not\prec\Delta_{j}$ whenever $i<j$. In this case, the representation $\mathfrak{z}(\mathfrak{m})=Z(\Delta_{1})\times\dots\times Z(\Delta_{k})$ (resp., $\mathfrak{l}(\mathfrak{m})=L(\Delta_{1})\times\dots\times L(\Delta_{k})$) is SI (resp., CSI) and depends only on $\mathfrak{m}$. The maps

[TABLE]

are bijections between $\mathbb{N}({\mathcal{SEG}})$ and $\operatorname{Irr}$. For any $\mathfrak{m}\in\mathbb{N}({\mathcal{SEG}})$ we have

[TABLE]

Moreover, if $\mathfrak{m}_{1},\mathfrak{m}_{2}\in\mathbb{N}({\mathcal{SEG}})$ then $Z(\mathfrak{m}_{1}+\mathfrak{m}_{2})$ occurs with multiplicity one in $\operatorname{JH}(Z(\mathfrak{m}_{1})\times Z(\mathfrak{m}_{2}))$; similarly for $L(\mathfrak{m}_{1}+\mathfrak{m}_{2})$. We have

For simplicity, if $\mathfrak{m}$ is a multisegment we write $\mathfrak{S}^{l}(\mathfrak{m})=\mathfrak{S}^{l}(Z(\mathfrak{m}))$ and $\mathfrak{S}^{r}(\mathfrak{m})=\mathfrak{S}^{r}(Z(\mathfrak{m}))$. We have the following combinatorial description of $\mathfrak{S}^{l}(\mathfrak{m})$ and $\mathfrak{S}^{r}(\mathfrak{m})$.

Proposition 2.5**.**

([LM16, Theorem 5.11] which is based on [Jan07] and [Mín09]) Let $\mathfrak{m}=\Delta_{1}+\dots+\Delta_{k}$ be a multisegment. For any $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$ let

[TABLE]

Then $\rho\notin\mathfrak{S}^{l}(\mathfrak{m})$ if and only if there exists an injective map $f:X_{\rho}\rightarrow X_{\overset{\rightarrow}{\rho}}$ such that $\Delta_{i}\prec\Delta_{f(i)}$ for all $i\in X_{\rho}$. Moreover, there exists a subset $A\subset X_{\rho}$ such that

[TABLE]

Similarly, $\rho\notin\mathfrak{S}^{r}(\mathfrak{m})$ if and only if there exists an injective map $f:Y_{\rho}\rightarrow Y_{\overset{\leftarrow}{\rho}}$ such that $\Delta_{f(i)}\prec\Delta_{i}$ for all $i\in Y_{\rho}$; there exists a subset $A\subset Y_{\rho}$ such that

[TABLE]

The following consequence will be useful.

Corollary 2.6**.**

Suppose that $\Delta\leq\mathfrak{m}$ and $\rho\in\tensor*[^{-}]\Delta{}$ (i.e., $\rho\in\Delta$ but $\rho\neq b(\Delta)$). Then $\rho\in\operatorname{supp}L_{\rho}(Z(\mathfrak{m}))$. Hence $\rho\in\operatorname{supp}D_{\rho;\rho^{\prime}}(Z(\mathfrak{m}))$ if $\rho^{\prime}\neq e(\Delta)$ or $\rho^{\prime}\neq\rho$.

Indeed, the first statement follows from (2.7). The second statement follows from (2.3) and (2.8).

2.4. Ladder representations

We define a partial order on $\operatorname{Cusp}$ by $\rho_{1}\leq\rho_{2}$ if $\rho_{2}=\rho_{1}[n]$ for some $n\in\mathbb{Z}_{\geq 0}$.

Recall that a ladder (cf. [LM14]) is a multisegment of the form

[TABLE]

By [KL12], if $\mathfrak{m}$ is a ladder then

[TABLE]

where each summand is the tensor product of two ladders. In particular,

[TABLE]

We will need another fact

Lemma 2.7**.**

Suppose that $\mathfrak{m}$ is a ladder as in (2.9) and let $\Delta_{k+1}\in\mathcal{SEG}$ with $\Delta_{k+1}\prec\Delta_{k}$. Then we have a short exact sequence

[TABLE]

where

[TABLE]

Moreover,

[TABLE]

Proof.

Let $\Pi=Z(\Delta_{k+1})\times Z(\mathfrak{m})$ and $\mathfrak{m}^{\prime}=\Delta_{1}+\dots+\Delta_{k+1}$. Note that

[TABLE]

Thus, $\operatorname{soc}(\Pi^{\vee})=Z(\mathfrak{m}^{\prime\vee})$, or equivalently

[TABLE]

On the other hand, let $\tau=Z(\Delta_{k}\cap\Delta_{k+1})\times Z(\mathfrak{m})$ which is irreducible. (For instance, this easily follows from [LM16, Corollary 5.14].) Then

[TABLE]

and by the recipe of [LM16, Proposition 5.6 and Theorem 5.11] (which easily reduces to the case where $\Delta_{k+1}\setminus\Delta_{k}$ is a singleton) we have

[TABLE]

We conclude that

[TABLE]

To finish the proof of the lemma, it remains to show that $\Pi$ is of length two. We will prove it by induction on $\deg\mathfrak{m}$. The base of the induction is trivial.

For the induction step, note that by (2.1) and (2.11) we have

[TABLE]

and

[TABLE]

Moreover, by (2.10) and the geometric lemma, for any $\rho\in\mathfrak{S}^{l}(\Pi)$ we have

[TABLE]

where $\mathfrak{m}^{(i)}=\Delta_{1}^{\prime}+\dots+\Delta^{\prime}_{k}$ with $\Delta_{i}^{\prime}=\tensor*[^{-}]\Delta{}_{i}$ and $\Delta^{\prime}_{j}=\Delta_{j}$ if $j\neq i$. Note that $\mathfrak{m}^{(i)}$ is a ladder since $b(\overset{\rightarrow}{\Delta_{i}})\neq b(\Delta_{i-1})$ if $b(\Delta_{i})\in\mathfrak{S}^{l}(\mathfrak{m})$. Thus, by induction hypothesis we have for any $\rho\in\mathfrak{S}^{l}(\Pi)$

[TABLE]

On the other hand, it follows from (2.2), (2.11) and the fact that

[TABLE]

and

[TABLE]

that

[TABLE]

It follows that

[TABLE]

and in particular,

[TABLE]

This implies that $\Pi\leq Z(\mathfrak{m}^{\prime})+Z(\mathfrak{n})$ since for any $0\neq\pi^{\prime}\leq\Pi$ we have $J^{\operatorname{GL}}_{\{\rho\};*}(\pi^{\prime})\neq 0$ for some $\rho\in\mathfrak{S}^{l}(\Pi)$. Thus $\Pi$ is of length two. ∎

Passing to the contragredient, we get

Corollary 2.8**.**

Suppose that $\mathfrak{m}=\Delta_{1}+\dots+\Delta_{k}$ is a ladder and $\Delta_{1}\prec\Delta_{0}$. Then we have a short exact sequence

[TABLE]

where

[TABLE]

Moreover,

[TABLE]

3. Classical groups

Next, we turn to classical groups which are the main object of the paper. Again, most of the results in this section are standard.

3.1.

Let $E$ be either $F$ or a quadratic Galois extension of $F$. In the former case let $\varrho$ be the Galois involution of $E/F$. In the latter case $\varrho=\operatorname{Id}$.

All the notation of the previous section will be used with respect to $E$. Since we work with groups over $F$ this means that formally $\operatorname{GL}_{n}$ should be replaced by its restriction of scalars with respect to $E/F$. In order to avoid extra notation we use this convention implicitly throughout.

Let $\iota$ be the involution $g\mapsto\tensor[^{t}]g{{}^{-1}}$ of the general linear group and let $\tilde{}$ be the composition of $\iota$ with $\varrho$ (which commute). We use the same notation for the induced actions on $\mathcal{C}^{\operatorname{GL}}$, $\mathcal{R}^{\operatorname{GL}}$, $\mathcal{SEG}$, $\mathbb{N}({\mathcal{SEG}})$ etc.. (For convenience we sometimes write $\tilde{Z}(\mathfrak{m})$ for $\widetilde{Z(\mathfrak{m})}=Z(\tilde{\mathfrak{m}})$.) Note that $\iota$ is a covariant functor of $\mathcal{C}^{\operatorname{GL}}$. We have

[TABLE]

Also, $\mathfrak{c}(\pi^{\varrho})=\mathfrak{c}(\pi)$ and $\mathfrak{c}(\tilde{\pi})=-\mathfrak{c}(\pi)$ for any $\pi\in\mathcal{C}^{\operatorname{GL}}$ which admits a central character.

We consider an anisotropic $\epsilon$-hermitian space $V_{0}$ over $E$ with $\epsilon\in\{\pm 1\}$. Thus, $V_{0}$ is trivial in the symplectic case, of dimension $\leq 4$ in the quadratic case and of dimension $\leq 2$ in the hermitian case. We then have a tower $V_{n}$, $n\geq 0$ of $\epsilon$-hermitian spaces where $V_{n}$ is obtained from $V_{0}$ by adding $n$ copies of a hyperbolic plane. Consider the sequence of isometry groups $G_{n}=\operatorname{Isom}(V_{n})$ (of $F$-rank $n$). (See [MVW87, Chapitre 1] for basic facts about classical groups.) Note that the center of $G_{n}$ is anisotropic. Let

[TABLE]

(Note that $\operatorname{Irr}G_{0}\subset\operatorname{Cusp}^{G}$ and in particular $\mathbf{1}\in\operatorname{Cusp}^{G}$ if $G_{0}=1$. On the other hand, if $V_{0}$ is the [math]-dimensional quadratic space then $\operatorname{Cusp}G_{1}=\emptyset$.) As before we write $\deg\pi=n$ if $\pi\in\operatorname{Irr}G_{n}$. For any $m\leq n$ the stabilizer of a totally isotropic $m$-dimensional subspace $U$ of $V_{n}$ is a parabolic subgroup of $G_{n}$, which is defined over $F$ and up to conjugation uniquely determined by $m$. The Levi part is canonically $\operatorname{GL}(U)\times\operatorname{Isom}(U^{\perp}/U)$ which we identify with $\operatorname{GL}_{m}\times G_{n-m}$. This gives rise to parabolic induction

[TABLE]

which are bilinear biexact bifunctors with an associativity constraint. Thus, $\mathcal{C}^{G}$ is a left module category over $\mathcal{C}^{\operatorname{GL}}$ in the sense of [EGNO15, §7.1]. The Grothendieck group $\mathcal{R}^{G}$ of $\mathcal{C}^{G}$ becomes a left $\mathcal{R}^{\operatorname{GL}}$-module. We will continue to denote the action of $\mathcal{R}^{\operatorname{GL}}$ on $\mathcal{R}^{G}$ by $\rtimes$. This gives rise to an action of $\mathcal{R}^{\operatorname{GL}}\otimes\mathcal{R}^{\operatorname{GL}}$ on $\mathcal{R}^{\operatorname{GL}}\otimes\mathcal{R}^{G}$ (also denoted by $\rtimes$) given by

[TABLE]

We have

[TABLE]

Once again, the left-adjoint

[TABLE]

of $\rtimes$ is given by $\oplus_{n\geq 0}(\oplus_{n_{1}+n_{2}=n}J^{G}_{n_{1};n_{2}})$ where

[TABLE]

is the normalized Jacquet functor. On the level of Grothendieck groups we get a map

[TABLE]

(By abuse of notation we sometimes consider $\mu^{*}$ as a map from $\mathcal{C}^{G}$ via $\operatorname{JH}$.) Using the geometric lemma (see [Tad95, Ban99] and the comments in §1 and §15 of [MT02]) $\mu^{*}$ satisfies

[TABLE]

where

[TABLE]

is the ring homomorphism corresponding to the composition of exact functors

[TABLE]

where $s(\alpha\otimes\beta)=\tilde{\beta}\otimes\alpha$. Thus,

[TABLE]

Note that

[TABLE]

in accordance with (3.2) and (3.3).

We will also let

[TABLE]

be the homomorphism corresponding to

[TABLE]

i.e., to the composition of (3.4) with the functor

[TABLE]

where $p_{0}$ is the projection of $\mathcal{C}^{\operatorname{GL}}=\oplus_{n\geq 0}\mathcal{C}(\operatorname{GL}_{n})$ to $\mathcal{C}(\operatorname{GL}_{0})$ (the category of finite-dimensional vector spaces). Thus, $M^{*}_{\max}=\times\circ s\circ\operatorname{m}^{*}$. Explicitly,

[TABLE]

For example, for any $\Delta\in\mathcal{SEG}$ we have (using (2.5))

[TABLE]

and

[TABLE]

More generally, for any ladder $\mathfrak{m}$ as in (2.9) we have

[TABLE]

and

[TABLE]

3.2.

The following is an immediate consequence of Frobenius reciprocity (cf. [LM16, Lemma 2.5]).

Lemma 3.1**.**

Suppose that $\pi\in\mathcal{C}^{\operatorname{GL}}$ and $\sigma\in\mathcal{C}^{G}$ are SI and that $\operatorname{soc}(\pi)\otimes\operatorname{soc}(\sigma)$ occurs with multiplicity one in $\mu^{*}(\pi\rtimes\sigma)$. Then $\pi\rtimes\sigma$ is also SI. In particular, (by (1.1)) $\operatorname{soc}(\pi\rtimes\sigma)=\operatorname{soc}(\operatorname{soc}(\pi)\rtimes\operatorname{soc}(\sigma))$.

Let $X\subset\mathbb{N}({\operatorname{Cusp}^{\operatorname{GL}}})$. As before, we write $J^{G}_{X;*}$ for the composition of $J^{G}$ with

[TABLE]

We will also write

[TABLE]

for the corresponding map of Grothendieck groups. Similarly, let

[TABLE]

be the composition of $M^{*}$ with $p_{X}\otimes\operatorname{Id}$. For any $A\subset\operatorname{Cusp}^{\operatorname{GL}}$ we have

[TABLE]

and

[TABLE]

Definition 3.2**.**

Let $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$. We say that $\sigma\in\mathcal{C}^{G}$ is $\rho$-reduced if $J^{G}_{\{\rho\};*}(\sigma)=0$. For a subset $A\subset\operatorname{Cusp}^{\operatorname{GL}}$ we say that $\sigma\in\mathcal{C}^{G}$ is $A$-reduced if it is $\rho$-reduced for all $\rho\in A$.

Note that if $\sigma\in\operatorname{Irr}G$ then $\sigma$ is $\rho$-reduced if and only if there does not exist $\sigma^{\prime}\in\operatorname{Irr}G$ such that $\sigma\hookrightarrow\rho\rtimes\sigma^{\prime}$.

The following is proved using the same argument as in Lemma 2.3.

Lemma 3.3**.**

For any $\sigma\in\operatorname{Irr}G$ and $A\subset\operatorname{Cusp}^{\operatorname{GL}}$ there exist $\pi\in\operatorname{Irr}\operatorname{GL}$ and $\sigma^{\prime}\in\operatorname{Irr}G$ such that:

1. (1)

$\sigma\hookrightarrow\pi\rtimes\sigma^{\prime}$. 2. (2)

$\operatorname{supp}\pi\subset A$. 3. (3)

$\sigma^{\prime}$* is $A$-reduced.*

Moreover, $\sigma^{\prime}$ is uniquely determined by $\sigma$ and is characterized by the following properties:

1. (A)

There exists $\alpha\in\operatorname{Irr}\operatorname{GL}$ such that $\alpha\otimes\sigma^{\prime}\leq\mu^{*}_{\mathbb{N}({A});*}(\sigma)$. 2. (B)

If $\alpha\otimes\beta\leq\mu^{*}_{\mathbb{N}({A});*}(\sigma)$ with $\alpha\in\operatorname{Irr}\operatorname{GL}$ and $\beta\in\operatorname{Irr}G$ then either $\beta=\sigma^{\prime}$ or $\deg\beta>\deg\sigma^{\prime}$.

We write $D_{A}(\sigma)=\sigma^{\prime}$.

We will need the following result which is a direct consequence of [Tad09, Proposition 1.3].

Lemma 3.4**.**

For any $\pi\in\operatorname{Irr}\operatorname{GL}$ and $\sigma\in\operatorname{Irr}G$ there exists $\sigma^{\prime}\in\operatorname{Irr}G$ which occurs with multiplicity one in $\operatorname{JH}(\pi\rtimes\sigma)$.

Corollary 3.5**.**

Suppose that $\pi\rtimes\sigma$ is irreducible. Then $D_{A;\tilde{A}}(\pi)\rtimes D_{A}(\sigma)=D_{A}(\pi\rtimes\sigma)$. In particular, $D_{A;\tilde{A}}(\pi)\rtimes D_{A}(\sigma)$ is irreducible.

Proof.

By Lemma 3.4 it is enough to show that

[TABLE]

Indeed, it follows from (3.11), (3.12) and Lemma 2.3 that

[TABLE]

while if $\mu^{*}_{\mathbb{N}({A});*}(\pi\rtimes\sigma)\geq\pi^{\prime}\otimes\sigma^{\prime}$ for some $\pi^{\prime}\in\operatorname{Irr}\operatorname{GL}$ and $\sigma^{\prime}\in\operatorname{Irr}G$ then $\deg\sigma^{\prime}\geq\deg D_{A;\tilde{A}}(\pi)+\deg D_{A}(\sigma)$. Hence, (3.13) follows from Lemma 3.3. ∎

Remark 3.6*.*

(Compare with Remark 2.4.) Let $A$ be a subset of $\operatorname{Cusp}^{\operatorname{GL}}$. Denote by $\mathcal{C}^{G}_{A\operatorname{-red}}$ the Serre subcategory of $\mathcal{C}^{G}$ consisting of $A$-reduced representations, $(\operatorname{Irr}G)_{A\operatorname{-red}}=\operatorname{Irr}\mathcal{C}^{G}_{A\operatorname{-red}}\subset\operatorname{Irr}G$ and $\mathcal{R}^{G}_{A\operatorname{-red}}=\mathcal{R}(\mathcal{C}^{G}_{A\operatorname{-red}})\subset\mathcal{R}^{G}$. Suppose that $\tilde{A}=A=\overset{\rightarrow}{A}$. Then $\pi\in(\operatorname{Irr}G)_{A\operatorname{-red}}$ if and only if the cuspidal data of $\pi$ is of the form $\rho_{1}+\dots+\rho_{k};\sigma$ where $\rho_{i}\notin A$ for all $i$. (The $\rho_{i}$’s are defined up to $\tilde{}$.) The main result of [Jan97] is that the map

[TABLE]

defines a bijection between $\operatorname{Irr}G$ and $\operatorname{Irr}G_{A\operatorname{-red}}\times\operatorname{Irr}G_{A^{c}\operatorname{-red}}$ which induces an isomorphism

[TABLE]

of $\mathcal{R}^{\operatorname{GL}}=\mathcal{R}^{\operatorname{GL}}_{A\operatorname{-red}}\otimes\mathcal{R}^{\operatorname{GL}}_{A^{c}\operatorname{-red}}$-modules. It is likely that there is an equivalence of module categories over $\mathcal{C}^{\operatorname{GL}}\simeq\mathcal{C}^{\operatorname{GL}}_{A\operatorname{-red}}\otimes\mathcal{C}^{\operatorname{GL}}_{A^{c}\operatorname{-red}}$ between $\mathcal{C}^{G}$ and $\mathcal{C}^{G}_{A\operatorname{-red}}\otimes\mathcal{C}^{G}_{A^{c}\operatorname{-red}}$. See [Hei15] for a related result.

3.3. Classification

Recall that by Casselman’s criterion, a representation $\sigma\in\operatorname{Irr}G$ is tempered if and only if $\mathfrak{c}(\pi)\geq 0$ whenever $\pi\otimes\sigma^{\prime}\leq\mu^{*}(\sigma)$, $\pi\in\operatorname{Irr}\operatorname{GL}$, $\sigma^{\prime}\in\operatorname{Irr}G$. (Cf. [MT02, §16] for the even orthogonal case.) Dually, a representation $\sigma\in\operatorname{Irr}G$ is called cotempered777Note that in [HM08] these representations were called negative. However, we prefer to call them cotempered to emphasize the analogy with tempered representations. if $\mathfrak{c}(\pi)\leq 0$ whenever $\pi\otimes\sigma^{\prime}\leq\mu^{*}(\sigma)$, $\pi\in\operatorname{Irr}\operatorname{GL}$, $\sigma^{\prime}\in\operatorname{Irr}G$. We denote by $\operatorname{Irr}_{\operatorname{temp}}G$ (resp., $\operatorname{Irr}_{\operatorname{cotemp}}G$) the set of irreducible tempered (resp., cotempered) representations of $G_{n}$, $n\geq 0$. Thus, $(\operatorname{Irr}_{\operatorname{temp}}G)^{t}=\operatorname{Irr}_{\operatorname{cotemp}}G$.

A segment $\Delta\in\mathcal{SEG}$ is called positive (resp., non-negative) if $\mathfrak{c}(\Delta)>0$ (resp., $\mathfrak{c}(\Delta)\geq 0$). We denote by $\mathcal{SEG}_{>0}$ (resp., $\mathcal{SEG}_{\geq 0}$) the set of positive (resp., non-negative) segments. An element $\mathfrak{m}$ of $\mathbb{N}({\mathcal{SEG}_{>0}})$ (resp., $\mathbb{N}({\mathcal{SEG}_{\geq 0}})$) is called a positive (resp., non-negative) multisegment. In this case we will also call the representation $Z(\mathfrak{m})$ Z-positive (to emphasize the relation to the Zelevinsky classification).

The Langlands classification for classical groups asserts that for any positive multisegment $\mathfrak{m}$ and $\theta\in\operatorname{Irr}_{\operatorname{temp}}G$ the representation $\mathfrak{l}(\mathfrak{m})\rtimes\theta$ is CSI and the map

[TABLE]

is a bijection between $\mathbb{N}({\mathcal{SEG}_{>0}})\times\operatorname{Irr}_{\operatorname{temp}}G$ and $\operatorname{Irr}G$. (See [BJ03, Appendix] for the split even orthogonal case.)

Dually, we have the following

Theorem 3.7**.**

[HM08]888[loc. cit.] does not treat the unitary case, but it can be handled the same way. We omit the details.

1. (1)

For any positive multisegment $\mathfrak{m}$ and $\theta\in\operatorname{Irr}_{\operatorname{cotemp}}G$ the representation

[TABLE]

is SI. 2. (2)

The map

[TABLE]

is a bijection between $\mathbb{N}({\mathcal{SEG}_{>0}})\times\operatorname{Irr}_{\operatorname{cotemp}}G$ and $\operatorname{Irr}G$. 3. (3)

$\mathfrak{z}(\mathfrak{m})\rtimes\theta$* is irreducible if and only if*

1. (a)

$Z(\Delta)\rtimes\theta$* is irreducible for all $\Delta\leq\mathfrak{m}$, and,* 2. (b)

$Z(\Delta)\times Z(\Delta^{\prime})$* and $Z(\Delta)\times Z(\widetilde{\Delta^{\prime}})$ are irreducible whenever $\Delta+\Delta^{\prime}\leq\mathfrak{m}$.* 4. (4)

$Z(\mathfrak{m};\theta)^{\vee}=Z(\mathfrak{m}^{\varrho};\theta^{\vee})$* for any $\mathfrak{m}\in\mathbb{N}({\mathcal{SEG}_{>0}})$ and $\sigma\in\operatorname{Irr}_{\operatorname{cotemp}}G$.* 5. (5)

$L(\mathfrak{m};\theta)^{t}=Z(\mathfrak{m};\theta^{t})$.

Note that part 5 is implicit in [HM08] but it can be proved using a well-known argument of Rodier [Rod82]. Namely, for a fixed supercuspidal data we prove the relation $L(\mathfrak{m};\theta)^{t}=Z(\mathfrak{m};\theta^{t})$ by induction on $\beta(\mathfrak{m})$ where $\beta(\mathfrak{m})$ is defined as follows. Suppose that $\mathfrak{m}=\Delta_{1}+\dots+\Delta_{k}$ with $\mathfrak{c}(\Delta_{1})\geq\dots\geq\mathfrak{c}(\Delta_{k})>0$ and $G$ has rank $n$. Let $(x_{1},\dots,x_{n})\in\mathbb{R}^{n}$ be the vector

[TABLE]

Then $\beta(\mathfrak{m}):=\sum_{i=1}^{n}(n+1-i)x_{i}$. We have (see e.g. [Tad09])

[TABLE]

with $\beta(\mathfrak{m}_{i})<\beta(\mathfrak{m})$. (The pairs $(\mathfrak{m}_{i},\theta_{i})$ are not necessarily distinct.) Applying t and the induction hypothesis we get

[TABLE]

Since $Z(\mathfrak{m};\theta^{t})$ occurs in $\mathfrak{z}(\mathfrak{m})\rtimes\theta^{t}$ and $L(\mathfrak{m};\theta)^{t}$ is irreducible we necessarily have $L(\mathfrak{m};\theta)^{t}=Z(\mathfrak{m};\theta^{t})$ by the uniqueness part of Theorem 3.7.

For any segment $\Delta$ define

[TABLE]

Given any multisegment $\mathfrak{m}=\Delta_{1}+\dots+\Delta_{k}$ and $\sigma\in\operatorname{Irr}_{\operatorname{cotemp}}G$ we set

[TABLE]

Note that $\mathfrak{z}(\mathfrak{m};\sigma)$ is SI if and only if $Z(\mathfrak{m};\sigma)$ is irreducible if and only if $Z(\mathfrak{m}_{=0})\rtimes\sigma$ is irreducible.

The following is an analogue of [Tad09, Proposition 1.3], which is obtained from it by applying t.

Proposition 3.8**.**

For any $\mathfrak{m},\mathfrak{m}^{\prime}\in\mathbb{N}({\mathcal{SEG}})$ and $\theta\in\operatorname{Irr}_{\operatorname{cotemp}}G$ we have

[TABLE]

Thus, if $Z(\mathfrak{m})\rtimes Z(\mathfrak{m}^{\prime};\theta)$ is irreducible then it is equal to $Z(\mathfrak{m}+\mathfrak{m}^{\prime};\theta)$.

Specializing to the case $\mathfrak{m}^{\prime}=0$ we get

Corollary 3.9**.**

For any $\mathfrak{m}\in\mathbb{N}({\mathcal{SEG}})$ and $\theta\in\operatorname{Irr}_{\operatorname{cotemp}}G$ we have $Z(\mathfrak{m};\theta)\leq Z(\mathfrak{m})\rtimes\theta$. Thus, if $Z(\mathfrak{m})\rtimes\theta$ is irreducible then it is equal to $Z(\mathfrak{m};\theta)$ and in particular, $Z(\mathfrak{m}_{=0})\rtimes\theta$ is irreducible.

Theorem 3.10**.**

[Tad98]999Once again, this is only stated for odd orthogonal and symplectic groups, but the same argument works in general Let $\Delta\in\mathcal{SEG}$ and $\sigma\in\operatorname{Cusp}^{G}$. Then the following conditions are equivalent.

1. (1)

$Z(\Delta)\rtimes\sigma$* is irreducible.* 2. (2)

$L(\Delta)\rtimes\sigma$* is irreducible.* 3. (3)

$\rho\rtimes\sigma$* is irreducible for every $\rho\in\Delta$.*

Corollary 3.11**.**

Let $\mathfrak{m}=\Delta_{1}+\dots+\Delta_{k}$ with $\mathfrak{c}(\Delta_{i})=0$ for all $i$ and let $\sigma\in\operatorname{Cusp}^{G}$. Let $\Sigma=\mathfrak{z}(\mathfrak{m})\rtimes\sigma=Z(\mathfrak{m})\rtimes\sigma$ and $\Sigma^{\prime}=\mathfrak{l}(\mathfrak{m})\rtimes\sigma=L(\mathfrak{m})\rtimes\sigma$ Then the following conditions are equivalent.

1. (1)

$\Sigma$* is irreducible (and cotempered).* 2. (2)

$Z(\Delta_{i})\rtimes\sigma$* is irreducible for all $i$.* 3. (3)

$\Sigma^{\prime}$* is irreducible (and tempered).* 4. (4)

$L(\Delta_{i})\rtimes\sigma$* is irreducible for all $i$.* 5. (5)

$\rho\rtimes\sigma$* is irreducible for every $\rho\in\cup_{i}\Delta_{i}$.*

In this case, $\Sigma^{\vee}=\mathfrak{z}(\mathfrak{m}^{\varrho})\rtimes\sigma^{\vee}$ and $\Sigma^{\prime\vee}=\mathfrak{l}(\mathfrak{m}^{\varrho})\rtimes\sigma^{\vee}$.

Indeed, the equivalence of conditions 3 and 4 (for any $\sigma$ square-integrable) follows from [MT02, Theorem 13.1] which is based on the results and techniques of Goldberg [Gol94, Gol95a, Gol95b]. The equivalence of conditions 1 and 3 follows from the properties of t. The rest follows from Theorem 3.10 and (3.2).

Definition 3.12**.**

For any multisegment $\mathfrak{m}=\Delta_{1}+\dots+\Delta_{k}$ we write

[TABLE]

and similarly for $\mathfrak{m}_{\geq 0}$, $\mathfrak{m}_{<0}$, $\mathfrak{m}_{\leq 0}$, $\mathfrak{m}_{=0}$. Thus,

[TABLE]

If $\pi=Z(\mathfrak{m})$ then we write $\pi_{+}=Z(\mathfrak{m}_{>0})$.

The following consequence will be our main tool for proving irreducibility.

Corollary 3.13**.**

1. (1)

Let $\mathfrak{m}\in\mathbb{N}({\mathcal{SEG}_{>0}})$, $\theta\in\operatorname{Irr}_{\operatorname{cotemp}}G$ and $\pi\in\mathcal{C}^{G}$. Assume that

[TABLE]

and

[TABLE]

Then $\pi$ is irreducible (and isomorphic to $Z(\mathfrak{m};\theta)$). 2. (2)

Assume that $\mathfrak{m}\in\mathbb{N}({\mathcal{SEG}_{\geq 0}})$ and $\sigma\in\operatorname{Cusp}^{G}$ are such that $\rho\rtimes\sigma$ is irreducible for every $\rho\in\operatorname{supp}\mathfrak{m}_{=0}$. Let $\pi\in\mathcal{C}^{G}$ and assume that

[TABLE]

and

[TABLE]

Then $\pi$ is irreducible (and isomorphic to $Z(\mathfrak{m};\sigma)=Z(\mathfrak{m}_{>0};\mathfrak{z}(\mathfrak{m}_{=0})\rtimes\sigma)$).

Proof.

1. (1)

By Theorem 3.7 and (1.1) both $\pi$ and $\pi^{\vee}$ are SI and

[TABLE]

Hence $\pi$ is irreducible by (1.2). 2. (2)

By Corollary 3.11 $\mathfrak{z}(\mathfrak{m}_{=0})\rtimes\sigma$ is irreducible. The irreducibility of $\pi$ now follows from the first part.∎

4. A reducibility result

For any $\sigma\in\operatorname{Irr}G$ we write

[TABLE]

By (3.2) we have $\tilde{S_{\sigma}}=S_{\sigma}$.

In this section we prove the following result.

Theorem 4.1**.**

Suppose that $\sigma\in\operatorname{Cusp}^{G}$ and $\pi\in\operatorname{Irr}\operatorname{GL}$ are such that $\operatorname{supp}\pi\cap S_{\sigma}\neq\emptyset$. Then $\pi\rtimes\sigma$ is reducible.

4.1.

We start with the following definition. Recall the notation of §6.

Definition 4.2**.**

Let $\pi\in\operatorname{Irr}\operatorname{GL}$ and $A\subset\operatorname{Cusp}^{\operatorname{GL}}$.

1. (1)

We say that $\pi$ is $A$-confined if $D_{A^{c};A^{c}}(\pi)=\pi$, i.e., if $\mathfrak{S}^{l}(\pi)\cup\mathfrak{S}^{r}(\pi)\subset A$. 2. (2)

We say that $\pi$ is $A$-critical if it is $A$-confined, different from $\mathbf{1}$, and for any $\rho\in A$, either $D_{\rho;\tilde{\rho}}(\pi)=\pi$ (i.e., $\pi$ is left $\rho$-reduced and right $\tilde{\rho}$-reduced) or $\operatorname{supp}D_{\rho;\tilde{\rho}}(\pi)\cap A=\emptyset$.

Note that $\pi$ is $A$-confined (resp., $A$-critical) if and only if $\tilde{\pi}$ is $\tilde{A}$-confined (resp., $\tilde{A}$-critical).

Next, we recall basic facts about the set $S_{\sigma}$ when $\sigma\in\operatorname{Cusp}^{G}$.

Theorem 4.3**.**

(Casselman, Silberger, Mœglin) Let $\sigma\in\operatorname{Cusp}^{G}$ and $\rho\in S_{\sigma}$. Then

1. (1)

$\tilde{\rho}\in\rho[\mathbb{R}]:=\{\rho[x]:x\in\mathbb{R}\}$* (e.g., [Ren10, Théorème VII.1.3]).* 2. (2)

$\rho[\mathbb{R}]\cap S_{\sigma}=\{\rho,\tilde{\rho}\}$* [Sil80].* 3. (3)

$\tilde{\rho}\in\rho[\mathbb{Z}]:=\{\rho[m]:m\in\mathbb{Z}\}$. Thus, if $\mathfrak{c}(\rho)\geq 0$ then $[\tilde{\rho},\rho]\in\mathcal{SEG}$ **[Mœg14, Théorème 3.1]**.101010Technically, only quasi-split groups are considered in [Mœg14] but as mentioned there, this is only for simplicity.

The first two parts are special cases of general results about reductive groups. The proof of the third part lies deeper – it uses the stabilization of the twisted trace formula (at least in a simple form). We will only use it in a superficial way to simplify the statements.

Proposition 4.4**.**

Let $\sigma\in\operatorname{Cusp}^{G}$. Then the $S_{\sigma}$-critical representations are of the form

[TABLE]

as we vary over $\alpha\in S_{\sigma}$ with $\mathfrak{c}(\alpha)\geq 0$.

Proof.

It is clear that the above representations are well-defined and $S_{\sigma}$-critical. Conversely, suppose that $\pi=Z(\mathfrak{m})\in\operatorname{Irr}\operatorname{GL}$ is $S_{\sigma}$-critical. We first remark that since $\operatorname{supp}\pi\neq\emptyset$ and $\pi$ is $S_{\sigma}$-confined, necessarily $\operatorname{supp}\pi\cap S_{\sigma}\neq\emptyset$. By passing to $\tilde{\pi}$ if necessary we may assume that there exists $\alpha\in\operatorname{supp}\pi\cap S_{\sigma}$ with $\mathfrak{c}(\alpha)\geq 0$. We fix such $\alpha$ for the rest of the proof.

We first claim that $\operatorname{supp}\pi\subset\alpha[\mathbb{Z}]$.

For otherwise we could write $\pi=\pi_{1}\times\pi_{2}$ where $\pi_{1},\pi_{2}\neq\mathbf{1}$, $\operatorname{supp}\pi_{1}\subset\alpha[\mathbb{Z}]$, and $\operatorname{supp}\pi_{2}\cap\alpha[\mathbb{Z}]=\emptyset$. If $\rho\in\mathfrak{S}^{l}(\pi_{2})$ then $D_{\rho;\tilde{\rho}}(\pi)=\pi_{1}\times D_{\rho;\tilde{\rho}}(\pi_{2})$. Thus, $D_{\rho;\tilde{\rho}}(\pi)\neq\pi$ but $\alpha\in\operatorname{supp}D_{\rho;\tilde{\rho}}(\pi)$, in contradiction to the assumption on $\pi$.

Next we claim that

[TABLE]

Indeed, let $\Delta\leq\mathfrak{m}$ with $b(\Delta)$ maximal. Then by Proposition 2.5 $b(\Delta)\in\mathfrak{S}^{l}(\pi)$ and hence, $b(\Delta)\in S_{\sigma}\cap\alpha[\mathbb{Z}]=\{\tilde{\alpha},\alpha\}$ by Theorem 4.3.

Similarly,

[TABLE]

In particular,

[TABLE]

Henceforth, let $r$ be the maximal length of a segment in $\mathfrak{m}$ and for any segment $\Delta$ denote by $a_{\mathfrak{m}}(\Delta)$ its multiplicity in $\mathfrak{m}$.

Proposition 2.5 and the assumption that $\pi$ is $S_{\sigma}$-confined imply that for any segment $\Delta$ of length $r$ we have

[TABLE]

From now on, let $\Delta$ be the segment in $\mathfrak{m}$ of length $r$ with $b(\Delta)$ maximal. For $\rho=b(\Delta)$ we have

[TABLE]

Indeed, Proposition 2.5 $\rho\in\mathfrak{S}^{l}(\pi)\subset S_{\sigma}\cap\operatorname{supp}\pi=\{\alpha,\tilde{\alpha}\}$. The second statement follows from the condition on $\pi$ since $D_{\rho;\tilde{\rho}}(\pi)\neq\pi$.

Consider now the case $\tilde{\alpha}=\alpha$.

1. (1)

If $r=1$, i.e., if all elements of $\mathfrak{m}$ are singletons, then by (4.3) we have $\pi=\alpha^{\times k}$ for some $k>0$. 2. (2)

Suppose that $r=2$. Then by (4.5), $\Delta=[\alpha,\overset{\rightarrow}{\alpha}]$ and by (4.4) $a_{\mathfrak{m}}(\overset{\leftarrow}{\Delta})=a_{\mathfrak{m}}(\Delta)$. Using (4.1), (4.2) and (4.3) we necessarily have

[TABLE]

for some $k\geq 0$ and $l>0$. 3. (3)

Assume on the contrary that $r>2$. Then by (4.4), $\overset{\leftarrow}{\Delta}\leq\mathfrak{m}$. On the other hand, $\alpha=b(\Delta)\in\overset{\leftarrow}{\Delta}$ but $\alpha\neq b(\overset{\leftarrow}{\Delta}),e(\overset{\leftarrow}{\Delta})$ since $r>2$. By Corollary 2.6 (applied to $\overset{\leftarrow}{\Delta}$) we have $\alpha\in\operatorname{supp}D_{\alpha;\alpha}(\pi)$ in contradiction with (4.5).

This concludes the case $\tilde{\alpha}=\alpha$. From now on we assume that $\tilde{\alpha}\neq\alpha$.

1. (1)

Suppose that $r=1$, i.e., all segments in $\mathfrak{m}$ are singletons. Then either

[TABLE]

for some $k>0$ or

[TABLE]

for some $k,l\geq 0$ with $k+l>0$ where $kl=0$ if $\tilde{\alpha}=\overset{\leftarrow}{\alpha}$.

Indeed, by (4.3) and (4.4) we have

[TABLE]

for some $k,l,m\geq 0$. If $\tilde{\alpha}=\overset{\leftarrow}{\alpha}$ we may assume that $l$ or $m$ equals [math]. Thus,

[TABLE]

We have

[TABLE]

and therefore either $k=0$ or $l=m=0$ for otherwise $\pi$ is not $S_{\sigma}$-critical. 2. (2)

Suppose that $r>1$. Then $\Delta=[\tilde{\alpha},\alpha]$.

Assume on the contrary that $\Delta\neq[\tilde{\alpha},\alpha]$. Then $e(\Delta)\not\in S_{\sigma}$ (since $b(\Delta)\in\{\alpha,\tilde{\alpha}\}$ and $\Delta\neq\{\alpha\},\{\tilde{\alpha}\}$) and hence $\overset{\leftarrow}{\Delta}\leq\mathfrak{m}$ by (4.4). Let $\rho=b(\Delta)$. Since $\rho\in\overset{\leftarrow}{\Delta}$, $\rho\neq b(\overset{\leftarrow}{\Delta})$ and $\tilde{\rho}\neq\rho$ we have $\rho\in\operatorname{supp}D_{\rho;\tilde{\rho}}(\pi)$ by Corollary 2.6 in contradiction with (4.5).

For the rest of the proof we assume that $\Delta=[\tilde{\alpha},\alpha]$ (and $\tilde{\alpha}\neq\alpha$). 3. (3)

We have

[TABLE]

and $\operatorname{supp}D_{\tilde{\alpha};\alpha}(\pi)\cap S_{\sigma}=\emptyset$.

Indeed, by Corollary 2.6 $\alpha\in\operatorname{supp}D_{\alpha;\tilde{\alpha}}(\pi)$. Thus, $D_{\alpha;\tilde{\alpha}}(\pi)=\pi$ by the assumption on $\pi$. On the other hand, $D_{\tilde{\alpha};\alpha}(\pi)\neq\pi$ since $\tilde{\alpha}\in\mathfrak{S}^{l}(\pi)$ and therefore $\operatorname{supp}D_{\tilde{\alpha};\alpha}(\pi)\cap S_{\sigma}=\emptyset$ by the assumption on $\pi$. 4. (4)

Assume that there exists $\Delta^{\prime}\neq\Delta$ with $\Delta^{\prime}\leq\mathfrak{m}$ and take such $\Delta^{\prime}$ with $b(\Delta^{\prime})$ maximal. Then either $b(\Delta^{\prime})=\tilde{\alpha}$ or $b(\Delta^{\prime})=\overset{\leftarrow}{\tilde{\alpha}}$.

Otherwise, we would have $b(\Delta^{\prime})\in\mathfrak{S}^{l}(\pi)$ (by Proposition 2.5 and the maximality of $b(\Delta^{\prime})$) and this contradicts either the assumption that $\pi$ is $S_{\sigma}$-confined if $b(\Delta^{\prime})\neq\alpha$ or (4.6) if $b(\Delta^{\prime})=\alpha$. 5. (5)

The condition $b(\Delta^{\prime})=\tilde{\alpha}$ implies $\Delta^{\prime}=\{\tilde{\alpha}\}$.

Otherwise, $e(\Delta^{\prime})\notin S_{\sigma}$ and therefore $e(\Delta^{\prime})\notin\mathfrak{S}^{r}(\pi)$. Thus, by Proposition 2.5 there exists $\Delta^{\prime\prime}\in\mathfrak{m}$ such that $\Delta^{\prime\prime}\prec\Delta^{\prime}$ and $e(\Delta^{\prime\prime})=e(\overset{\leftarrow}{\Delta^{\prime}})$. Hence, $b(\Delta^{\prime\prime})\neq\tilde{\alpha}\in\Delta^{\prime\prime}$ and therefore by Corollary 2.6 $\tilde{\alpha}\in\operatorname{supp}D_{\tilde{\alpha};\alpha}(\pi)$ which contradicts (4.6). 6. (6)

$\Delta^{\prime}\neq\{\tilde{\alpha}\}$.

Otherwise, by (4.2) and Proposition 2.5 $\tilde{\alpha}\in\mathfrak{S}^{r}(\pi)$ and we again get a contradiction to (4.6). 7. (7)

$b(\Delta^{\prime})\neq\overset{\leftarrow}{\tilde{\alpha}}$.

Otherwise, $\tilde{\alpha}\in\Delta^{\prime}$ (by (4.2)) and therefore by Corollary 2.6 $\tilde{\alpha}\in\operatorname{supp}D_{\tilde{\alpha};\alpha}(\pi)$ and we get a contradiction to (4.6). 8. (8)

Thus, $\pi=Z(\Delta)^{\times k}$ for some $k>0$.

The proposition follows. ∎

4.2. Proof of Theorem 4.1

Assume on the contrary that $\pi\rtimes\sigma$ is irreducible for some $\pi\in\operatorname{Irr}\operatorname{GL}$ such that $\operatorname{supp}\pi\cap S_{\sigma}\neq\emptyset$ and let $\pi$ be a counterexample with minimal $\deg\pi$. Clearly $\pi\neq\mathbf{1}$. We claim that $\pi$ is $S_{\sigma}$-critical. Indeed, $\pi$ is $S_{\sigma}$-confined since otherwise $D_{S_{\sigma}^{c};S_{\sigma}^{c}}(\pi)$ would be a counterexample of smaller degree in view of (2.4) and Corollary 3.5. Further, for any $\rho\in S_{\sigma}$ we have either $D_{\rho;\tilde{\rho}}(\pi)=\pi$ or $\operatorname{supp}D_{\rho;\tilde{\rho}}(\pi)\cap S_{\sigma}=\emptyset$ for otherwise $D_{\rho;\tilde{\rho}}(\pi)$ would be a counterexample of smaller degree.

By Proposition 4.4 and Theorem 3.10 it remains to show the following lemma. (It is clear that if $\pi_{1}\rtimes\sigma$ is reducible then $\pi_{1}\times\pi_{2}\rtimes\sigma$ is reducible for any $\pi_{2}$.)

Lemma 4.5**.**

Suppose that $\tilde{\alpha}=\alpha\in S_{\sigma}$ and

[TABLE]

Then $\pi\rtimes\sigma$ is reducible.

Proof.

Note that $\pi=L([\alpha,\overset{\rightarrow}{\alpha}]+[\overset{\leftarrow}{\alpha},\alpha])$. By [Tad09, Proposition 1.3] we have

[TABLE]

On the other hand, clearly

[TABLE]

Thus, to prove the reducibility of $\pi\rtimes\sigma$ it suffices to show that

[TABLE]

We show in fact that

[TABLE]

By (3.9) and (3.3), the left-hand side of (4.7) is

[TABLE]

On the other hand, we have

[TABLE]

and in particular

[TABLE]

Hence,

[TABLE]

Thus, the right-hand side of (4.7) is

[TABLE]

and (4.7) follows since $L([\overset{\leftarrow}{\alpha},\alpha])\rtimes\sigma$ is reducible. ∎

This concludes the proof of Theorem 4.1.

5. Case of two segments

Throughout this section we fix $\sigma\in\operatorname{Cusp}^{G}$. Thus,

[TABLE]

We will prove the following result.

Theorem 5.1**.**

Let $\Delta_{1},\Delta_{2}$ be two segments such that $\Delta_{i}\cap S_{\sigma}=\emptyset$, $i=1,2$. Then $Z(\Delta_{1}+\Delta_{2})\rtimes\sigma$ is reducible if and only if $\Delta_{1}$ and $\widetilde{\Delta_{2}}$ are linked and at least one of the following conditions is satisfied:

1. (1)

$\Delta_{1}$* and $\Delta_{2}$ are unlinked.* 2. (2)

$\mathfrak{c}(\Delta_{1})\cdot\mathfrak{c}(\Delta_{2})<0$.

Remark 5.2*.*

In the next section we will generalize this result. Nevertheless, although not logically necessarily, we opted to first give a proof in the special case in order to illustrate the ideas.

Suppose first that $\Delta_{1}$ and $\Delta_{2}$ are unlinked, i.e., that $Z(\Delta_{1}+\Delta_{2})=Z(\Delta_{1})\times Z(\Delta_{2})$. Then by Theorem 3.7 part 3, $Z(\Delta_{1}+\Delta_{2})\rtimes\sigma$ is irreducible if and only if $\Delta_{1}$ and $\widetilde{\Delta_{2}}$ are unlinked. (Of course, for the “only if” direction we do not need the assumption $\Delta_{i}\cap S_{\sigma}=\emptyset$.)

Thus, for the rest of the proof we may assume that $\Delta_{1}$ and $\Delta_{2}$ are linked, and without loss of generality that $\Delta_{2}\prec\Delta_{1}$. In this case, we split Theorem 5.1 to the following two assertions which will be proved below.

Lemma 5.3**.**

Suppose that $\Delta_{2}\prec\Delta_{1}$ and $\Delta_{i}\cap S_{\sigma}=\emptyset$, $i=1,2$. Suppose further that $\Delta_{1}$ and $\widetilde{\Delta_{2}}$ are unlinked or that $\mathfrak{c}(\Delta_{1})\cdot\mathfrak{c}(\Delta_{2})\geq 0$. Then $Z(\Delta_{1}+\Delta_{2})\rtimes\sigma$ is irreducible.

Lemma 5.4**.**

Suppose that $\Delta_{2}\prec\Delta_{1}$, $\Delta_{1}$ and $\widetilde{\Delta_{2}}$ are linked and $\mathfrak{c}(\Delta_{1})>0>\mathfrak{c}(\Delta_{2})$. Then $Z(\Delta_{1}+\Delta_{2})\rtimes\sigma$ is reducible.

5.1. Auxiliary results

Denote by $\operatorname{Irr}_{\operatorname{gen}}\operatorname{GL}\subset\operatorname{Irr}\operatorname{GL}$ the subset of irreducible generic representations. In terms of the Zelevinsky classifications, these are the irreducible representations corresponding to multisegments consisting of singleton segments, i.e.,

[TABLE]

In terms of the Langlands classification, $\operatorname{Irr}_{\operatorname{gen}}\operatorname{GL}$ corresponds to the multisegments consisting of pairwise unlinked segments (i.e., such that $\mathfrak{l}(\mathfrak{m})=L(\mathfrak{m})$). Dually, we say that $\pi\in\operatorname{Irr}\operatorname{GL}$ is cogeneric if $\pi=\mathfrak{z}(\mathfrak{m})$ for some $\mathfrak{m}\in\mathbb{N}({\mathcal{SEG}})$.111111The cogeneric representations with a non-trivial vector fixed under the Iwahori subgroup are exactly the unramified representations. Denote by $\operatorname{Irr}_{\operatorname{cogen}}\operatorname{GL}$ the set of irreducible cogeneric representations. The sets $\operatorname{Irr}_{\operatorname{gen}}\operatorname{GL}$ and $\operatorname{Irr}_{\operatorname{cogen}}\operatorname{GL}$ correspond under the Zelevinsky involution.

The following is well known.

Lemma 5.5**.**

Let $\pi_{1},\pi_{2}\in\operatorname{Irr}\operatorname{GL}$. Then $\pi\leq\pi_{1}\times\pi_{2}$ for some $\pi\in\operatorname{Irr}_{\operatorname{gen}}\operatorname{GL}$ if and only if $\pi_{1},\pi_{2}\in\operatorname{Irr}_{\operatorname{gen}}\operatorname{GL}$. In this case, $\pi$ is uniquely determined by $\pi_{1}$ and $\pi_{2}$ and it occurs with multiplicity one in $\operatorname{JH}(\pi_{1}\times\pi_{2})$. An analogous statement holds for $\operatorname{Irr}_{\operatorname{cogen}}\operatorname{GL}$ (by applying the Zelevinsky involution).

Lemma 5.6**.**

For any $\mathfrak{m}\in\mathbb{N}({\mathcal{SEG}})$ there exists $\pi\in\operatorname{Irr}_{\operatorname{cogen}}\operatorname{GL}$ such that $\mathfrak{c}(\rho)\leq 0$ for all $\rho\in\operatorname{supp}\pi$ and

[TABLE]

Consequently,

[TABLE]

(In fact, this holds for any $\sigma\in\mathcal{C}^{G}$.)

Proof.

Since $M^{*}_{\max}$ is a homomorphism, it is enough by Lemma 5.5 to consider the case $\mathfrak{m}=\Delta\in\mathcal{SEG}$. This case follows from the formula (3.8). Indeed, if $\Delta=\{\rho_{1},\dots,\rho_{l}\}$ with $\rho_{i+1}=\overset{\rightarrow}{\rho_{i}}$, $i=1,\dots,l-1$ then we take $\rho=\rho_{i}$ where $i$ is the largest index such that $\mathfrak{c}(\rho_{i})\leq 0$ if $\mathfrak{c}(\rho_{1})\leq 0$ and $\rho=\overset{\leftarrow}{\rho_{1}}$ otherwise. ∎

Lemma 5.7**.**

Let $\Delta_{1},\Delta_{2}\in\mathcal{SEG}$ be such that $\Delta_{2}\prec\Delta_{1}$ and $\mathfrak{c}(\Delta_{2})\geq 0$. Suppose that

[TABLE]

for some $\pi\in\operatorname{Irr}_{\operatorname{cogen}}\operatorname{GL}$. Then there exists $\rho\in\operatorname{supp}\pi$ such that $\mathfrak{c}(\rho)>0$.

Proof.

By (3.3) and the supercuspidality of $\sigma$ we have $\pi\leq M^{*}_{\max}(Z(\Delta_{1}+\Delta_{2}))$. By (3.10) there exist $\rho_{1}\in\tensor*[^{+}]\Delta{}_{1}$ and $\rho_{2}\in\tensor*[^{+}]\Delta{}_{2}$ with $\rho_{2}<\rho_{1}$ such that

[TABLE]

where $\mathfrak{m}_{1}=[b(\Delta_{1}),\rho_{1}]+[b(\Delta_{2}),\rho_{2}]$ and $\mathfrak{m}_{2}=[\overset{\rightarrow}{\rho_{1}},e(\Delta_{1})]+[\overset{\rightarrow}{\rho_{2}},e(\Delta_{2})]$. Thus, by Lemma 5.5 $Z(\mathfrak{m}_{2})$ is cogeneric and therefore $\rho_{1}>e(\Delta_{2})$. Hence, $e(\overset{\rightarrow}{\Delta_{2}})\in[b(\Delta_{1}),\rho_{1}]\subset\operatorname{supp}\pi$ (since $\Delta_{2}\prec\Delta_{1}$) while $\mathfrak{c}(e(\Delta_{2}))\geq\mathfrak{c}(\Delta_{2})\geq 0$. ∎

5.2. Proof of Lemma 5.3

We show that

[TABLE]

and

[TABLE]

We first consider the case where $\mathfrak{c}(\Delta_{1})>0>\mathfrak{c}(\Delta_{2})$ and $\Delta_{1}$, $\widetilde{\Delta_{2}}$ are unlinked. In this case,

[TABLE]

while

[TABLE]

as required.

It remains to consider the case $\mathfrak{c}(\Delta_{1})\cdot\mathfrak{c}(\Delta_{2})\geq 0$. Without loss of generality we may assume that $\mathfrak{c}(\Delta_{1})>\mathfrak{c}(\Delta_{2})\geq 0$ – otherwise replace $(\Delta_{1},\Delta_{2})$ by $(\widetilde{\Delta_{2}},\widetilde{\Delta_{1}})$. We assume it for the rest of the proof. Then $\mathfrak{z}(\Delta_{1}+\Delta_{2};\sigma)=Z(\Delta_{1})\times Z(\Delta_{2})\rtimes\sigma$ and (5.1) is clear. It remains to prove (5.2). We have

[TABLE]

As before, in the case where $\Delta_{1}$, $\widetilde{\Delta_{2}}$ are unlinked we have

[TABLE]

Thus, we may assume that $\Delta_{1}$, $\widetilde{\Delta_{2}}$ are linked, in which case necessarily $\widetilde{\Delta_{2}}\prec\Delta_{1}$ (or equivalently, $\widetilde{\Delta_{1}}\prec\Delta_{2}$) since $\mathfrak{c}(\Delta_{1})>0\geq\mathfrak{c}(\widetilde{\Delta_{2}})$. We thus have an exact sequence

[TABLE]

where $\Delta_{1}^{\prime}=\Delta_{2}^{\vee}\cup\Delta_{1}^{\varrho}$ and $\Delta_{2}^{\prime}=\Delta_{2}^{\vee}\cap\Delta_{1}^{\varrho}\subset\Delta_{1}^{\prime}$. We claim that $\Pi$ is irreducible. Indeed, $\widetilde{\Delta_{1}^{\prime}}=\Delta_{2}^{\varrho}\cup\Delta_{1}^{\vee}=[e(\Delta_{1})^{\vee},e(\Delta_{2}^{\varrho})]$ and $\Delta_{2}^{\prime}=[b(\Delta_{1}^{\varrho}),b(\Delta_{2})^{\vee}]$ since $\widetilde{\Delta_{2}}\prec\Delta_{1}$. Thus, $\widetilde{\Delta_{1}^{\prime}}\supset\Delta_{2}^{\prime}$ (since $\mathfrak{c}(\Delta_{1}),\mathfrak{c}(\Delta_{2})\geq 0$) and in particular $\widetilde{\Delta_{1}^{\prime}}$ and $\Delta_{2}^{\prime}$ are unlinked. The irreducibility of $\Pi$ follows from Theorem 3.7 part 3.

By Lemmas 5.6 and 5.7 we have

[TABLE]

and therefore

[TABLE]

Hence, it follows from (5.3) and (5.4) that

[TABLE]

as required.

5.3. Proof of Lemma 5.4

By Theorem 4.1 we may assume that $\Delta_{i}\cap S_{\sigma}=\emptyset$, $i=1,2$. Upon replacing $(\Delta_{1},\Delta_{2})$ by $(\widetilde{\Delta_{2}},\widetilde{\Delta_{1}})$ if necessary, we may assume without loss of generality that $\Delta_{1}\prec\widetilde{\Delta_{2}}$. Then

[TABLE]

Since $\mathfrak{c}(\Delta_{1})>0$ and $\Delta_{2}\prec\Delta_{1}$, it is easy to see that

[TABLE]

Hence the irreducible representation

[TABLE]

occurs with multiplicity one in $\mu^{*}(\Pi)$. Thus, $\Pi$ is SI by Lemma 3.1 and therefore $Z(\Delta_{1}+\Delta_{2})\rtimes\sigma$ is SI and $\operatorname{soc}(Z(\Delta_{1}+\Delta_{2})\rtimes\sigma)=\operatorname{soc}(\Pi)$. On the other hand,

[TABLE]

where we write $\Delta^{\prime}_{1}=\Delta_{1}\cup\widetilde{\Delta_{2}}$, $\Delta^{\prime}_{2}=\widetilde{\Delta_{1}}\cap\Delta_{2}$. Note that $\Delta^{\prime}_{2}\prec\Delta^{\prime}_{1}$ since $\mathfrak{c}(\Delta_{1})>0>\mathfrak{c}(\Delta_{2})$ and $\Delta_{2}\prec\Delta_{1}$. Thus,

[TABLE]

(In fact, $Z(\Delta^{\prime}_{1}+\Delta^{\prime}_{2})\rtimes\sigma$ is irreducible by Lemma 5.3 since $\Delta^{\prime}_{1}$ contains (and in particular, is unlinked with) $\widetilde{\Delta_{2}^{\prime}}$. However, we will not need to use this fact.)

Suppose on the contrary that $Z(\Delta_{1}+\Delta_{2})\rtimes\sigma$ is irreducible. Then

[TABLE]

We will show that this is impossible, arriving at a contradiction.

Note that $\Delta^{\prime}_{1}=[b(\Delta_{1}),e(\widetilde{\Delta_{2}})]$, $\Delta^{\prime}_{2}=[b(\widetilde{\Delta_{1}}),e(\Delta_{2})]$ and $\Delta^{\prime}_{1}\cap\Delta^{\prime}_{2}=[b(\Delta_{1}),e(\Delta_{2})]=\Delta_{1}\cap\Delta_{2}$. Let

[TABLE]

Finally, denote by $J^{\operatorname{GL}}_{\min}$ the Jacquet functor with respect to the standard parabolic subgroup of type $(d,\dots,d)$ ($N$ times). Then

[TABLE]

Note that $l_{1},l_{2}<l_{1}^{\prime}$ (since $\Delta_{1}^{\prime}\supsetneq\Delta_{1},\widetilde{\Delta_{2}}$) and therefore

[TABLE]

Denoting by $J^{G}_{\min}$ the Jacquet functor with respect to the standard parabolic subgroup with Levi part $\overbrace{\operatorname{GL}_{d}\times\dots\times\operatorname{GL}_{d}}^{N}\times G_{\deg\sigma}$. It is easy to see by the geometric lemma that $\ell(J^{G}_{\min}(\pi\rtimes\sigma))=2^{N}\ell(J^{\operatorname{GL}}_{\min}(\pi))$ for any $\pi\in\mathcal{C}(\operatorname{GL}_{dN})_{\mathbb{N}({\operatorname{Cusp}\operatorname{GL}_{d}})}$. Thus,

[TABLE]

in contradiction to (5.5). Lemma 5.4 follows.

This concludes the proof of Theorem 5.1.

6. An irreducibility criterion

6.1.

In this subsection we reduce the question of irreducibility of $\pi\rtimes\sigma$ where $\pi\in\operatorname{Irr}$ and $\sigma\in\operatorname{Cusp}^{G}$ to the case where $\operatorname{supp}\pi\subset\rho[\mathbb{Z}]$ for some $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$ with $\tilde{\rho}\subset\rho[\mathbb{Z}]$, assuming knowledge of irreducibility of parabolic induction for the general linear group. For convenience and as a preparation for the subsequent subsections, we give a self-contained argument although some of the results are available in greater generality in the literature.

Lemma 6.1**.**

Let $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$ with $\tilde{\rho}\notin\rho[\mathbb{Z}]$ and let $\pi\in\operatorname{Irr}\operatorname{GL}$ with $\operatorname{supp}\pi\subset\rho[\mathbb{Z}]$. Assume that $\pi$ is Z-positive. Then $\pi\rtimes\sigma$ is irreducible for any $\sigma\in\operatorname{Cusp}^{G}$.

Proof.

We prove it by induction on $\deg\pi$. The case $\pi=\mathbf{1}$ is trivial. For the induction step write $\pi=Z(\mathfrak{m})$ and $\mathfrak{m}=\Delta+\mathfrak{m}^{\prime}$ where $\mathfrak{c}(\Delta)\geq\mathfrak{c}(\Delta^{\prime})$ for any $\Delta^{\prime}\leq\mathfrak{m}^{\prime}$. Since $\mathfrak{m}$ is positive, $\pi\rtimes\sigma\hookrightarrow\mathfrak{z}(\mathfrak{m};\sigma)$. On the other hand, by the assumption on $\rho$ and $\pi$, $Z(\Delta^{\vee})\rtimes\sigma^{\vee}$ and $Z(\mathfrak{m}^{\prime\vee})\times Z(\Delta^{\varrho})$ are irreducible. Thus, by the induction hypothesis

[TABLE]

and the lemma follows from Corollary 3.13. ∎

Lemma 6.2**.**

Let $\rho\in\operatorname{Cusp}^{\operatorname{GL}}$ with $\tilde{\rho}\notin\rho[\mathbb{Z}]$ and let $\pi\in\operatorname{Irr}\operatorname{GL}$ with $\operatorname{supp}\pi\subset\rho[\mathbb{Z}]$. Then $\pi\rtimes\sigma$ is irreducible for any $\sigma\in\operatorname{Cusp}^{G}$.

Proof.

Write $\pi=Z(\mathfrak{m})$. Clearly, $Z(\mathfrak{m}_{=0})\rtimes\sigma$ is irreducible. The condition on $\rho$ implies that $\pi_{1}\times\pi_{2}$ is irreducible for any $\pi_{1},\pi_{2}\in\operatorname{Irr}\operatorname{GL}$ such that $\operatorname{supp}\pi_{1}\subset\rho[\mathbb{Z}]$ and $\operatorname{supp}\pi_{2}\subset\tilde{\rho}[\mathbb{Z}]$. In particular, $\pi_{+}\times\tilde{\pi}_{+}$ and $\tilde{\pi}_{+}\times Z(\mathfrak{m}_{=0})$ are irreducible. (Recall that by our convention $\tilde{\pi}_{+}$ means $(\tilde{\pi})_{+}$.) Thus, by Lemma 6.1

[TABLE]

In a similar vein,

[TABLE]

and the lemma follows from Corollary 3.13. ∎

Remark 6.3*.*

Lemma 6.2 is also a consequence of [Tad09, Proposition 3.2].

Lemma 6.4**.**

Let $\pi_{i}\in\operatorname{Irr}\operatorname{GL}$, $i=1,2$ and $\sigma\in\operatorname{Irr}_{\operatorname{cotemp}}G$. Assume that for any $\rho_{i}\in\operatorname{supp}\pi_{i}$, $i=1,2$ we have $\rho_{1},\tilde{\rho}_{1}\notin\rho_{2}[\mathbb{Z}]$. Then $\pi_{1}\times\pi_{2}\rtimes\sigma$ is irreducible if and only if $\pi_{i}\rtimes\sigma$ is irreducible, $i=1,2$.

Proof.

The condition is clearly necessary (for any $\sigma\in\operatorname{Irr}G$). Suppose that it is satisfied. Write $\pi_{i}=Z(\mathfrak{m}_{i})$, $i=1,2$ so that $\pi=Z(\mathfrak{m})$ where $\mathfrak{m}=\mathfrak{m}_{1}+\mathfrak{m}_{2}$. Then

[TABLE]

Similarly,

[TABLE]

and the lemma follows from Corollary 3.13. ∎

Remark 6.5*.*

In fact Lemma 6.4 holds for any $\sigma\in\operatorname{Irr}G$. This easily follows from [Jan97, Theorem 10.5].

Recall that any $\pi\in\operatorname{Irr}\operatorname{GL}$ can be written uniquely (up to permutation) as

[TABLE]

where for all $i$, $\pi_{i}\neq\mathbf{1}$ and there exists $\rho_{i}\in\operatorname{Cusp}^{\operatorname{GL}}$ such that $\operatorname{supp}\pi_{i}\subset\rho_{i}[\mathbb{Z}]$ and $\rho_{j}\notin\rho_{i}[\mathbb{Z}]$ for all $i\neq j$.

Proposition 6.6**.**

Let $\pi\in\operatorname{Irr}\operatorname{GL}$ and $\sigma\in\operatorname{Cusp}^{G}$. Write $\pi=\pi_{1}\times\dots\times\pi_{k}$ as in (6.1). Then $\pi\rtimes\sigma$ is irreducible if and only if $\pi_{i}\rtimes\sigma$ is irreducible for all $i$ such that $\tilde{\rho}_{i}\in\rho_{i}[\mathbb{Z}]$ and $\pi_{i}\times\tilde{\pi}_{j}$ is irreducible for all $i\neq j$ such that $\tilde{\rho}_{j}\in\rho_{i}[\mathbb{Z}]$.

Proof.

By Lemma 6.4 we reduces the statement to the case where $k=2$ and $\tilde{\rho}_{2}\in\rho_{1}[\mathbb{Z}]$. This case follows from Lemma 6.2 since the irreducibility of $\pi\rtimes\sigma=\pi_{1}\times\pi_{2}\rtimes\sigma$ is equivalent to the irreducibility of $\pi_{1}\times\tilde{\pi}_{2}\rtimes\sigma$. ∎

6.2.

Lemma 6.7**.**

Assume that $\theta\in\operatorname{Irr}_{\operatorname{cotemp}}G$ and $\pi\leq\mathfrak{z}(\mathfrak{m})$ where $\mathfrak{m}$ is a positive multisegment. Then

[TABLE]

Consequently, if $\pi$ is SI then $\pi\rtimes\theta$ is SI and $\operatorname{soc}(\pi\rtimes\theta)=\operatorname{soc}(\operatorname{soc}(\pi)\rtimes\theta)$.

Proof.

Define

[TABLE]

by

[TABLE]

Clearly $M^{*}_{\operatorname{ex}}(\tau)\geq 0$ for all $\tau\in\mathcal{C}^{\operatorname{GL}}$. We claim that for any irreducible $\alpha\otimes\beta\leq M^{*}_{\operatorname{ex}}(\pi)$ we have $\mathfrak{c}(\alpha)<\mathfrak{c}(\pi)$. In fact, this is true for any irreducible $\alpha\otimes\beta\leq M^{*}_{\operatorname{ex}}(\mathfrak{z}(\mathfrak{m}))$. Indeed, by the multiplicativity of $M^{*}$ and the positivity of $\mathfrak{m}$, it is enough to consider the case where $\mathfrak{m}=\Delta$ with $\mathfrak{c}(\Delta)>0$. We use formula (3.7). For any $\Delta\leq\mathfrak{m}$ and $\rho,\rho^{\prime}\in\tensor*[^{+}]\Delta{}$ with $b(\Delta)\neq\rho\leq\rho^{\prime}$ we have

[TABLE]

Hence

[TABLE]

Our claim follows.

On the other hand, since $\theta$ is cotempered, for any irreducible $\alpha\otimes\beta\leq\mu^{*}(\theta)$ we have $\mathfrak{c}(\alpha)\leq 0$. Since

[TABLE]

it follows that for any irreducible $\alpha\otimes\beta\leq\mu^{*}(\pi\rtimes\theta)-\pi\otimes\theta$ we have $\mathfrak{c}(\alpha)<\mathfrak{c}(\pi)$. This clearly implies the first part of the lemma. The second part follows from lemma 3.1. ∎

Lemma 6.8**.**

Suppose that $\pi_{1},\pi_{2}\in\operatorname{Irr}\operatorname{GL}$ are Z-positive and $\pi\in\operatorname{Irr}\operatorname{GL}$ is such that $\pi\leq\pi_{1}\times\pi_{2}$. Then $\pi$ is Z-positive.

Proof.

Let $\pi_{i}=Z(\mathfrak{m}_{i})$, $i=1,2$ where $\mathfrak{m}_{1}$ and $\mathfrak{m}_{2}$ are positive multisegments. Then $\pi=Z(\mathfrak{m})$ where $\mathfrak{m}$ is smaller than or equal to $\mathfrak{m}_{1}+\mathfrak{m}_{2}$ with respect to the partial order on multisegments introduced by Zelevinsky in [Zel80, §7]. It remains to observe that if $\mathfrak{n}$ is positive then the same is true for any multisegments less than or equal to it. This follows immediately from (2.6) and the definition of the partial order on multisegments. ∎

Lemma 6.9**.**

Let $\mathfrak{m}$ be a multisegment, $\pi=Z(\mathfrak{m})$ and $\sigma\in\operatorname{Irr}_{\operatorname{cotemp}}G$. Suppose that $Z(\mathfrak{m}_{\leq 0})\rtimes\sigma$ is irreducible and $\pi_{+}\times\tilde{\pi}_{+}$ is SI. Write $\operatorname{soc}(\pi_{+}\times\tilde{\pi}_{+})=Z(\mathfrak{n})$. Then $\pi\rtimes\sigma$ is SI and $\operatorname{soc}(\pi\rtimes\sigma)=Z(\mathfrak{n};Z(\mathfrak{m}_{=0})\rtimes\sigma)$.

Proof.

By Corollary 3.9 and our assumption, $Z(\mathfrak{m}_{=0})\rtimes\sigma$ is irreducible, and hence $Z(\mathfrak{m}_{=0})\rtimes\sigma=Z(\tilde{\mathfrak{m}}_{=0})\rtimes\sigma$. We have

[TABLE]

By Lemma 6.7 we infer that $\pi\rtimes\sigma$ is SI and

[TABLE]

(Note that $\mathfrak{n}$ is positive by Lemma 6.8.) The proposition follows. ∎

Lemma 6.10**.**

(cf. [MW86, p. 173]) Let $\pi_{1},\pi_{2}\in\operatorname{Irr}\operatorname{GL}$. Suppose that at least one of $\pi_{1}\times\pi_{2}$ or $\pi_{2}\times\pi_{1}$ is SI. Then the following conditions are equivalent.

1. (1)

$\pi_{1}\times\pi_{2}$* is irreducible.* 2. (2)

$\pi_{1}\times\pi_{2}\simeq\pi_{2}\times\pi_{1}$. 3. (3)

$\operatorname{soc}(\pi_{1}\times\pi_{2})\simeq\operatorname{soc}(\pi_{2}\times\pi_{1})$.

Proof.

Clearly, 1$\implies$2$\implies$3. Let $\Pi=\pi_{1}\times\pi_{2}$. Interchanging $\pi_{1}$ and $\pi_{2}$ if necessary we may assume that $\pi_{2}\times\pi_{1}$ is SI. Applying the functor $\iota$ and using (3.1) we deduce that

[TABLE]

is SI and

[TABLE]

Therefore, condition 3 is equivalent to $\operatorname{soc}(\Pi)^{\vee}\simeq\operatorname{soc}(\Pi^{\vee})$ which in turn is equivalent to the irreducibility of $\Pi$ by (1.2). ∎

Corollary 6.11**.**

Let $\mathfrak{m}$ be a multisegment, $\pi=Z(\mathfrak{m})$ and $\sigma\in\operatorname{Irr}_{\operatorname{cotemp}}G$. Suppose that $Z(\mathfrak{m}_{\geq 0})\rtimes\sigma$ and $Z(\mathfrak{m}_{\leq 0})\rtimes\sigma$ are irreducible and that both $\pi_{+}\times\tilde{\pi}_{+}$ and $\tilde{\pi}_{+}\times\pi_{+}$ are SI. Then the following conditions are equivalent.

1. (1)

$\pi\rtimes\sigma$* is irreducible.* 2. (2)

$\pi\rtimes\sigma\simeq\tilde{\pi}\rtimes\sigma$. 3. (3)

$\operatorname{soc}(\pi\rtimes\sigma)\simeq\operatorname{soc}(\tilde{\pi}\rtimes\sigma)$. 4. (4)

$\pi_{+}\times\tilde{\pi}_{+}$* is irreducible.* 5. (5)

$\pi_{+}\times\tilde{\pi}_{+}\simeq\tilde{\pi}_{+}\times\pi_{+}$. 6. (6)

$\operatorname{soc}(\pi_{+}\times\tilde{\pi}_{+})\simeq\operatorname{soc}(\tilde{\pi}_{+}\times\pi_{+})$.

Proof.

By Lemma 6.10 the conditions 4, 5 and 6 are equivalent. Clearly, 1$\implies$2$\implies$3. Let $\Pi=\pi\rtimes\sigma$. By Lemma 6.9 (applied to both $\mathfrak{m}$ and $\tilde{\mathfrak{m}}$) and the uniqueness part of Theorem 3.7, the conditions 3 and 6 are equivalent. Similarly, applying Lemma 6.9 to both $\mathfrak{m}$ and $\mathfrak{m}^{\vee}$ (with $\sigma$ and $\sigma^{\vee}$ respectively) we deduce that $\Pi$ and $\Pi^{\vee}$ are SI and that the condition

[TABLE]

is equivalent to condition 6 in view of Theorem 3.7 part 4. It remains to apply (1.2). ∎

6.3.

From now on we fix $\sigma\in\operatorname{Cusp}^{G}$.

Lemma 6.12**.**

Suppose that $\mathfrak{m}$ is a ladder as in (2.9) with $k>1$ and $\mathfrak{c}(\Delta_{k})\geq 0$. Assume that $\widetilde{\Delta_{1}}\prec\Delta_{k}$. Then

[TABLE]

Hence,

[TABLE]

Proof.

The second statement follows from the first one since $\sigma$ is supercuspidal. To prove the first part, let

[TABLE]

Since $k>1$ it suffices to show that $M^{*}_{\{\mathfrak{d}\};*}(Z(\mathfrak{m}))$ is equal to

[TABLE]

if $\Delta_{i}\prec\Delta_{i-1}$ for all $i<k$ and $\widetilde{\Delta_{k}}\neq\Delta_{k}$ and $M^{*}_{\{\mathfrak{d}\};*}(Z(\mathfrak{m}))=0$ otherwise. Consider a term $\tau_{1}\otimes\tau_{2}$ on the right-hand side of (3.9) where

[TABLE]

with $\rho_{i},\rho_{i}^{\prime}\in\tensor*[^{+}]\Delta{}_{i}$, $\rho_{i}^{\prime}\geq\rho_{i}$ for all $i$, $\rho_{1}>\dots>\rho_{k}$, and $\rho^{\prime}_{1}>\dots>\rho^{\prime}_{k}$. If $\mathfrak{d}_{\tau_{1}}=\mathfrak{d}$ then we would necessarily have $\rho_{i}=b(\overset{\leftarrow}{\Delta_{i}})$ for all $i$, $\Delta_{i+1}\prec\Delta_{i}$, $\rho^{\prime}_{i}=e(\Delta_{i+1})$, $i=1,\dots,k-1$, $\rho^{\prime}_{k}=e(\widetilde{\Delta_{k}})<e(\Delta_{k})$. Our assertion follows. ∎

Lemma 6.13**.**

Suppose that $\mathfrak{m}$ is a ladder as in (2.9) with $\mathfrak{c}(\Delta_{k})\geq 0$. Assume that $\operatorname{supp}\mathfrak{m}\cap S_{\sigma}=\emptyset$. Then $Z(\mathfrak{m})\rtimes\sigma$ is irreducible.

Proof.

We prove this by induction on $k$. The case $k=1$ is Theorem 3.10. Assume that $k>1$ and that the statement holds for $k-1$. Clearly,

[TABLE]

By Corollary 3.13 it suffices to show that

[TABLE]

Write $\mathfrak{m}^{\prime}=\Delta_{2}+\dots+\Delta_{k}$ so that $\mathfrak{m}=\mathfrak{m}^{\prime}+\Delta_{1}$. We have

[TABLE]

If $\widetilde{\Delta_{k}}\not\prec\Delta_{1}$, or equivalently, $\Delta_{k}^{\vee}\not\prec\Delta_{1}^{\varrho}$ (in which case $\Delta_{i}^{\vee}\not\prec\Delta_{1}^{\varrho}$ for all $i>1$) then

[TABLE]

which by the induction hypothesis is isomorphic to

[TABLE]

as required.

Assume now that $\widetilde{\Delta_{k}}\prec\Delta_{1}$ and let $\Gamma=\Delta_{1}\cup\widetilde{\Delta_{k}}$, $\gamma=\Delta_{1}\cap\widetilde{\Delta_{k}}$, $\mathfrak{m}^{\prime\prime}=\Delta_{2}+\dots+\Delta_{k-1}$ and $\mathfrak{n}=\mathfrak{m}^{\prime\prime\vee}+\Gamma^{\varrho}+\gamma^{\varrho}$. Then by Corollary 2.8 we have an exact sequence

[TABLE]

We first show that $\Pi$ is irreducible. Note that $\Gamma=[b(\widetilde{\Delta_{k}}),e(\Delta_{1})]$ and $\gamma=[b(\Delta_{1}),e(\widetilde{\Delta_{k}})]$. Hence, $\mathfrak{c}(\Gamma),\mathfrak{c}(\gamma)\geq 0$, $\Gamma\supset\tilde{\gamma}$ and $\Gamma\supset\Delta_{i}\supset\gamma$ for all $1<i<k$. Thus,

[TABLE]

and therefore

[TABLE]

On the other hand,

[TABLE]

which by the induction hypothesis is isomorphic to

[TABLE]

Thus $\Pi$ is irreducible by Corollary 3.13.

By Corollary 2.8 and Lemma 6.12, respectively, we have

[TABLE]

while

[TABLE]

Hence,

[TABLE]

Together with (6.2) and (6.3) we conclude that

[TABLE]

which by the induction hypothesis is isomorphic to

[TABLE]

as required. ∎

Remark 6.14*.*

Lemma 6.13 does not hold for a positive multisegment in general. For instance, we can take $\mathfrak{m}=\rho+\rho$ where $\tilde{\rho}=\overset{\leftarrow}{\rho}$. (Such $\rho\notin S_{\sigma}$ always exists.) Clearly, $Z(\mathfrak{m})\rtimes\sigma$ is reducible since $\rho\times\tilde{\rho}\rtimes\sigma$ is reducible.

Using Lemma 6.13 and [LM16, Proposition 6.15] we can infer from Lemma 6.9 and Corollary 6.11 the following.

Corollary 6.15**.**

Suppose that $\mathfrak{m}$ is a ladder and $\operatorname{supp}\mathfrak{m}\cap S_{\sigma}=\emptyset$. Let $\pi=Z(\mathfrak{m})$. Then

1. (1)

$\pi\rtimes\sigma$* is SI.* 2. (2)

If $\operatorname{soc}(\pi_{+}\times\tilde{\pi}_{+})=Z(\mathfrak{n})$ then $\operatorname{soc}(\pi\rtimes\sigma)=Z(\mathfrak{n};Z(\mathfrak{m}_{=0})\rtimes\sigma)$. 3. (3)

The conclusion of Corollary 6.11 holds.

Remark 6.16*.*

Suppose that $\mathfrak{m}=\Delta_{1}+\dots+\Delta_{t}$ and $\mathfrak{m}^{\prime}=\Delta_{1}^{\prime}+\dots+\Delta_{t^{\prime}}^{\prime}$ are two ladders and let $\pi=Z(\mathfrak{m})$, $\pi^{\prime}=Z(\mathfrak{m}^{\prime})$. Then there is a simple combinatorial procedure to determine the multisegment corresponding to $\operatorname{soc}(\pi\times\pi^{\prime})$ [LM16, Corollary 6.16]. In particular ([LM16, Proposition 6.20 and Lemma 6.21]), $\operatorname{soc}(\pi\times\pi^{\prime})\not\simeq Z(\mathfrak{m}+\mathfrak{m}^{\prime})$ if and only if there exist integers $i,j\geq 0$ and $\ell\geq 1$ satisfying $i+\ell\leq t,j+\ell\leq t^{\prime}$, such that

1. (1)

$\Delta_{i+1}\prec\Delta^{\prime}_{j+1},\Delta_{i+2}\prec\Delta^{\prime}_{j+2},\dots,\Delta_{i+\ell}\prec\Delta^{\prime}_{j+\ell}.$ 2. (2)

Either $i=0$ or ${\overset{\leftarrow}{\Delta}}_{i}\not\prec\Delta^{\prime}_{j+1}$. 3. (3)

Either $j+\ell=t^{\prime}$ or ${\overset{\leftarrow}{\Delta}}_{i+\ell}\not\prec\Delta^{\prime}_{j+\ell+1}$.

Recall that $\pi\times\pi^{\prime}$ is irreducible if and only if $\operatorname{soc}(\pi\times\pi^{\prime})\simeq Z(\mathfrak{m}+\mathfrak{m}^{\prime})\simeq\operatorname{soc}(\pi^{\prime}\times\pi)$.

6.4.

In this subsection we fix $\alpha\in S_{\sigma}$ with $\mathfrak{c}(\alpha)\geq 0$. We will consider $\pi=Z(\mathfrak{m})$ such that

[TABLE]

We can uniquely write $\mathfrak{m}=\mathfrak{m}_{>\alpha}+\mathfrak{m}_{[\tilde{\alpha},\alpha]}+\mathfrak{m}_{<\alpha}$ where

[TABLE]

Correspondingly,

[TABLE]

Clearly

[TABLE]

Lemma 6.17**.**

Suppose that $\pi\in\operatorname{Irr}\operatorname{GL}$ satisfies (6.4) and in addition, $\mathfrak{m}=\mathfrak{m}_{>\alpha}$ (i.e., $\rho>\alpha$ for any $\rho\in\operatorname{supp}\pi$). Then $Z(\mathfrak{m})\rtimes\sigma$ is irreducible.

Proof.

Clearly $Z(\mathfrak{m})\rtimes\sigma\hookrightarrow\mathfrak{z}(\mathfrak{m};\sigma)$. We prove that $Z(\mathfrak{m}^{\vee})\rtimes\sigma^{\vee}\hookrightarrow\mathfrak{z}(\mathfrak{m}^{\varrho};\sigma^{\vee})$ by induction on $\deg\mathfrak{m}$. For the induction step, write $\mathfrak{m}=\mathfrak{m}^{\prime}+\Delta$ where $\mathfrak{c}(\Delta)\leq\mathfrak{c}(\Delta^{\prime})$ for any $\Delta^{\prime}\leq\mathfrak{m}^{\prime}$. Then

[TABLE]

which by induction hypothesis is

[TABLE]

Now $Z(\Delta^{\vee})\times Z(\mathfrak{m}^{\prime\varrho})$ is irreducible since $Z(\Delta^{\vee})\times Z(\Delta^{\prime\varrho})$ is irreducible for any $\Delta^{\prime}\leq\mathfrak{m}$ by the condition on $\mathfrak{m}$. Therefore

[TABLE]

as required. Thus, the lemma follows from Corollary 3.13. ∎

Lemma 6.18**.**

Suppose that $\pi=Z(\mathfrak{m})\in\operatorname{Irr}\operatorname{GL}$ satisfies (6.4). Then $\pi\rtimes\sigma$ is irreducible if and only if $\pi_{[\tilde{\alpha},\alpha]}\rtimes\sigma$ and $\pi_{>\alpha}\times\tilde{\pi}_{>\alpha}$ are irreducible.

Proof.

The “only if” part follows from (6.5) and (3.2). On the other hand, by (6.6) and Lemma 6.17 we have

[TABLE]

Thus, if $\pi_{[\tilde{\alpha},\alpha]}\rtimes\sigma$ and $\pi_{>\alpha}\times\tilde{\pi}_{>\alpha}$ are irreducible we get

[TABLE]

Similarly, we have

[TABLE]

Thus, $\pi\rtimes\sigma$ is irreducible by Corollary 3.13. ∎

Corollary 6.19**.**

Suppose that $\alpha\in\{\tilde{\alpha},\tilde{\alpha}[1],\tilde{\alpha}[2]\}$ and $\pi\in\operatorname{Irr}\operatorname{GL}$ satisfies (6.4). Then $\pi\rtimes\sigma$ is irreducible if and only if $\pi_{+}\times\tilde{\pi}_{+}$ is irreducible.

Indeed, note that in this case $\pi_{+}=\pi_{>\alpha}$ (and similarly for $\tilde{\pi}$). Moreover, $\pi_{[\tilde{\alpha},\alpha]}\neq\mathbf{1}$ only if $\beta:=\overset{\leftarrow}{\alpha}$ satisfies $\tilde{\beta}=\beta$, in which case $\pi_{[\tilde{\alpha},\alpha]}=\beta^{\times l}$ for some $l>0$.

Remark 6.20*.*

By the results of Shahidi [Sha90], the assumption on $\alpha$ is always satisfied if $G$ is quasi-split and $\sigma$ is generic. For an arbitrary $\sigma\in\operatorname{Cusp}^{G}$, we have $\alpha\in\{\tilde{\alpha},\tilde{\alpha}[\pm 1]\}$ for all but finitely many $\alpha\in S_{\sigma}$.

Remark 6.21*.*

Suppose that $\alpha=\tilde{\alpha}[m]$ with $m>2$. Let $\Delta=[\tilde{\alpha}[2],\overset{\leftarrow}{\alpha}]\in\mathcal{SEG}_{>0}$, so that $\tilde{\Delta}=\overset{\leftarrow}{\Delta}\prec\Delta$. Then $Z(\Delta+\Delta)\rtimes\sigma$ is reducible, since $Z(\Delta)\times Z(\tilde{\Delta})\rtimes\sigma$ is reducible. Thus, the assumption on $\alpha$ in Corollary 6.19 is essential.