Polynomial properties of unipotent radicals of parabolic subgroups in classical groups

TL;DR

This paper derives explicit formulas for counting irreducible characters of unipotent radicals in classical groups, enhancing understanding of their representation theory.

Contribution

It provides new explicit formulas for the number of irreducible characters of unipotent radicals in classical groups, a previously unquantified aspect.

Findings

Formulas for irreducible character counts in symplectic groups

Formulas for irreducible character counts in orthogonal groups

Formulas for irreducible character counts in unitary groups

Abstract

Let $R_u^{X,d}$ denote the unipotent radical of a (proper) maximal standard parabolic subgroup of the classical group $\mathrm{Sp}_{2n}(q)$, $\mathrm{SO}_{2n}(q)$, or $\mathrm U_{2n}({q^2})$. This paper establishes explicit formulas for the number of irreducible characters of $R_u^{X,d}$ with degree $q^e$.

Tables

Table 1. Table 1.

$X$	$u$	$v$	$w$	${𝔫}_H^X$	${𝔫}_A^X$
$C$
$D$

Table 2. Table 2.

$X$	$L$	$M$	$N$	$U$	$V$	$W$
$C$		$0_{n\times n}$		$I_n$		$I_n$
$D$		$0_{n\times n}$		$I_n$		$I_n$
$U$		$0_{n\times n}$		$I_n$		$I_n$

Equations

$${\mathfrak{n}}^{X,d}:=\left\{\left(\begin{array}{cc}u& v\\ 0& w\end{array}\right)\right\}.$$

$$R_u^{X,d}\simeq A^X⋊H^X.$$

$$J_{2n}=\begin{pmatrix}0&\cdots&1\\ \vdots&\begin{sideways}$\ddots$\end{sideways}&\vdots\\ 1&\cdots&0\\ \end{pmatrix}.$$

$$\text{dynkin}[labels\ast ={\alpha }_1,{\alpha }_2,,{\alpha }_n,,{\alpha }_{2n-2},{\alpha }_{2n-1},labeldirections=,,left,,,,right,,,scale=3,]AIIIb.$$

$$P^{d}=\left\{\begin{pmatrix}A_{d}^{1}&A^{2}&A^{3}\\ 0&A_{2n-2d}^{4}&A^{5}\\ 0&0&A_{d}^{6}\\ \end{pmatrix}\Bigg{|}\quad\begin{aligned} A_{d}^{1},A_{d}^{6}\in{GL}(d,q^{2}),A_{2n-2d}^{4}\in{GL}(2n-2d,q^{2}),\\ A^{2}\in{\mathrm{Mat}}_{d\times(2n-2d)}(q^{2}),A^{3}\in{\mathrm{Mat}}_{d\times d}(q^{2}),\\ A^{5}\in{\mathrm{Mat}}_{(2n-2d)\times d}(q^{2}).\end{aligned}\right\},$$

$$R_{u}^{d}=\left\{\begin{pmatrix}I_{d}&B^{1}&B^{2}\\ 0&I_{2n-2d}&B^{3}\\ 0&0&I_{d}\\ \end{pmatrix}\Bigg{|}\quad\begin{aligned} B^{1}\in{\mathrm{Mat}}_{d\times(2n-2d)}(q^{2}),\\ B^{2}\in{\mathrm{Mat}}_{d\times d}(q^{2}),\\ B^{3}\in{\mathrm{Mat}}_{(2n-2d)\times d}(q^{2}).\end{aligned}\right\}.$$

$$R_u^d\simeq A⋊H.$$

$$H=\left\{\left(\begin{array}{cccc}{I}_d& A_1& 0& 0\\ 0& I_{n-d}& 0& 0\\ 0& 0& I_{n-d}& A_2\\ 0& 0& 0& I_d\end{array}\right)\right\},A=\left\{\left(\begin{array}{cccc}{I}_d& 0& B_1& B_2\\ 0& I_{n-d}& 0& B_3\\ 0& 0& I_{n-d}& 0\\ 0& 0& 0& I_d\end{array}\right)\right\}.$$

$$R_u^{\mathrm{U},d}\simeq A^{\mathrm{U}}⋊H^{\mathrm{U}}.$$

$$\mathrm{tr}:{\mathrm{Mat}}_{2n}({\mathbb{F}}_q)\times {\mathrm{Mat}}_{2n}({\mathbb{F}}_q)\to {\mathbb{F}}_q,(A,B)\mapsto \mathrm{tr}(A\cdot B)$$

$$\widetilde{\mathrm{tr}}:{\mathfrak{n}}_A^{\mathrm{U}}\times ({\mathfrak{n}}_A^{\mathrm{U}}{)}^t\to {\mathbb{F}}_q,(A,B)\mapsto \mathrm{tr}(A\cdot B{)}^q+\mathrm{tr}(A\cdot B)$$

$${\chi }^h(a)=\chi (h^{-1}ah)\text{ for }a\in A.$$

$${\hat{A}}^X=\{{\psi }_T|T\in B_X^t\},$$

$${\psi }_T(I+x):=\psi (\mathrm{tr}Tx),\text{ for }x\in B_X.$$

$${\hat{A}}^{\mathrm{U}}=\{{\psi }_T|T\in B_{\mathrm{U}}^t\},$$

$${\psi }_T(I+x):=\psi (\widetilde{\mathrm{tr}}Tx),\text{ for }x\in B_{\mathrm{U}}.$$

$$\mathrm{Supp}(c):=\{(i,j)\mid c_{i,j}\ne 0\}.$$

$$\mathrm{Supp}(M):=\bigcup _{c\in M}\mathrm{Supp}(c).$$

$$(g\cdot \alpha )(I+b)=\mathrm{tr}(\alpha g^{-1}bg),\text{ for }b\in B_{\mathrm{C}}.$$

$$([m{]}_{B_{\mathrm{C}}}{)}_{i,j}=m_{i,j}\text{ if }(j,i)\in \mathrm{Supp}(B_{\mathrm{C}}),\text{ and }([m{]}_{B_{\mathrm{C}}}{)}_{i,j}=0\text{ if }(j,i)\notin \mathrm{Supp}(B_{\mathrm{C}}).$$

$$g{\circ }_{B_{\mathrm{C}}}\alpha :=[g\alpha g^{-1}{]}_{B_{\mathrm{C}}}\text{ for }g\in H^{\mathrm{C}}\text{ and }\alpha \in B_{\mathrm{C}}^t.$$

$$\left(\begin{array}{cccc}{0}_{d\times d}& 0& 0& 0\\ 0& 0_{(n-d)\times (n-d)}& 0& 0\\ B_1& B_3& 0_{d\times d}& 0\\ B_2& 0& 0& 0_{(n-d)\times (n-d)}\end{array}\right),$$

$$V_{\mathrm{C}}(A):=\left(\begin{array}{cccc}{I}_d& A& 0& 0\\ 0& I_{n-d}& 0& 0\\ 0& 0& I_d& 0\\ 0& 0& -A^t& I_{n-d}\end{array}\right),$$

$$[V_{\mathrm{C}}(A)\cdot P_{\mathrm{C}}(B_1,B_3,B_2,0)\cdot V_{\mathrm{C}}(-A){]}_{B_{\mathrm{C}}}$$

$$=P_{\mathrm{C}}(B_1,B_3,B_2,0)+[P_{\mathrm{C}}(0,-B_{1}A,-A^t{B}_1,A^t{B}_{1}A-B_{2}A-A^t{B}_2){]}_{B_{\mathrm{C}}}.$$

$$[P_{\mathrm{C}}(0,-B_{1}A,-A^t{B}_1,A^t{B}_{1}A-B_{2}A-A^t{B}_2){]}_{B_{\mathrm{C}}}=0$$

$$[P_{\mathrm{C}}(0,-B_{1}A,-A^t{B}_1,0){]}_{B_{\mathrm{C}}}=0$$

$$P_{\mathrm{U}}(B_1,B_3,B_2,0):=\left(\begin{array}{cccc}{0}_{d\times d}& 0& 0& 0\\ 0& 0_{(n-d)\times (n-d)}& 0& 0\\ B_1& 0& 0_{(n-d)\times (n-d)}& 0\\ B_2& B_3& 0& 0_{d\times d}\end{array}\right),$$

$$V_{\mathrm{U}}(A):=\left(\begin{array}{cccc}{I}_d& A_1& 0& 0\\ 0& I_{n-d}& 0& 0\\ 0& 0& I_{n-d}& A_2\\ 0& 0& 0& I_d\end{array}\right),$$

$$[P_{\mathrm{U}}(A_2{B}_2,-B_2{A}_1,0,0){]}_{B_{\mathrm{U}}}=0,$$

Full text

Keywords: Maximal standard parabolic subgroup, Unipotent radical, Orbit theory, Polynomial properties, Schubert variety.

1. Introduction

Let ${\mathbb{F}}_{q}$ be the finite field of $q$ elements, where $q$ is a power of an odd prime. Denote by $U_{n}(q)$ the unitriangular group of degree $n$ over ${\mathbb{F}}_{q}$. Let $\mathcal{D}$ be a subset of $\Delta_{n}:=\{(i,j)\mid 1\leq i<j\leq n\}$ which is closed in the sense that if $(i,j),(j,k)\in\mathcal{D}$ then $(i,k)\in\mathcal{D}$. A pattern group $G_{\mathcal{D}}(q)$ is a subgroup of $U_{n}(q)$ consisting of the matrices whose $(i,j)$-th entry is nonzero only if $(i,j)\in\mathcal{D}$. The classification of all irreducible representations of all pattern groups is a wild problem. Even counting the number of irreducible characters of such groups is not easy. By a result due to Isaacs [4], it is known that the dimension of a complex irreducible representation of a pattern group is always a power of $q$. In [8], Lehrer gave a conjecture as follows:

Conjecture 1.1** **(Lehrer’s conjecture).

The number of irreducible characters of $U_{n}(q)$ with degree $q^{e}$ $(e\in{\mathbb{N}})$ is a polynomial in $q$ with integral coefficients.

Let $N_{n,e}(q)$ denote the number of irreducible characters of $U_{n}(q)$ with degree $q^{e}$. Isaacs gave a strengthened form of Lehrer’s conjecture in [5].

Conjecture 1.2** **(Isaacs’s conjecture).

The functions $N_{n,e}(q)$ are polynomials in $q-1$ with non-negative integral coefficients.

The Sylow $p$-subgroups of the general linear groups are the groups of unitriangular matrices. There is an extensive literature on these groups and, specifically, on their characters. Isaacs showed that the degrees of the irreducible characters are always powers of $q$, the size of the underlying field [4, Corollary B]. In paper [5] this author has investigated the irreducible characters of maximal and second maximal degree and computed the exact number of such characters. In [12], Sangroniz delineated the Sylow $p$-subgroups of classical groups and their corresponding Lie algebras, further exploring the irreducible characters of the highest degree Sylow $p$-subgroup within these classical groups. Inspired by his work, we study the irreducible characters of unipotent radicals in maximal standard parabolic subgroups of classical groups. We refer to a group as admitting the polynomial property if an analogue of Lehrer’s conjecture holds for it. That is, the number of irreducible characters of a group with degree $q^{e}$ is a polynomial in $q$ with integral coefficients. In [11], Nien proved the polynomial property for unipotent radicals of standard parabolic subgroups with 3 parts. The motivation of the paper is an extension of the works of Lehrer-Isaacs-Nien.

Let $G$ be the symplectic group $\mathrm{Sp}_{2n}(q)$ (type $\mathrm{C}$) or even special orthogonal group $\mathrm{SO}_{2n}(q)$ (type $\mathrm{D}$) defined over ${\mathbb{F}}_{q}$, or even unitray group $\mathrm{U}_{2n}(q^{2})$ (type $\mathrm{U}$) defined over ${\mathbb{F}}_{q^{2}}$ (the notation for the even unitary group is typically ${}^{2}A_{2n}$, but for the sake of convenience in writing, we shall denote it by $\mathrm{U}$ in this context). Denote by ${R_{u}^{X,d}}$ the unipotent radical of a maximal standard parabolic subgroup in the group of type $X$, where $X=\mathrm{C},\mathrm{D},\mathrm{U}$. Each ${R_{u}^{X,d}}$ has a canonical semi-direct product decomposition into two of its abelian subgroups, i.e. ${R_{u}^{X,d}}\simeq A^{X}\rtimes H^{X}$ (see Table 2 for the abelian subgroups $H^{X}$ and $A^{X}$). We introduce a map $\pi_{c}$ from ${\mathrm{Lie}}(A^{X})^{t}$, the dual space of ${\mathrm{Lie}}(A^{X})$, to a subset of ${\mathrm{Mat}}_{h}(k)$ consisting of matrices of size $h$ over $k$. We give a numerical formula for the number of irreducible characters with degree $q^{e}$ for $e\in{\mathbb{N}}$. In essence, this formula is a $q$-power product with $|\tilde{\mathfrak{P}}_{X}({R_{u}},e)|$, where $|\tilde{\mathfrak{P}}_{X}({R_{u}},e)|$ is the preimage of $\pi_{c}$ over a subset of ${\mathrm{Mat}}_{h}(k)$; see 3.8. The problem of the polynomiality of ${R_{u}^{X,d}}$ then becomes the question of whether $|\tilde{\mathfrak{P}}_{X}({R_{u}},e)|$ is a polynomial in $q$ with integral coefficients for $e\in{\mathbb{N}}$. Using the methods of Clifford theory and finite field version of orbit theory, we know that $|\tilde{\mathfrak{P}}_{X}({R_{u}},e)|$ is a $q$-power product with the cardinality of a specific class of matrices (such as symmetric matrices for type $\mathrm{C}$). On the other hand, we can also use the geometric method to prove the polynomial property and get a stronger result. That is, the number of irreducible characters of ${R_{u}^{X,d}}$ with degree $q^{e}$ are polynomials in $q-1$ with non-negative integral coefficients.

This paper is organized as follows. In Section 2, we give the necessary preliminaries for the unipotent radicals of the maximal standard parabolic subgroups. In Section 3, we review Clifford theory and finite version orbit theory, establishing character number formulas of a given degree for all ${R_{u}^{X,d}}$. In Section 4, we explicitly give all irreducible character number formulas for these unipotent radicals. In Section 5, using geometric methods, we obtain a stronger result with respect to the number of irreducible characters of ${R_{u}^{X,d}}$ with degree $q^{e}$.

Throughout this paper, we use ${\mathbb{N}}$ to represent the set of non-negative integers and ${\mathbb{Z}}_{+}$ to denote the set of positive integers. For any finite set $S$, we express its cardinality as $|S|$. The Lie algebra of a closed linear algebraic group $A$ is represented by ${\mathrm{Lie}}(A)$. We indicate the transposition of a matrix $A$ as $A^{t}$ and the inverse of $A^{t}$ as $A^{-t}$.

2. Preliminaries

Let $k$ be a finite field, either ${\mathbb{F}}_{q}$ or ${\mathbb{F}}_{q^{2}}$, where $q$ is a power of an odd prime. Consider the symplectic group $\mathrm{Sp}_{2n}(q)$ and the even special orthogonal group $\mathrm{SO}_{2n}(q)$ over ${\mathbb{F}}_{q}$. Also, consider the even unitary group $\mathrm{U}_{2n}(q^{2})$ over ${\mathbb{F}}_{q^{2}}$. These groups are referred to as types $\mathrm{C}$, $\mathrm{D}$ and $\mathrm{U}$, respectively.

2.1. Root Systems of $\mathrm{Sp}_{2n}(q)$ and $\mathrm{SO}_{2n}(q)$

The Dynkin diagrams of $\mathrm{Sp}_{2n}(q)$ and $\mathrm{SO}_{2n}(q)$ are depicted as follows:

• •

$C_{n}$ ($n\geq 3$): \dynkin[Kac,labels=α_1,α_2,,α_n-1,α_n, label directions=,,left,,,,right,,, scale=2, ] C,

• •

$D_{n}$ ($n\geq 4$): \dynkin[Kac,labels=α_1,α_2,,α_n-2,α_n-1,α_n, label directions=,,left,,,,right,,, scale=2, ] D,

where $\alpha_{i}$ are simple roots.

2.2. Unipotent radicals of maximal standard parabolic subgroups

2.2.1. The $\mathrm{Sp}_{2n}(q)$ and $\mathrm{SO}_{2n}(q)$ cases

Let $G$ be $\mathrm{Sp}_{2n}(q)$ or $\mathrm{SO}_{2n}(q)$. Denote $T\subset G$ as the set of all diagonal elements and $B\subset G$ as the standard Borel subgroup containing $T$. We define $W$ as the Weyl group of $G$ and $S$ as the set of simple roots relative to the pair $(B,T)$. For a particular subset $S_{Q}\subset S$, let $Q$ be the standard parabolic subgroup associated with $S_{Q}$. We denote by $P^{\mathrm{C},d}$ the maximal standard parabolic subgroup of $\mathrm{Sp}_{2n}(q)$ with $S\backslash\{\alpha_{d}\}$ as its associated set of simple roots. Furthermore, we use ${R_{u}^{\mathrm{C},d}}$ to represent its unipotent radical and ${\mathfrak{n}}^{\mathrm{C},d}$ for the nilpotent radical of the corresponding parabolic subalgebra. Similar notation ${R_{u}^{\mathrm{D},d}}$ and ${\mathfrak{n}}^{\mathrm{D},d}$ are used for $\mathrm{SO}_{2n}(q)$. We have :

[TABLE]

It is the semi-direct product of ${\mathfrak{n}}_{H}^{X}$ and ${\mathfrak{n}}_{A}^{X}$ ($X=\mathrm{C},\mathrm{D}$). The algebraic groups corresponding to ${\mathfrak{n}}_{H}^{X}$ and ${\mathfrak{n}}_{A}^{X}$ are denoted by $H^{X}$ and $A^{X}$, respectively. These will be detailed in Tables 1 and 2. Consequently, ${R_{u}^{X,d}}$ is the semi-direct product of $H^{X}$ and $A^{X}$:

[TABLE]

2.2.2. The $\mathrm{U}_{2n}(q^{2})$ case

Let $G={GL}(2n,q^{2})$ with Steinberg endomorphism $F=F_{q}\circ\sigma$, where $F_{q}:G\to G,(a_{ij})\mapsto(\overline{a_{ij}}):=(a^{q}_{ij})$ is the standard Frobenius map and $\sigma:G\to G,(a_{ij})\mapsto J_{2n}((a_{ij})^{-t})J_{2n}$ with

[TABLE]

The fixed points subgroup $G^{F}\cong{\mathrm{U}}_{2n}(q^{2})$. Denote by $W$ the Weyl group of $G$ generated by the set of simple reflections $S=\{s_{i}\mid 1\leq i\leq 2n-1\}$. Let the group $W^{F}$ be the Weyl group of $G^{F}$. This group is generated by the set $S_{F}$ of involutions which are in natural bijection with the $F$-orbits on $S$ (see [10, Lemma 23.3]). Denote by $P_{I}$ the parabolic subgroup of $G$ corresponding to $I\subseteq S$ and represent $L_{I}$ as its standard Levi complement. According to the double coset decomposition in [10, Proposition 12.2], $P_{I}$ is $F$-stable if and only if $I$ is $F$-invariant. In other words, $I$ must be a union of $F$-orbits in $S$ and correspond to a subset $I_{F}\subseteq S_{F}$. We denote $P_{I_{F}}$ by the parabolic subgroups of $G^{F}$ defined by $I_{F}$. In this context, we have $F(s_{i})=s_{2n-i}$ ($1\leq i\leq 2n-1$) and $S_{F}=\{\beta_{i}=s_{i}s_{2n-i},\beta_{n}=s_{n}\mid 1\leq i\leq n-1\}$, where $s_{j}\in S$. Below is the Satake diagram for ${\mathrm{U}}_{2n}$:

[TABLE]

Lemma 2.1**.**

[10, Proposition 26.1]** Let G be a connected reductive group with a Steinberg endomorphism $F$ and let $I\subseteq S$ be an $F$-invariant subset.

• •

There is a unique $G^{F}$-conjugacy class of $F$-stable $G$-conjugates of $P_{I}$, and we have $P_{I}^{F}=P_{I_{F}}$.

• •

There is a unique $P_{I}^{F}$-conjugacy class of the $F$-stable Levi complements ${L_{I}}$ to $U_{I}=R_{u}(P_{I})$ in $P_{I}$, and we have $P_{I}^{F}\simeq L_{I}^{F}\ltimes U_{I}^{F}$.

The above lemma allows us to identify the maximal parabolic subgroup of $\mathrm{U}_{2n}(q^{2})$ corresponding to $S_{F}\backslash\{\beta_{d}\}$ ($1\leq d\leq n-1$). We denote this group by $P^{\mathrm{U},d}$ and refer to its unipotent radical as $R_{u}^{\mathrm{U},d}$. Let $P^{d}$ be the maximal $F$-stable parabolic subgroup of ${GL}(2n,q^{2})$ corresponding to $S\backslash\{\alpha_{d},\alpha_{2n-d}\}$ and $R_{u}^{d}$ be its unipotent radical. We have

[TABLE]

[TABLE]

Furthermore, the group $R_{u}^{d}$ can be decomposed as a semi-direct product in the following way:

[TABLE]

[TABLE]

Denote by $H^{\mathrm{U}}$ ($A^{\mathrm{U}}$, respectively) the fixed points subgroup $H^{F}$ ($A^{F}$, respectively). Then, we have

[TABLE]

We will describe them explicitly in Table 2. It follows from [10, Proposition 23.2] that the semi-direct product $A^{\mathrm{U}}\rtimes H^{\mathrm{U}}$ is also $F$-stable.

2.2.3.

Let $H^{X}=\left\{\begin{pmatrix}L&M\\ 0&N\\ \end{pmatrix}\right\},A^{X}=\left\{\begin{pmatrix}U&V\\ 0&W\\ \end{pmatrix}\right\}$. We have:

• •

The matrices $A\in{\mathrm{Mat}}_{d\times(n-d)}({\mathbb{F}}_{q})$, $A_{1}\in{\mathrm{Mat}}_{d\times(n-d)}({\mathbb{F}}_{q^{2}})$.

• •

The matrices $v$ and $V$ are symmetric, $B_{1}\in{\mathrm{Mat}}_{d\times d}({\mathbb{F}}_{q})$, $B_{2}\in{\mathrm{Mat}}_{d\times(n-d)}({\mathbb{F}}_{q})$, $B_{3}\in{\mathrm{Mat}}_{(n-d)\times d}({\mathbb{F}}_{q})$, if $X=\mathrm{C}$.

• •

The matrices $v$ and $V$ are skew-symmetric, $B_{1}\in{\mathrm{Mat}}_{d\times d}({\mathbb{F}}_{q})$, $B_{2}\in{\mathrm{Mat}}_{d\times(n-d)}({\mathbb{F}}_{q})$, $B_{3}\in{\mathrm{Mat}}_{(n-d)\times d}({\mathbb{F}}_{q})$, if $X=\mathrm{D}$.

• •

The matrices $B_{1}=-J_{d}\overline{B_{3}^{t}}J_{n-d}$, $B_{2}=-J_{d}\overline{B_{2}^{t}}J_{d}$, and $A_{1}=-J_{d}\overline{A_{2}^{t}}J_{n-d}$. The matrices $B_{1}\in{\mathrm{Mat}}_{d\times(n-d)}({\mathbb{F}}_{q^{2}})$, $B_{2}\in{\mathrm{Mat}}_{d\times d}({\mathbb{F}}_{q^{2}})$, $B_{3}\in{\mathrm{Mat}}_{(n-d)\times d}({\mathbb{F}}_{q^{2}})$, if $X=\mathrm{U}$.

We give some lemma which will be used in the next section.

Lemma 2.2**.**

The trace map

[TABLE]

restricted to $\mathfrak{n}^{\mathrm{C}}_{{A}}\times(\mathfrak{n}^{\mathrm{C}}_{A})^{t}$ becomes a perfect pairing.

Proof.

Let $X=\begin{pmatrix}0&B\\ 0&0\end{pmatrix}\in{\mathfrak{n}}_{A}^{\mathrm{C}}$ and $Y=\begin{pmatrix}0&0\\ A&0\end{pmatrix}\in(\mathfrak{n}^{\mathrm{C}}_{A})^{t}$, where $B=\begin{pmatrix}B_{1}&B_{2}\\ B_{3}&0\end{pmatrix}$ and $A=\begin{pmatrix}A_{1}&A_{3}\\ A_{2}&0\end{pmatrix}$ are symmetric. If ${\mathrm{tr}}(X\cdot Y)=0$ for all $Y\in(\mathfrak{n}^{\mathrm{C}}_{A})^{t}$, we get ${\mathrm{tr}}(B_{1}A_{1}+B_{2}A_{2}+B_{3}A_{3})=0$ for all $A$. Let $B_{k}=(b_{i,j}^{k})$ $(k=1,2,3)$. First, taking $A_{2}=0,A_{1}=E_{i,i}\ (E_{i,j}+E_{j,i},\text{respectively})$, we get $b_{i,i}^{1}=0\ (b_{i,j}^{1}=b_{j,i}^{1}=0,\text{respectively})$. Next, taking $A_{1}=0$ and $A_{2}=E_{i,j}$, we get $b_{i,j}^{2}=0$. Thus, $X=0$ and the map restricted to $\mathfrak{n}^{\mathrm{C}}_{{A}}\times(\mathfrak{n}^{\mathrm{C}}_{A})^{t}$ is non-degenerate. ∎

Lemma 2.3**.**

The trace map

[TABLE]

restricted to $\mathfrak{n}^{\mathrm{D}}_{{A}}\times(\mathfrak{n}^{\mathrm{D}}_{A})^{t}$ becomes a perfect pairing.

Proof.

The proof is similar to that of Lemma 2.2. ∎

Let $\mathfrak{n}^{\mathrm{U}}_{A}:={\mathrm{Lie}}(A^{\mathrm{U}})$ and ${\mathfrak{n}}^{\mathrm{U},d}:={\mathrm{Lie}}(R_{u}^{\mathrm{U},d})$.

Lemma 2.4**.**

The map

[TABLE]

is a perfect pairing.

Proof.

Let $X=\begin{pmatrix}0&B\\ 0&0\end{pmatrix}\in{\mathfrak{n}}_{A}^{\mathrm{U}}$ and $Y=\begin{pmatrix}0&0\\ A&0\end{pmatrix}\in({\mathfrak{n}}_{A}^{\mathrm{U}})^{t}$, where $B=\begin{pmatrix}B_{1}&B_{2}\\ 0&B_{3}\end{pmatrix}$ and $A=\begin{pmatrix}A_{1}&0\\ A_{2}&A_{3}\end{pmatrix}$. By definition, $B_{1}=-J_{d}\overline{B_{3}^{t}}J_{n-d}$, $B_{2}=-J_{d}\overline{B_{2}^{t}}J_{d}$, $A_{1}=-J_{n-d}\overline{A_{3}^{t}}J_{d}$ and $A_{2}=-J_{d}\overline{A_{2}^{t}}J_{d}$. If $\widetilde{{\mathrm{tr}}}(X\cdot Y)=0$ for all $Y\in({\mathfrak{n}}_{A}^{\mathrm{U}})^{t}$, we get $W=\mathrm{tr}(B_{1}A_{1}+B_{2}A_{2}+B_{3}A_{3})+\mathrm{tr}(\overline{B_{1}A_{1}+B_{2}A_{2}+B_{3}A_{3}})=0$ for all $A$. First, letting $A_{1}=A_{3}=0$, we have $W=2\mathrm{tr}(B_{2}A_{2})=0$. Thus, $B_{2}=0$. Next, letting $A_{2}=0$, we have $W=2\mathrm{tr}(B_{1}A_{1})+2\mathrm{tr}(\overline{B_{1}A_{1}})=0$. If $B_{1}\neq 0$, then there exists a matrix $C_{1}$ such that $\mathrm{tr}(B_{1}C_{1})=c\neq 0$ and $c+c^{q}=0$. For all ${a}\in{\mathbb{F}}_{q^{2}}$, we also have $ac+(ac)^{q}=0$. Thus, $a=a^{q}$ since $c\neq 0$. That is a contradiction, and the map is non-degenerate. ∎

3. Character Formulas via Coadjoint Orbits

3.1. Clifford theory and coadjoint orbits

Let us review Clifford theory and coadjoint orbit theory [13, 6]. Assume that a finite group $G=A\rtimes H$ is a semidirect product of $A$ by $H$, where $A$ is an abelian normal subgroup of $G$. Let $\widehat{A}$ be the set of all irreducible characters of $A$. Let $h\in H$ act on $\widehat{A}$ by $h\cdot\chi=\chi^{h},$ where $\chi^{h}\in\widehat{A}$ is given by

[TABLE]

Let $H_{\chi}=\{h\in H\ |\ \chi^{h}=\chi\}$ be the stabilizer of $\chi$ in $H$. For $\chi\in\widehat{A}$, we identify it with a character of $A\rtimes H_{\chi}$ by $\chi(ah):=\chi(a)$, where $a\in A$ and $h\in H_{\chi}$. For $\tau\in\widehat{H}_{\chi}$, we identify it with a character of $A\rtimes H_{\chi}$ via the canonical projection $A\rtimes H_{\chi}\to H_{\chi}$. Then the irreducible characters of $G$ can be described in terms of induced characters of the following form:

Theorem 3.1**.**

[13, Section 8.2, Proposition 25]** For $\chi,\ \chi^{\prime}\in\widehat{A}$, and $\tau,\ \tau^{\prime}\in\widehat{H}_{\chi},$ the following holds.

1. (1)

${\textup{Ind}}_{A\rtimes H_{\chi}}^{G}\chi\otimes\tau$* is an irreducible character of $G$.* 2. (2)

${\textup{Ind}}_{A\rtimes H_{\chi}}^{G}\chi\otimes\tau\cong{\textup{Ind}}_{A\rtimes H_{\chi^{\prime}}}^{G}\chi^{\prime}\otimes\tau^{\prime}$* if and only if $\chi$ and $\chi^{\prime}$ are in the same $H$-orbit, and $\tau=\tau^{\prime}$.* 3. (3)

Every irreducible character $\pi$ of $G$ is isomorphic to ${\textup{Ind}}_{A\rtimes H_{\chi}}^{G}\chi\otimes\tau,$ for some $\chi\in\widehat{A}$, and $\tau\in\widehat{H}_{\chi}.$

Fix a nontrivial one-dimensional character $\psi$ of ${\mathbb{F}}_{q}$. Note that $A^{X}=I+B_{X}$ is an abelian group with $B_{X}={\mathrm{Lie}}(A^{X})={\mathfrak{n}}_{A}^{X}$ ($X=\mathrm{C},\mathrm{D}$). By Lemma 2.2 and Lemma 2.3,

[TABLE]

where

[TABLE]

Also, we have $A^{\mathrm{U}}=I+B_{\mathrm{U}}$ is an abelian group with $B_{\mathrm{U}}={\mathrm{Lie}}(A^{\mathrm{U}})={\mathfrak{n}}_{A}^{\mathrm{U}}$. By Lemma 2.4,

[TABLE]

where

[TABLE]

Definition 3.2**.**

For a matrix $c=(c_{i,j})\in{\mathfrak{n}}^{X,d}$, denote by

[TABLE]

For a subset $M\subset{\mathfrak{n}}^{X,d}$, denote by

[TABLE]

For $A^{\mathrm{C}}=I+B_{\mathrm{C}}$, we may identify its dual space with $B_{\mathrm{C}}^{t}$ via the trace map. The coadjoint action of $g\in H^{\mathrm{C}}$ on $\alpha\in B_{\mathrm{C}}^{t}$ in [6] is given by

[TABLE]

It can be identified with $\alpha\mapsto[g\alpha g^{-1}]_{B_{\mathrm{C}}}$, where $[\cdot]_{B_{\mathrm{C}}}$ is the projection from ${\mathrm{Mat}}_{2n}({\mathbb{F}}_{q})$ to $B_{\mathrm{C}}^{t}$. More precisely, for $m=(m_{i,j})\in\bigcup_{g\in H_{\mathrm{C}}}gB_{\mathrm{C}}^{t}g^{-1}$,

[TABLE]

Define the action of $H^{\mathrm{C}}$ on $B_{\mathrm{C}}^{t}$ by

[TABLE]

Since $(g\cdot\alpha)(I+b)={\mathrm{tr}}([g\alpha g^{-1}]_{B_{\mathrm{C}}}b)$ for all $b\in B_{\mathrm{C}}$, the action $\circ_{B_{\mathrm{C}}}$ of $H^{\mathrm{C}}$ on $B_{\mathrm{C}}^{t}$ coincides with the coadjoint action, whose orbits are called coadjoint orbits. The definition of coadjoint orbits for type $\mathrm{D}$ are similar to that of type $\mathrm{C}$, while for type $\mathrm{U}$ it is necessary to replace $\widetilde{{\mathrm{tr}}}$ with ${\mathrm{tr}}$ in Equation (3.3).

3.2. Characters number formula

Recall ${R_{u}^{X,d}}\simeq A^{X}\rtimes H^{X}$. When $X=\mathrm{C}$, we simply write $P_{\mathrm{C}}(B_{1},B_{3},B_{2},0)\in{\mathrm{Lie}}(A^{\mathrm{C}})^{t}$ for

[TABLE]

where $B_{1}$ is symmetric and $B_{3}^{t}=B_{2}$.

Let

[TABLE]

where $A$ be the matrix which take the values in ${\mathbb{F}}_{q}$. Consider the coadjoint action of $V_{\mathrm{C}}(A)$ on $P_{\mathrm{C}}(B_{1},B_{3},B_{2},0)$. That is,

[TABLE]

It is clear that

[TABLE]

gives algebraic equations in variables $\mathbf{a}_{i,j}$ for $(i,j)\in{\mathrm{Supp}}({\mathfrak{n}}_{H}^{\mathrm{C}})$, while

[TABLE]

gives linear equations.

The results for ${R_{u}^{{\mathrm{D}},d}}$ are similar. The difference is $B_{1}=-B_{1}^{T}$ and $B_{2}^{T}=-B_{3}$.

When $X=\mathrm{U}$, $P_{\mathrm{C}}(B_{1},B_{3},B_{2},0)$ is replaced by

[TABLE]

where $B_{2}J_{d}$ is skew-Hermitian and $B_{1}=-J_{n-d}\overline{B_{3}^{t}}J_{d}$. The matrix $V_{\mathrm{C}}(A)$ is replaced by

[TABLE]

where $A_{1}$ is be the matrix which take the values in ${\mathbb{F}}_{q^{2}}$. Besides, $A_{1}=-J_{d}\overline{A_{2}^{t}}J_{n-d}$. Repeating the above process, we get

[TABLE]

which gives linear equations in variables $\mathbf{a}_{i,j}$ for $(i,j)\in{\mathrm{Supp}}({\mathfrak{n}}_{H}^{\mathrm{U}})$. We denote the coefficient matrix of (3.6) and (3.7) by ${\mathscr{P}}_{X}(B_{1},B_{3},B_{2},0)$ under a fixed total order of $\mathbf{a}_{i,j}$ ($X=\mathrm{C},\mathrm{D},\mathrm{U}$). Define

[TABLE]

Let

[TABLE]

be the canonical map. For $e\in{\mathbb{N}}$, define

[TABLE]

Then $\mathfrak{P}_{X}({R_{u}})=\bigcup_{e=0}^{\infty}\mathfrak{P}_{X}({R_{u}},e)$ is a finite partition of $\mathfrak{P}_{X}({R_{u}})$. Denote the fiber of $\pi_{c}$ over $\mathfrak{P}_{X}({R_{u}},e)$ by $\tilde{\mathfrak{P}}_{X}({R_{u}},e)$.

Let $\psi:({\mathfrak{n}}_{A}^{\mathrm{X}})^{t}\to\widehat{A}^{X},T\mapsto\psi_{T}$ ($X=\mathrm{C},\mathrm{D},\mathrm{U}$). We have the following lemma that establishes a one-to-one correspondence between the $H^{X}$-orbit on $\widehat{A}^{X}$ and the coadjoint orbits on $({\mathfrak{n}}_{A}^{\mathrm{X}})^{t}$.

Lemma 3.3**.**

$\psi$* is compatible with the action of $H^{X}$ on $\widehat{A}^{X}$ and the coadjoint action of $H^{X}$ on $({\mathfrak{n}}_{A}^{\mathrm{X}})^{t}$.*

Proof.

We give just the proof of type $\mathrm{C}$. Let $g\in H^{\mathrm{C}}$, $a\in({\mathfrak{n}}_{A}^{\mathrm{C}})^{t}$. We have

[TABLE]

[TABLE]

∎

We no longer distinguish between the coadjoint orbit $\mathbb{O}_{a}$ in $({\mathfrak{n}}_{A}^{\mathrm{X}})^{t}$ and its corresponding orbit $\mathbb{O}_{\psi_{a}}$ in $\widehat{A}^{X}$, as well as the stabilizer subgroups of $a\in({\mathfrak{n}}_{A}^{\mathrm{X}})^{t}$ and $\psi_{a}\in\widehat{A}^{X}$.

Theorem 3.4** (Characters number formula).**

The number of irreducible characters of degree $|k|^{e}$ in ${R_{u}^{X,d}}$ is

[TABLE]

Furthermore, the set of integers that occur as degrees of irreducible characters of ${R_{u}^{X,d}}$ is precisely $\{|k|^{e}\mid\mathfrak{P}_{X}({R_{u}},e)\neq\emptyset\}$ ($$|k|=q if $X=\mathrm{C}$ or $\mathrm{D}$, $|k|=q^{2}$ if X=\mathrm{U}$$).

Proof.

We give just the proof of ${R_{u}^{\mathrm{C},d}}$ ($k={\mathbb{F}}_{q}$). Fix a nontrivial one-dimensional character $\psi$ of ${\mathbb{F}}_{q}$. Then $\hat{A}^{\mathrm{C}}=\{\psi_{T}\mid T\in({\mathfrak{n}}_{A}^{\mathrm{C}})^{t}\}$, which we identify $\hat{A}^{\mathrm{C}}$ with $({\mathfrak{n}}_{A}^{\mathrm{C}})^{t}$ via $T\leftrightarrow\psi_{T}$ by Lemma 2.2. Moreover, the action of $H^{\mathrm{C}}$ on $\hat{A}^{\mathrm{C}}$ coincides with the coadjoint action of $H^{\mathrm{C}}$ on $({\mathfrak{n}}_{A}^{\mathrm{C}})^{t}$. Thus, we can identify the $H^{\mathrm{C}}$-orbits on $\hat{A}^{\mathrm{C}}$ with the coadjoint orbits of $H^{\mathrm{C}}$ on $({\mathfrak{n}}_{A}^{\mathrm{C}})^{t}$. For a coadjoint orbit $\mathbb{O}_{a}$ with $a\in\mathbb{O}_{a}$, let $H_{a}^{\mathrm{C}}$ be the stabilizer of $a$ in $H^{\mathrm{C}}$. Clearly, $H_{a}^{\mathrm{C}}$ is an abelian group, and $\hat{H}_{a}^{\mathrm{C}}$ consists of $|{H}_{a}^{\mathrm{C}}|$ one-dimensional irreducible characters. By Theorem 3.1, ${\mathbb{O}}_{a}$ gives $|{H}_{a}^{\mathrm{C}}|$ irreducible characters of ${R_{u}^{\mathrm{C},d}}$ of degree $[H^{\mathrm{C}}:{H}_{a}^{\mathrm{C}}]$, where $[H^{\mathrm{C}}:{H}_{a}^{\mathrm{C}}]$ is the index of ${H}_{a}^{\mathrm{C}}$ in $H^{\mathrm{C}}$. We write

[TABLE]

where $A=(a_{i,j})$ , $a_{i,j}=0$ if $(i,j)\notin{\mathrm{Supp}}({\mathfrak{n}}_{H}^{\mathrm{U}})$. We have

[TABLE]

We write $a=P_{\mathrm{C}}(B_{1},B_{3},B_{2},0)$, then $|{H}_{a}^{\mathrm{C}}|=q^{-{\mathrm{rank}}({\mathscr{P}}_{\mathrm{C}}(B_{1},B_{3},B_{2},0))}\cdot|H^{\mathrm{C}}|$, and the degree of irreducible characters associated with ${\mathbb{O}}_{a}$ is $q^{{\mathrm{rank}}({\mathscr{P}}_{C}(B_{1},B_{3},B_{2},0)}$. We denote the number of $H^{\mathrm{C}}$-coadjoint orbits with stabilizer (of a representative element) of index $q^{e}$ by $N_{e}$, then the number of irreducible characters of ${R_{u}^{\mathrm{C},d}}$ of degree $q^{e}$ is

[TABLE]

Let

[TABLE]

be the set of union of all coadjoint orbits whose stabilizer in $H^{\mathrm{C}}$ has index $q^{e}$. Then

[TABLE]

which is just the set $\tilde{\mathfrak{P}}_{\mathrm{C}}({R_{u}},e)$. Since the cardinality of an $H^{\mathrm{C}}$-orbit is equal to the index of its stabilizer, the coadjoint orbits in $\mathcal{O}_{e}$ have the same cardinality $q^{e}$. Hence $N_{e}=q^{-e}\cdot|\tilde{\mathfrak{P}}_{\mathrm{C}}({R_{u}},e)|$, and the number of irreducible characters of ${R_{u}^{\mathrm{C},d}}$ of degree $q^{e}$ is

[TABLE]

The rest part is clear, since the map $\pi_{c}$ is surjective and all irreducible characters can be given as above. The proof for ${R_{u}^{\mathrm{D},d}}$ follows a pattern similar to the one presented here. For ${R_{u}^{\mathrm{U},d}}$, it is necessary to use Lemma 2.4 instead of Lemma 2.2. Furthermore, we must replace $q$ with $q^{2}$ in the relevant equations. Following these adjustments, the resultant formula remains consistent with the original. ∎

Remark 3.5**.**

In the case of $d=n$, the group ${R_{u}^{X,n}}$ is abelian. Similarly, in the case of $d=n-1$, the group ${R_{u}^{\mathrm{D},n-1}}$ is also abelian. It is clear that the number of irreducible characters is equal to the number of their elements.

4. The polynomial property of ${R_{u}^{X,d}}$

4.1. Polynomial property for unipotent radicals of maximal parabolic subgroups

We define ${R_{u}^{X,d}}$ as having the polynomial property if the number of its irreducible characters with degree $q^{e}$ can be expressed as a polynomial in $q$ with integral coefficients for any given $e\in{\mathbb{N}}$.

According to Theorem 3.4, the problem of determining the polynomial property of ${R_{u}^{X,d}}$ can be simplified to assess the polynomial property of $\tilde{\mathfrak{P}}_{X}({R_{u}},e)$ instead. In other words, ${R_{u}^{X,d}}$ possesses a polynomial property if and only if $|\tilde{\mathfrak{P}}_{X}({R_{u}},e)|$ is a polynomial in $q$ with integral coefficients.

Lemma 4.1**.**

[9, Theorem 2]** Let $N(n,r)$ denote the number of symmetric matrices of size $n\times n$ and rank $r$, with entries in ${\mathbb{F}}_{q}$. Then,

[TABLE]

[TABLE]

Lemma 4.2**.**

Let $S(n,2s)$ denote the number of skew-symmetric matrices in $\mathrm{Mat}_{n}({\mathbb{F}}_{q})$ with rank $2s$. We have

[TABLE]

Proof.

Let $S:=S(n,2s)$. Consider $G=GL(n,q)$ action on $S$ via congruence. That is to say, $P$ actions on $X$ is $PXP^{t}$ ($P\in G$, $X\in S$) and this action is transitive because any two skew-symmetric matrix with the same rank are congruent. Let $G_{X}=\{P\in G\mid P\cdot X=X\}$ . Without loss of generality, let

[TABLE]

where $E_{s}=\begin{pmatrix}0&\mathrm{Id}_{s}\\ -\mathrm{Id}_{s}&0\\ \end{pmatrix}$ is the matrix representing the standard non-degenerate symplectic form on a $2s$-dimensional space. Let $P\in GL(n,q)$ and decompose it as a block matrix

[TABLE]

$PXP^{t}=X$ if and only if

[TABLE]

The equation $AE_{s}A^{t}=E_{s}$ implies that $A$ is symplectic and in particular is of full rank. Then, the matrix $AE_{s}C^{t}=0$ implies that $C=0$, since $AE_{s}$ has full rank. Hence,

[TABLE]

So we have

[TABLE]

∎

Lemma 4.3**.**

For any skew-Hermitian matrix $C$ of size $n\times n$ and rank $r$, with entries in ${\mathbb{F}}_{q^{2}}$. There exists an invertible matrix $A$ such that

[TABLE]

where $\alpha\neq 0$ satisfies $\alpha+\alpha^{q}=0$.

Lemma 4.4**.**

Let $U(n,r)$ denote the number of skew-Hermitian matrices of size $n\times n$ and rank $r$, with entries in ${\mathbb{F}}_{q^{2}}$. Then,

[TABLE]

Proof.

Let $U_{\alpha}(\alpha\neq 0)$ be the subset of $U(n,r)$ consisting of all elements of $U(n,r)$ that are congruent to $X=\alpha\begin{pmatrix}I_{r}&0\\ 0&0\\ \end{pmatrix}$. Consider the action of $G=GL(n,{q^{2}})$ on $U_{\alpha}$ via congruence which is transitive by Lemma 4.3. Let $G_{X}=\{P\in G\mid P\cdot X=X\}$. Picking a matrix $P\in GL(n,{q^{2}})$ and decompose it as a block matrix

[TABLE]

$PX\overline{P^{t}}=X$ if and only if

[TABLE]

The equation $A\overline{A^{t}}=I_{r}$ implies that $A$ is unitary and in particular is of full rank. Then, the matrix $A\overline{C^{t}}=0$ implies that $C=0$, since $A$ has full rank. Hence

[TABLE]

So we have

[TABLE]

∎

Denote the number of irreducible characters of ${R_{u}^{X,d}}$ with degree $q^{e}$ by $N_{(X,d),e}$.

Theorem 4.5**.**

For any $1\leq d<n$, $0\leq r\leq d$. Let $e=(n-d)r$, we have

[TABLE]

Moreover, the set of integers that occur as degrees of irreducible characters of $R_{u}^{\mathrm{C},d}$ is precisely $\{q^{e}\mid 0\leq r\leq d,\ e=(n-d)r\}$.

Proof.

The coefficient matrix of (3.6) is ${\mathscr{P}}_{\mathrm{C}}(B_{1},B_{3},B_{2},0)$ under a fixed total order of $\mathbf{a}_{i,j}$, where

[TABLE]

is a block diagonal matrix of degree $d(n-d)$. Thus, $|\tilde{\mathfrak{P}}_{\mathrm{C}}({R_{u}},e)|=N(d,r)q^{d(n-d)}$. By Proposition 3.4 and Lemma 4.1, we complete the proof. ∎

The proof of the following theorem is similar to that of the above theorem.

Theorem 4.6**.**

For any $1\leq d<{n-1}$, $0\leq 2s\leq d$. Let $e=2(n-d)s$, we have

[TABLE]

Moreover, the set of integers that occur as degrees of irreducible characters of $R_{u}^{\mathrm{D},d}$ is precisely $\{q^{e}\mid 0\leq 2s\leq d,\ e=2(n-d)s\}$.

Proof.

By Proposition 3.4 and Lemma 4.2. ∎

Remark 4.7**.**

The method is not applicable for $\mathrm{SO}_{2n+1}(q)$ due to our inability to factor $R_{u}^{\mathrm{B},d}$ into a semi-direct product of two abelian groups.

Theorem 4.8**.**

For any $0\leq d\leq{n-1}$, $0\leq r\leq{d}$. Let $e=(n-d)r$, we have

[TABLE]

Moreover, the set of integers that occur as degrees of irreducible characters of $R_{u}^{\mathrm{U},d}$ is precisely $\{q^{e}\mid 0\leq r\leq{d},\ e=(n-d)r\}$.

Proof.

The coefficient matrix of (3.7) is

[TABLE]

which is a block diagonal matrix of degree $d(n-d)$. Thus, $|\tilde{\mathfrak{P}}_{\mathrm{U}}({R_{u}},e)|=U(d,r)q^{2d(n-d)}$. By Proposition 3.4 and Lemma 4.4, we complete the proof. ∎

5. GEOMETRIC CORRESPONDENCE

In this section, we establish a geometric correspondence for the matrix mentioned above to derive a stronger result.

We provide some preliminary content about Schubert varieties. For more details, see [1, 3, 7].

5.1. Schubert varieties in $G/Q$

Let $G$ be a semisimple and simply connected algebraic group defined over $k$. We use the notation in Section 2.2.1. Denote by $W_{Q}$ the Weyl group of $Q$. In each coset $wW_{Q}\in W/W_{Q}$, there exists a unique element of minimal length. Let $W^{Q}$ be the set of minimal representatives of $W/W_{Q}$. For $w\in W$, we denote the coset $wQ$ in $G/Q$ by $e_{w,Q}$. We have Bruhat decomposition

[TABLE]

Each $Be_{w,Q}$ is called a Bruhat cell, which is isomorphic to an affine space. Let $X_{Q}(w)$ be the Zariski closure of $Be_{w,Q}$ in $G/Q$. Then $X_{Q}(w)$ with the canonical reduced structure is called the Schubert variety. It is well-known that the Schubert variety $X_{Q}(w)$ can be written as $X_{Q}(w)=\bigsqcup_{u\in W^{Q},u\leq w}Be_{u,Q}$, where $\leq$ is the Bruhat order. Let $B^{-}$ be the Borel subgroup of $G$ opposite to $B$. We have opposite decomposition

[TABLE]

Here $B^{-}e_{w,Q}$ is called an opposite Bruhat cell, which is also an affine space. Consider the canonical projection $\pi:G/B\to G/Q$, we have the following properties:

• (1)

$\pi^{-1}(B^{-}e_{w,Q})=\bigsqcup_{u\in wW_{Q}}B^{-}e_{u,B}$.

• (2)

For $w\in W^{Q}$, the restriction of $\pi$ to $Be_{w,B}$ is an isomorphism onto $Be_{w,Q}$.

For $u,v\in W$, denote $X_{u}^{v}:=Be_{u,B}\cap B^{-}e_{v,B}$ to be the intersection of $Be_{u,B}$ and $B^{-}e_{v,B}$; for $u,v\in W^{Q}$, define $Y_{u}^{v}:=Be_{u,Q}\cap B^{-}e_{v,Q}$ to be the intersection of $Be_{u,Q}$ and $B^{-}e_{v,Q}$. $X_{u}^{v}$ and $Y_{u}^{v}$ are nonempty if and only if $v\leq u$. All varieties above can be defined over arbitrary field. Fix a base field $\mathbb{F}_{q}$. Consider the varieties $Y_{u}^{v}$ and $X_{u}^{v}$ that is defined over $\mathbb{F}_{q}$. We have :

Lemma 5.1**.**

$|Y_{u}^{v}(\mathbb{F}_{q})|=\sum_{w\in vW_{Q},w\leq u}|X_{u}^{w}(\mathbb{F}_{q})|$.

Proof.

Noting that the isomorphism $\pi|_{Be_{u,B}}:Be_{u,B}\to Be_{u,Q}$ is defined over $\mathbb{F}_{q}$, we have

[TABLE]

Hence

[TABLE]

∎

The $B$-orbit $Be_{w_{0},Q}$ in $G/Q$ ($w_{0}$ being the unique element of maximal length in $W$) is called the big cell in $G/Q$. It is a dense open subset of $G/Q$, and it gets identified with $R_{u}(Q)$, the unipotent radical of $Q$. Let $B^{-}$ be the Borel subgroup of $G$ opposite to $B$. The $B^{-}$-orbit $B^{-}e_{id,Q}$ is called the opposite big cell in $G/Q$. This is again a dense open subset of $G/Q$, and it gets identified with the unipotent subgroup of $B^{-}$ associated with $S_{Q}$. Observe that both the big cell and the opposite big cell can be identified with affine space.

For a Schubert variety $X_{Q}(w)\subset G/Q$, $B^{-}e_{{id},Q}\cap X_{Q}(w)$ is called the opposite cell in $X_{Q}(w)$ (by abuse of language). In general, it is not a cell. It is a non-empty open affine subvariety of $X_{Q}(w)$ and a closed subvariety of the affine space $B^{-}e_{\text{id}}$. Note that the big cell $Be_{w_{0},Q}$ being a $B$-orbit has empty intersection with any proper Schubert variety of $G/Q$. We denote $B^{-}e_{\text{id},Q}\cap X_{Q}(w)$ by $Y_{Q}(w)$.

5.1.1. The $\mathrm{Sp}_{2n}(q)$ case

Let $G=\mathrm{Sp}_{2n}(q)$ and $Q=P^{\mathrm{C},n}$. Denote by $R^{-}_{u}(Q)$ the unipotent subgroup of $B^{-}$ associated with $S_{Q}$. We already know that $R^{-}_{u}(Q)$ can be identified with the opposite big cell $B^{-}e_{id,Q}$ in $G/Q$. In this case,

[TABLE]

So, we can identify the symmetric matrices with $B^{-}e_{id,Q}$. Furthermore, the rank conditions on the matrices are given by intersecting with certain Schubert varieties $X_{Q}(w)$ ([7, Theorem 6.2.5.2]). More precisely, let $Y_{Q}(w)$ ($Y_{Q}(u)$, respectively) be identified with all symmetric matrices of rank at most $r$ ($r-1$, respectively) for a certain $w$($u\leq w$, respectively). Then, $N(n,r)$ is equal to the number of ${\mathbb{F}}_{q}$-rational points of the intersection of $B^{-}e_{id,Q}$ with $\bigsqcup_{u\textless y\leq w,y\in W^{Q}}Be_{y,Q}$. Deodhar defined a polynomial $R_{w,v}(x)$ in [2] which generalizes the $R$-polynomials of Kazhdan-Lusztig. They have the property that $R_{w,v}(q)=|B^{-}e_{v,B}\cap Be_{w,B}|$. Furthermore, $R_{w,v}(q)$ is actually a polynomial in $q-1$ with non-negative integral coefficients [2, Theorem 1.3]. Thus, $N(n,r)$ are polynomials in $q-1$ with non-negative integral coefficients by Lemma 5.1.

5.1.2. The $\mathrm{SO}_{2n}(q)$ case

Let $G=\mathrm{SO}_{2n}(q)$ and $Q=P^{\mathrm{D},n}$. We can identify the skew-symmetric matrices with $B^{-}e_{id,Q}$. This is analogous to the preceding exposition. The rank conditions on the matrices are given by intersecting with certain Schubert varieties $X_{Q}(w)$([7, Theorem 7.2.5.2]). Thus, $S(n,2s)$ are also polynomials in $q-1$ with non-negative integral coefficients.

5.1.3. The $\mathrm{U}_{2n}(q^{2})$ case

We use the notation in Section 2.2.2. Fix the integers $0\leq r\leq{n}$ and let $V=\mathbb{F}_{q^{2}}^{n}$. The Grassmannian $G_{r,n}$ is the set of $r$-dimensional subspaces $U\subseteq V$.

Let $G={GL}(n,q^{2})$, $S_{Q}=S\backslash\{\alpha_{r}\}$. We have $G/Q\cong G_{r,n}$; see [7]. We want to prove that $U(n,r)$ are polynomials in $q-1$ with non-negative integral coefficients. It is enough to prove it for

[TABLE]

The latter factor is the number of points in $G_{r,n}$. Thus, the problem becomes the question of whether $|G/Q|$ is a polynomial in $q-1$ with non-negative integral coefficients. It is a well-known result.

In summary, we have the following theorem:

Theorem 5.2**.**

The functions $N_{(X,d),e}$ are polynomials in $q-1$ with non-negative integral coefficients.