Spherical actions on isotropic flag varieties and related branching rules

TL;DR

This paper classifies spherical actions of subgroups on flag varieties of symplectic and orthogonal groups and determines the restriction rules for irreducible representations in these settings.

Contribution

It provides a complete classification of triples (G,H,X) with spherical H-actions on flag varieties and explicitly describes the restriction of G-representations to H.

Findings

Classification of all spherical triples (G,H,X) for symplectic and orthogonal groups.

Explicit restriction rules for irreducible representations from G to H.

Identification of cases with spherical subgroup actions on flag varieties.

Abstract

Let $G$ be a symplectic or special orthogonal group, let $H$ be a connected reductive subgroup of $G$, and let $X$ be a flag variety of $G$. We classify all triples $(G,H,X)$ such that the natural action of $H$ on $X$ is spherical. For each of these triples, we determine the restrictions to $H$ of all irreducible representations of $G$ realized in spaces of sections of homogeneous line bundles on $X$.

Tables

Table 1. Table 1.

$G$	Number of min. elements	Minimal elements
${\mathrm{SL}}_n$	1	$\left[\right[\mathbb{P}\left({𝔽}^n\right)\left]\right]$
${\mathrm{Sp}}_{2n}$	1	$\left[\right[\mathbb{P}\left({𝔽}^{2n}\right)\left]\right]$
${\mathrm{SO}}_{2k+1}$	1	$\left[\right[{\mathrm{SOGr}}_1\left({𝔽}^{2k+1}\right)\left]\right]$
${\mathrm{SO}}_{4k+2}$	2	$\left[\right[{\mathrm{SOGr}}_1\left({𝔽}^{4k+2}\right)\left]\right],\left[\right[{\mathrm{SOGr}}_{\max }^{+}\left({𝔽}^{4k+2}\right)\left]\right]=\left[\right[{\mathrm{SOGr}}_{\max }^{-}\left({𝔽}^{4k+2}\right)\left]\right]$
${\mathrm{SO}}_{4k}$	3	$\left[\right[{\mathrm{SOGr}}_1\left({𝔽}^{4k}\right)\left]\right],\left[\right[{\mathrm{SOGr}}_{\max }^{+}\left({𝔽}^{4k}\right)\left]\right],\left[\right[{\mathrm{SOGr}}_{\max }^{-}\left({𝔽}^{4k}\right)\left]\right]$

Table 2. Table 2.

$({\mathrm{SO}}_2,{𝔽}^2)$	$({\mathrm{SO}}_3,{𝔽}^3)$	$({\mathrm{SO}}_4,{𝔽}^4)$	$({\mathrm{SO}}_5,{𝔽}^5)$	$({\mathrm{SO}}_6,{𝔽}^6)$
$({\mathrm{GL}}_1,\Omega \left({𝔽}^1\right))$	$({\mathrm{SL}}_2,S^2{𝔽}^2)$	$({\mathrm{SL}}_2\times {\mathrm{SL}}_2,{𝔽}^2\otimes {𝔽}^2)$	$({\mathrm{Sp}}_4,{\wedge }_0^2{𝔽}^4)$	$({\mathrm{SL}}_4,{\wedge }^2{𝔽}^4)$

Table 3. Table 3.

No.	$K$	$V$	Note
$V$ irreducible
1	${\mathrm{Sp}}_{2n}$	${𝔽}^{2n}$	$n\ge 1$
$V$ weakly reducible
2	${\mathrm{GL}}_n$	$\Omega \left({𝔽}^n\right)$	$n\ge 1$
3	${\mathrm{SL}}_n$	$\Omega \left({𝔽}^n\right)$	$n\ge 3$
4	${\mathrm{Sp}}_{2n}\times {𝔽}^{\times }$	$\Omega \left({\left[{𝔽}^{2n}\right]}_{\chi }\right)$	$n\ge 2$

Table 4. Table 4.

No.	$K$	$V$	Note
$V$ irreducible
1	${\mathrm{SO}}_n$	${𝔽}^n$	$n\ge 1$, $n\ne 2$
2	${\mathrm{Sp}}_{2n}\times {\mathrm{SL}}_2$	${𝔽}^{2n}\otimes {𝔽}^2$	$n\ge 2$
3	${\mathrm{Spin}}_7$	${𝔽}^8$
4	${\mathrm{Spin}}_9$	${𝔽}^{16}$
5	${𝖦}_2$	${𝔽}^7$
$V$ weakly reducible
6	${\mathrm{GL}}_n$	$\Omega \left({𝔽}^n\right)$	$n\ge 1$
7	${\mathrm{SL}}_n$	$\Omega \left({𝔽}^n\right)$	$n\ge 3$
8	${\mathrm{Sp}}_{2n}\times {𝔽}^{\times }$	$\Omega \left({\left[{𝔽}^{2n}\right]}_{\chi }\right)$	$n\ge 2$

Table 5. Table 5. The symplectic case for I = { 1 } 𝐼 1 I=\{1\}

No.	$H$	$V$	Note
1	${\mathrm{Sp}}_{2n_1}\times \dots \times {\mathrm{Sp}}_{2n_k}$	$\underset{1\phantom{2n_1}}{{𝔽}^{2n_1}}\oplus \dots \oplus \underset{k\phantom{2n_k}}{{𝔽}^{2n_k}}$	$k\ge 1$, $n_i\ge 1$
2	${\mathrm{Sp}}_{2n_1}\times \dots \times {\mathrm{Sp}}_{2n_k}\times {\mathrm{GL}}_m$	$\underset{1\phantom{2n_1}}{{𝔽}^{2n_1}}\oplus \dots \oplus \underset{k\phantom{2n_k}}{{𝔽}^{2n_k}}\oplus \Omega \left(\underset{k+1\phantom{m}}{{𝔽}^m}\right)$	$k\ge 0$, $n_i\ge 1$, $m\ge 1$
3	${\mathrm{Sp}}_{2n_1}\times \dots \times {\mathrm{Sp}}_{2n_k}\times {\mathrm{SL}}_m$	$\underset{1\phantom{2n_1}}{{𝔽}^{2n_1}}\oplus \dots \oplus \underset{k\phantom{2n_k}}{{𝔽}^{2n_k}}\oplus \Omega \left(\underset{k+1\phantom{m}}{{𝔽}^m}\right)$	$k\ge 0$, $n_i\ge 1$, $m\ge 3$
4	${\mathrm{Sp}}_{2n_1}\times \dots \times {\mathrm{Sp}}_{2n_k}\times {\mathrm{Sp}}_{2m}\times {𝔽}^{\times }$	$\underset{1\phantom{2n_1}}{{𝔽}^{2n_1}}\oplus \dots \oplus \underset{k\phantom{2n_k}}{{𝔽}^{2n_k}}\oplus \Omega \left({\left[\underset{k+1\phantom{2m}}{{𝔽}^{2m}}\right]}_{\chi }\right)$	$k\ge 0$, $n_i\ge 1$, $m\ge 2$

Table 6. Table 6. The symplectic case for I ≠ { 1 } 𝐼 1 I\neq\{1\}

No.	Conditions	Rank	Indecomposable elements of ${\Gamma }_I(G,H)$
1	$({\mathrm{Sp}}_{2m}\times {\mathrm{SL}}_2\times {\mathrm{SL}}_2,\underset{1\phantom{2m}}{{𝔽}^{2m}}\oplus \underset{2\phantom{2}}{{𝔽}^2}\oplus \underset{3\phantom{2}}{{𝔽}^2})$, $m\ge 1$
1.1	$I=\left\{n\right\}$	$5-{\delta }_m^1$	$({\pi }_n;{\pi }_m+{\pi }_1^{\prime }+{\pi }_1^{\prime \prime })$, $({\pi }_n;{\pi }_m)$, $({\pi }_n;{\pi }_{m-1}+{\pi }_1^{\prime })$, $({\pi }_n;{\pi }_{m-1}+{\pi }_1^{\prime \prime })$, $({\pi }_n;{\pi }_{m-2})$ ($m\ge 2$)

Table 7. Table 7.

No.	$H$	$V$	Note
1	${\mathrm{Sp}}_{2l}\times {\mathrm{SL}}_2\times {\mathrm{Sp}}_{2m}$	$\underset{1\phantom{2l}}{{𝔽}^{2l}}\otimes \underset{2\phantom{2}}{{𝔽}^2}\oplus \underset{3\phantom{2m}}{{𝔽}^{2m}}\otimes \underset{2\phantom{2}}{{𝔽}^2}$	$l,m\ge 1$
2	${\mathrm{Spin}}_8$	${𝔽}_{+}^8\oplus {𝔽}_{-}^8$
3	${\mathrm{SL}}_l\times {\mathrm{SL}}_m\times {𝔽}^{\times }$	$\Omega \left({\left[\underset{1\phantom{l}}{{𝔽}^l}\right]}_{a\chi }\right)\oplus \Omega \left({\left[\underset{2\phantom{m}}{{𝔽}^m}\right]}_{b\chi }\right)$, $a,b\in \mathbb{Z}\setminus \left\{0\right\}$	$l,m\ge 3$
4	${\mathrm{SL}}_l\times {\mathrm{Sp}}_{2m}\times {𝔽}^{\times }$	$\Omega \left({\left[\underset{1\phantom{l}}{{𝔽}^l}\right]}_{a\chi }\right)\oplus \Omega \left({\left[\underset{2\phantom{2m}}{{𝔽}^{2m}}\right]}_{b\chi }\right)$, $a,b\in \mathbb{Z}\setminus \left\{0\right\}$	$l\ge 3$, $m\ge 2$

Table 8. Table 8. The odd orthogonal case for I ≠ { 1 } 𝐼 1 I\neq\{1\}

No.	Conditions	Rank	Indecomposable elements of ${\Gamma }_I(G,H)$
1	$({𝖦}_2,{𝔽}^7)$
1.1	$I=\{1,2\}$	$4$	$({\pi }_1;{\pi }_1)$, $({\pi }_2;{\pi }_1)$, $({\pi }_2;{\pi }_2)$, $({\pi }_1+{\pi }_2;{\pi }_2)$
1.2	$I=\{1,3\}$	$4$	$({\pi }_1;{\pi }_1)$, $({\pi }_3;{\pi }_1)$, $({\pi }_3;0)$, $({\pi }_1+{\pi }_3;{\pi }_2)$
1.3	$I=\{2,3\}$	$4$	$({\pi }_2;{\pi }_1)$, $({\pi }_2;{\pi }_2)$, $({\pi }_3;{\pi }_1)$, $({\pi }_3;0)$
2	$({\mathrm{Spin}}_7,{𝔽}^8\oplus {𝔽}^1)$
2.1	$I=\{1,2\}$	5	$({\pi }_1;{\pi }_3)$, $({\pi }_1;0)$, $({\pi }_2;{\pi }_1)$, $({\pi }_2;{\pi }_2)$, $({\pi }_2;{\pi }_3)$
2.2	$I=\left\{3\right\}$	4	$({\pi }_3;{\pi }_1)$, $({\pi }_3;{\pi }_2)$, $({\pi }_3;{\pi }_3)$, $({\pi }_3;{\pi }_1+{\pi }_3)$
2.3	$I=\left\{4\right\}$	3	$({\pi }_4;{\pi }_1)$, $({\pi }_4;0)$, $({\pi }_4;{\pi }_3)$

Table 9. Table 9. The even orthogonal case for I ≠ { 1 } 𝐼 1 I\neq\{1\}

No.	Conditions	Rank	Indecomposable elements of ${\Gamma }_I(G,H)$
1	$({\mathrm{SO}}_{2l}\times {\mathrm{SO}}_{2m+1},\underset{1\phantom{2l}}{{𝔽}^{2l}}\oplus \underset{2\phantom{2m+1}}{{𝔽}^{2m+1}}\oplus {𝔽}^1)$, $l\ge 2$, $m\ge 1$, $2l+2m+2=2n$
1.1	$I=\left\{n\right\}$	$\min (2l,2m+1)+1$	$({\pi }_n;{\pi }_{l-1}+{\pi }_m^{\prime })$, $({\pi }_n;{\pi }_l+{\pi }_m^{\prime })$, $(2{\pi }_n;{\pi }_{l-1}+{\pi }_l+{\pi }_{m-1}^{\prime })$, $(2{\pi }_n;{\pi }_{l-k}+{\pi }_{m-k}^{\prime })$ for $2\le k\le \left[\frac{\min (2l,2m+1)}{2}\right]$, $(2{\pi }_n;{\pi }_{l-k}+{\pi }_{m+1-k}^{\prime })$ for $2\le k\le \left[\frac{\min (2l,2m+1)+1}{2}\right]$
2	$({\mathrm{SO}}_{2m+1}\times {𝔽}^{\times },{𝔽}^{2m+1}\oplus \Omega \left({𝔽}_{\chi }^1\right)\oplus {𝔽}^1)$, $m\ge 2$, $2m+4=2n$
2.1	$I=\left\{n\right\}$	3	$({\pi }_n;{\pi }_m+\frac{\chi }{2})$, $({\pi }_n;{\pi }_m-\frac{\chi }{2})$, $(2{\pi }_n;{\pi }_{m-1})$
3	$({\mathrm{Spin}}_7\times {\mathrm{SO}}_{2l},\underset{1\phantom{8}}{{𝔽}^8}\oplus \underset{2\phantom{2l}}{{𝔽}^{2l}})$, $l\ge 2$, $2l+8=2n$
3.1	$I=\left\{n\right\}$	$\min (7,2l)+1$	$({\pi }_n;{\pi }_3+{\pi }_l^{\prime })$, $({\pi }_n;{\pi }_1+{\pi }_{l-1}^{\prime })$, $({\pi }_n;{\pi }_{l-1}^{\prime })$, $(2{\pi }_n;{\pi }_2+{\pi }_{l-2}^{\prime })$, $(2{\pi }_n;{\pi }_1+{\pi }_{l-2}^{\prime })$, $(2{\pi }_n;{\pi }_1+{\pi }_{l-3}^{\prime })$, ($l\ge 3$), $(2{\pi }_n;{\pi }_{l-4}^{\prime })$ ($l\ge 4$), $(3{\pi }_n;{\pi }_2+{\pi }_l^{\prime }+{\pi }_{l-3}^{\prime })$ ($l\ge 3$)
4	$({\mathrm{Spin}}_7\times {𝔽}^{\times },{𝔽}^8\oplus \Omega \left({𝔽}_{\chi }^1\right))$
4.1	$I=\left\{2\right\}$	5	$({\pi }_2;{\pi }_1)$, $({\pi }_2;{\pi }_2)$, $({\pi }_2;0)$, $({\pi }_2;{\pi }_3+\chi )$, $({\pi }_2;{\pi }_3-\chi )$
4.2	$I=\left\{5\right\}$	3	$({\pi }_5;{\pi }_1+\frac{\chi }{2})$, $({\pi }_5,\frac{\chi }{2})$, $({\pi }_5;{\pi }_3-\frac{\chi }{2})$
5	$({\mathrm{Spin}}_7,{𝔽}^8\oplus {𝔽}^1\oplus {𝔽}^1)$
5.1	$I=\left\{5\right\}$	3	$({\pi }_5;{\pi }_1)$, $({\pi }_5;0)$, $({\pi }_5;{\pi }_3)$
6	$({\mathrm{Spin}}_7\times {\mathrm{SL}}_2\times {𝔽}^{\times },\underset{1\phantom{8}}{{𝔽}^8}\oplus \Omega \left({\left[\underset{2\phantom{2}}{{𝔽}^2}\right]}_{\chi }\right))$
6.1	$I=\left\{6\right\}$	6	$({\pi }_6;{\pi }_1+{\pi }_1^{\prime }+\chi )$, $({\pi }_6;{\pi }_1^{\prime }+\chi )$, $({\pi }_6;{\pi }_3+\chi )$, $({\pi }_6;{\pi }_3-\chi )$, $(2{\pi }_6;{\pi }_1+\chi )$, $(2{\pi }_6;2\chi )$
7	$({𝖦}_2\times {\mathrm{SO}}_3,\underset{1\phantom{7}}{{𝔽}^7}\oplus \underset{2\phantom{3}}{{𝔽}^3})$
7.1	$I=\left\{5\right\}$	4	$({\pi }_5;{\pi }_1+{\pi }_1^{\prime })$, $({\pi }_5;{\pi }_1^{\prime })$, $(2{\pi }_5;{\pi }_1)$, $(2{\pi }_5;{\pi }_2)$
8	$({𝖦}_2,{𝔽}^7\oplus {𝔽}^1)$
8.1	$I=\left\{4\right\}$	2	$({\pi }_4;{\pi }_1)$, $({\pi }_4;0)$

Table 10. Table 10.

No.	$H$	$V$	$dim{\mathrm{SOGr}}_1\left(V\right)$	$dim{B}_H$	Note
1	${\mathrm{SO}}_n$	${𝔽}^n$	$n-2$	$\left[\frac{n}{2}\right]\left[\frac{n+1}{2}\right]$	$n\ge 3$
2	${\mathrm{Sp}}_{2n}\times {\mathrm{SL}}_2$	${𝔽}^{2n}\otimes {𝔽}^2$	$4n-2$	$n^2+n+2$	$n\ge 2$
3	${𝖦}_2$	${𝔽}^7$	$5$	$8$
4	${\mathrm{Spin}}_7$	${𝔽}^8$	$6$	$12$
5	${\mathrm{Spin}}_9$	${𝔽}^{16}$	$14$	$20$
6	${\mathrm{SL}}_3$	$R\left({\pi }_1+{\pi }_2\right)$	$6$	$5$
7	${\mathrm{Sp}}_4$	$S^2{𝔽}^4$	$8$	$6$
8	${𝖦}_2$	$R\left({\pi }_2\right)$	$12$	$8$
9	${\mathrm{SL}}_2$	$S^4{𝔽}^2$	$3$	$2$
10	${\mathrm{Sp}}_6$	${\wedge }_0^2{𝔽}^6$	$12$	$12$
11	${𝖥}_4$	$R\left({\pi }_4\right)$	$24$	$28$
12	${\mathrm{Sp}}_{2n}\times {\mathrm{Sp}}_4$	${𝔽}^{2n}\otimes {𝔽}^4$	$8n-2$	$n^2+n+6$	$n\ge 2$
13	${\mathrm{SL}}_2\times {\mathrm{SL}}_2$	${𝔽}^2\otimes S^3{𝔽}^2$	$6$	$4$

Table 11. Table 11.

No.	$H$	$W$	$dim{\mathrm{SOGr}}_1\left(V\right)$	$dim{B}_H$	Note
1	${\mathrm{GL}}_n$	${𝔽}^n$	$2n-2$	$\frac{n\left(n+1\right)}{2}$	$n\ge 2$
2	${\mathrm{SL}}_n$	${𝔽}^n$	$2n-2$	$\frac{n\left(n+1\right)}{2}-1$	$n\ge 2$
3	${\mathrm{Sp}}_{2n}\times {𝔽}^{\times }$	${\left[{𝔽}^{2n}\right]}_{\chi }$	$4n-2$	$n^2+n+1$	$n\ge 2$
4	${\mathrm{Sp}}_{2n}$	${𝔽}^{2n}$	$4n-2$	$n^2+n$	$n\ge 2$
5	${\mathrm{SO}}_n\times {𝔽}^{\times }$	${\left[{𝔽}^n\right]}_{\chi }$	$2n-2$	$\left[\frac{n}{2}\right]\left[\frac{n+1}{2}\right]+1$	$n\ge 3$
6	${\mathrm{GL}}_n\times {\mathrm{SL}}_2$	${𝔽}^n\otimes {𝔽}^2$	$4n-2$	$\frac{n\left(n+1\right)}{2}+2$	$n\ge 3$
7	${\mathrm{SL}}_n\times {\mathrm{SL}}_2$	${𝔽}^n\otimes {𝔽}^2$	$4n-2$	$\frac{n\left(n+1\right)}{2}+1$	$n\ge 3$
8	${\mathrm{Spin}}_7\times {𝔽}^{\times }$	${\left[{𝔽}^8\right]}_{\chi }$	$14$	$13$
9	${𝖦}_2\times {𝔽}^{\times }$	${\left[{𝔽}^7\right]}_{\chi }$	$12$	$9$
10	${\mathrm{GL}}_5$	${\wedge }^2{𝔽}^5$	$18$	$15$
11	${\mathrm{SL}}_5$	${\wedge }^2{𝔽}^5$	$18$	$14$
12	${\mathrm{Spin}}_{10}\times {𝔽}^{\times }$	${\left[{𝔽}^{16}\right]}_{\chi }$	$30$	$26$
13	${\mathrm{Spin}}_{10}$	${𝔽}^{16}$	$30$	$25$

Table 12. Table 12.

No.	$H$	$V$	$dim{\mathrm{SOGr}}_{\max }\left(V\right)$	$dim{B}_H$	Note
1	${\mathrm{SO}}_{2n}$	${𝔽}^{2n}$	$\frac{n\left(n-1\right)}{2}$	$n^2$	$n\ge 2$
2	${\mathrm{Sp}}_{2n}\times {\mathrm{SL}}_2$	${𝔽}^{2n}\otimes {𝔽}^2$	$2n^2-n$	$n^2+n+2$	$n\ge 2$
3	${\mathrm{Spin}}_7$	${𝔽}^8$	$6$	$12$
4	${\mathrm{Spin}}_9$	${𝔽}^{16}$	$28$	$20$
5	${\mathrm{SL}}_3$	$R\left({\pi }_1+{\pi }_2\right)$	$6$	$5$
6	${\mathrm{Sp}}_4$	$S^2{𝔽}^4$	$10$	$6$
7	${\mathrm{Sp}}_6$	${\wedge }_0^2{𝔽}^6$	$21$	$12$

Table 13. Table 13.

No.	$(H,W)$	$\ast$	Note
1	$({\mathrm{GL}}_n,{𝔽}^n)$	$\pm$	$n\ge 2$
2	$({\mathrm{SL}}_{2n+1},{𝔽}^{2n+1})$	$+$	$n\ge 1$
3	$({\mathrm{SL}}_n,{𝔽}^n)$	$-$	$n\ge 2$
4	$({\mathrm{SO}}_3\times {𝔽}^{\times },{\left[{𝔽}^3\right]}_{\chi })$	$\pm$
5	$({\mathrm{Sp}}_4\times {𝔽}^{\times },{\left[{𝔽}^4\right]}_{\chi })$	$-$

Table 14. Table 14.

No.	$(H,W)$	$\ast$	$(M,𝔤/\left({𝔭}_I^{-}+𝔥\right))$	Conclusion
1	$({\mathrm{GL}}_n,{𝔽}^n)$, $n\ge 2$	$+$	$({\mathrm{GL}}_n,{\wedge }^2{𝔽}^n)$	spherical
		$-$	$({\mathrm{GL}}_{n-1}\times {𝔽}^{\times },{\wedge }^2{𝔽}^{n-1})$	spherical
2	$({\mathrm{SL}}_n,{𝔽}^n)$, $n\ge 2$	$+$	$({\mathrm{SL}}_n,{\wedge }^2{𝔽}^n)$	spherical iff $n$ is odd
		$-$	$({\mathrm{GL}}_{n-1},{\wedge }^2{𝔽}^{n-1})$	spherical
3	$({\mathrm{SO}}_3\times {𝔽}^{\times },{\left[{𝔽}^3\right]}_{\chi })$	$+$	$({\mathrm{SO}}_3\times {𝔽}^{\times },{\left[{𝔽}^3\right]}_{\chi })$	spherical
		$-$	$({𝔽}^{\times }\times {𝔽}^{\times },{𝔽}_{{\chi }_1}^1\oplus {𝔽}_{{\chi }_1+{\chi }_2}^1)$	spherical
4	$({\mathrm{Sp}}_4\times {𝔽}^{\times },{\left[{𝔽}^4\right]}_{\chi })$	$+$	$({\mathrm{Sp}}_4\times {𝔽}^{\times },{\left[{\wedge }^2{𝔽}^4\right]}_{\chi })$	not spherical
		$-$	$({\mathrm{SL}}_2\times {𝔽}^{\times }\times {𝔽}^{\times },{\left[{𝔽}^2\right]}_{{\chi }_1+{\chi }_2}\oplus {𝔽}_{{\chi }_1}^1)$	spherical

Table 15. Table 15.

No.	$(H,V)$	$\ast$	Note
1	$({\mathrm{SO}}_p\times {\mathrm{SO}}_q,\underset{1\phantom{p}}{{𝔽}^p}\oplus \underset{2\phantom{q}}{{𝔽}^q})$	$\pm$	$p,q\ge 1$, $p+q=2k\ge 6$
2	$({\mathrm{Spin}}_7\times {\mathrm{SO}}_{2l},\underset{1\phantom{8}}{{𝔽}^8}\oplus \underset{2\phantom{2l}}{{𝔽}^{2l}})$	$\pm$	$l\ge 1$
3	$({\mathrm{GL}}_m\times {𝔽}^{\times },\Omega \left({𝔽}^m\right)\oplus \Omega \left({𝔽}_{\chi }^1\right))$	$\pm$	$m\ge 2$
4	$({\mathrm{SL}}_{2m+1}\times {𝔽}^{\times },\Omega \left({𝔽}^{2m+1}\right)\oplus \Omega \left({𝔽}_{\chi }^1\right))$	$\pm$	$m\ge 1$
5	$({\mathrm{SL}}_{2m}\times {𝔽}^{\times },\Omega \left({\left[{𝔽}^{2m}\right]}_{a\chi }\right)\oplus \Omega \left({𝔽}_{b\chi }^1\right))$, $a,b\in \mathbb{Z}\setminus \left\{0\right\}$	$\pm$	$m\ge 1$
6	$({\mathrm{SL}}_{2m+1}\times {𝔽}^{\times },\Omega \left({\left[{𝔽}^{2m+1}\right]}_{a\chi }\right)\oplus \Omega \left({𝔽}_{b\chi }^1\right))$, $a,b\in \mathbb{Z}\setminus \left\{0\right\}$, $b\ne -\left(2m+1\right)a$	$+$	$m\ge 1$
7	$({\mathrm{SL}}_{2m+1}\times {𝔽}^{\times },\Omega \left({\left[{𝔽}^{2m+1}\right]}_{a\chi }\right)\oplus \Omega \left({𝔽}_{b\chi }^1\right))$, $a,b\in \mathbb{Z}\setminus \left\{0\right\}$, $b\ne \left(2m+1\right)a$	$-$	$m\ge 1$
8	$({\mathrm{GL}}_3\times {\mathrm{SO}}_{2l},\Omega \left(\underset{1\phantom{3}}{{𝔽}^3}\right)\oplus \underset{2\phantom{2l}}{{𝔽}^{2l}})$	$\pm$	$l\ge 2$
9	$({\mathrm{GL}}_2\times {\mathrm{SO}}_{2l},\Omega \left(\underset{1\phantom{2}}{{𝔽}^2}\right)\oplus \underset{2\phantom{2l}}{{𝔽}^{2l}})$	$\pm$	$l\ge 2$
10	$({𝖦}_2,{𝔽}^7\oplus {𝔽}^1)$	$\pm$
11	$({𝖦}_2\times {\mathrm{SO}}_3,\underset{1\phantom{7}}{{𝔽}^7}\oplus \underset{2\phantom{3}}{{𝔽}^3})$	$\pm$
12	$({\mathrm{SL}}_2\times {\mathrm{SL}}_2\times {\mathrm{SL}}_2,\underset{1\phantom{2}}{{𝔽}^2}\otimes \underset{3\phantom{2}}{{𝔽}^2}\oplus \underset{2\phantom{2}}{{𝔽}^2}\otimes \underset{3\phantom{2}}{{𝔽}^2})$	$+$	$H_0\subset {\mathrm{Spin}}_7^{+}\subset \mathrm{SO}\left(V\right)$
13	$({\mathrm{Spin}}_7\times {\mathrm{GL}}_2,\underset{1\phantom{8}}{{𝔽}^8}\oplus \Omega \left(\underset{2\phantom{2}}{{𝔽}^2}\right))$	$+$

Equations

$$({\mathrm{SL}}_2\times {\mathrm{SL}}_2\times {\mathbb{F}}^{\times }\times {\mathbb{F}}^{\times },[\underset{1}{{\mathbb{F}}^2}\otimes \underset{2}{{\mathbb{F}}^2}{]}_{{\chi }_1+{\chi }_2}\oplus \underset{1}{[{\mathbb{F}}^2}{]}_{2{\chi }_1}\oplus \underset{2}{{\mathrm{S}}^2{\mathbb{F}}^2}\oplus {\mathbb{F}}_{2{\chi }_2}^1).$$

$$H^0(X_I,L(\lambda ))\simeq \{\begin{array}{ll}{R}_G(\lambda )& \text{if}\lambda \in {\Lambda }_I^{+}(G);\\ 0& \text{otherwise};\end{array}$$

$$V_1\oplus \dots \oplus V_p\oplus \Omega (W_1)\oplus \dots \oplus \Omega (W_q)$$

$${\Lambda }_X=\{\lambda \in \mathfrak{X}(T_K)\mid \mathbb{F}(X{)}_{\lambda }^{(B)}\ne \{0\}\}.$$

$$\Gamma (G,H)=\{(\lambda ;\mu )\in {\Lambda }^{+}(G)\times {\Lambda }^{+}(H)\mid R_H(\mu )\text{is a submodule of}{R_G(\lambda )|}_H\}.$$

$${\Gamma }_I(G,H)=\{(\lambda ;\mu )\in \Gamma (G,H)\mid \lambda \in {\Lambda }_I^{+}(G)\}$$

$${R_G(\lambda )|}_H\simeq \underset{\mu \in {\Lambda }^{+}(H):(\lambda ;\mu )\in {\Gamma }_I(G,H)}{⨁}{R}_H(\mu ).$$

$${R_G(\sigma (\lambda ))|}_{\sigma (H)}\simeq \underset{\mu \in {\Lambda }^{+}(H):(\lambda ;\mu )\in {\Gamma }_I(G,H)}{⨁}{R}_{\sigma (H)}(\sigma (\mu )).$$

$$({\mathrm{GL}}_{n_1}\times \dots \times {\mathrm{GL}}_{n_k}\times {\mathrm{Sp}}_{2m},\Omega (\underset{1}{{\mathbb{F}}^{n_1}})\oplus \Omega (\underset{2}{{\mathbb{F}}^{n_2}})\oplus \Omega (\underset{k}{{\mathbb{F}}^{n_k}})\oplus \underset{k+1}{{\mathbb{F}}^{2m}})$$

$$({\mathrm{GL}}_{n_1}\times \dots \times {\mathrm{GL}}_{n_k}\times {\mathrm{SO}}_m,\Omega (\underset{1}{{\mathbb{F}}^{n_1}})\oplus \Omega (\underset{2}{{\mathbb{F}}^{n_2}})\oplus \Omega (\underset{k}{{\mathbb{F}}^{n_k}})\oplus \underset{k+1}{{\mathbb{F}}^m})$$

$${\Lambda }^{+}(H\times {\mathbb{F}}^{\times })\to {\Lambda }_{\{1\}}^{+}(G)\times {\Lambda }^{+}(H),\lambda =\mu +k\delta \mapsto \overline{\lambda }=(k{\pi }_1;\mu ).$$

$$\{\overline{\lambda }\mid \lambda \in {\mathrm{E}}_{H\times {\mathbb{F}}^{\times }}^0(V^{\ast })\setminus \{2\delta \}\}.$$

$${\Lambda }^{+}(H\times {\mathbb{F}}^{\times }\times {\mathbb{F}}^{\times })\to {\Lambda }_{\{1\}}^{+}(G)\times {\Lambda }^{+}(H),\lambda =\mu +k_1{\delta }_1+k_2{\delta }_2\mapsto \overline{\lambda }=((k_1+k_2){\pi }_1;\mu ).$$

$$\{\overline{\lambda }\mid \lambda \in {\mathrm{E}}_{H\times {\mathbb{F}}^{\times }\times {\mathbb{F}}^{\times }}^0(V^{\ast })\}.$$

$$\mathcal{N}(G/P)=G{O}_P\subset \mathfrak{g}.$$

$$a_1+\dots +a_i\le b_1+\dots +b_i\text{for all}i=1,\dots ,d.$$

$${\mathcal{P}}_{\epsilon }(d)=\{\begin{array}{ll}\{\mathbf{a}\in \mathcal{P}(d)\mid \text{each odd part of a has even multiplicity}\}& \text{if}\epsilon =-1;\\ \{\mathbf{a}\in \mathcal{P}(d)\mid \text{each even part of a has even multiplicity}\}& \text{if}\epsilon =1.\end{array}$$

$$I=\{a_1,a_1+a_2,\dots ,a_1+\dots +a_{[p/2]}\}.$$

$$I=\{\begin{array}{ll}\{a_1,a_1+a_2,\dots ,a_1+\dots +a_{[p/2]}\}& \text{if}{a}_{[p/2]+1}\ge 4;\\ \{a_1,a_1+a_2,\dots ,a_1+\dots +a_{[p/2]-1},n-1,n\}& \text{if}{a}_{[p/2]+1}=2;\end{array}$$

$$I=\{a_1,a_1+a_2,\dots a_1+\dots +a_{q-1},n-1,n\};$$

$$I=\{a_1,a_1+a_2,\dots ,a_1+\dots +a_{q-1},n\}$$

$$I=\{a_1,a_1+a_2,\dots ,a_1+\dots +a_{q-1},n-1\}.$$

$$m=\max (\{0\}\cup \{i\ge 1\mid (a_1,\dots ,a_i)\in {\mathcal{P}}_{\epsilon }(a_1+\dots +a_i)\}).$$

$$l=\min \{j\ge m+2\mid \epsilon (-1{)}^{a_j}=1\},$$

$${\hat{a}}_i=|\{j\mid a_j\ge i\}|,i=1,\dots ,q.$$

$$\mbox{$[[$}\operatorname{SOGr}_{1}(V)\mbox{$]]$}=\{\operatorname{SOGr}_{1}(V),\operatorname{SOGr}_{\max}(V)\}$$

$$\mbox{$[[$}\operatorname{SOGr}_{2}(V)\mbox{$]]$}=\{\operatorname{SOGr}_{2}(V),\operatorname{SOGr}_{3}(V),\operatorname{SOFl}^{+}_{(1,3,3,1)}(V),\operatorname{SOFl}^{-}_{(1,3,3,1)}(V)\}$$

$$\mbox{$[[$}\operatorname{SOGr}_{2}(V)\mbox{$]]$}=\{\operatorname{SOGr}_{2}(V),\operatorname{SOFl}^{+}_{(1,2,2,1)}(V),\operatorname{SOFl}^{-}_{(1,2,2,1)}(V)\}$$

$$\mathrm{rk}{\Gamma }_I(G,H)=|I|+{\mathrm{rk}}_M(\mathfrak{g}/({\mathfrak{p}}_I^{-}+\mathfrak{h})).$$

Full text

1. Introduction

Throughout this paper, we work over an algebraically closed field $\mathbb{F}$ of characteristic zero. The notation $\mathbb{F}^{\times}$ stands for the multiplicative group of $\mathbb{F}$.

Let $H$ be a connected reductive algebraic group and let $X$ be an $H$-variety (that is, an algebraic variety equipped with a regular action of $H$). The action of $H$ on $X$, as well as $X$ itself, is said to be spherical (or $H$-spherical if one needs to emphasize the acting group) if $X$ possesses a dense open orbit with respect to the induced action of a Borel subgroup of $H$. When $X$ is a flag variety of a connected semisimple group $G$ containing $H$ as a subgroup, a result of Vinberg and Kimelfeld [ViKi] asserts that $X$ is $H$-spherical if and only if, for every irreducible representation $R$ of $G$ realized in the space of sections of a homogeneous line bundle on $X$, the restriction of $R$ to $H$ is multiplicity free. In view of the importance of the latter representation-theoretic property, this result naturally raises the following two problems:

1. (P1)

classify all triples $(G,H,X)$ such that $X$ is $H$-spherical; 2. (P2)

for each such triple $(G,H,X)$ determine the restrictions to $H$ of all irreducible representations of $G$ realized in spaces of sections of homogeneous line bundles on $X$.

By now, problem (P1) has been solved in the following particular cases (in all of them $G$ is assumed to be simple):

1. (C1)

$H$ is a Levi subgroup of $G$ (with contributions of [Lit, MWZ1, MWZ2, Stem], see also [Pon1]); 2. (C2)

$H$ is a symmetric subgroup of $G$ (see [HNOO]); 3. (C3)

$G=\operatorname{SL}_{n}$ (see [AvPe1]); 4. (C4)

$G$ is an exceptional simple group, $H$ is a maximal reductive subgroup of $G$, and $X=G/P$ with $P$ a maximal parabolic subgroup of $G$ (see the preprint [Nie]).

In all these cases, problem (P2) has also been already solved. Namely, in case (C1) a solution follows from results of the papers [Pon2], [Pon3] (see details in [AvPe2]), cases (C2) and (C3) were completed in [AvPe2], and case (C4) was settled in [Nie].

In this paper, which may be regarded as a continuation of [AvPe1] and [AvPe2], we solve problems (P1) and (P2) in the cases $G=\operatorname{Sp}_{2n}$ and $G=\operatorname{SO}_{n}$; our results are stated in §§ 4.3, 4.4. In particular, we complete solutions of problems (P1) and (P2) for all the classical simple algebraic groups. Below we outline the main ideas of the employed approaches and obtained results.

To solve problem (P1), we apply a general strategy developed in [AvPe1] (see § 5.1 for details). Let $\mathscr{F}(G)$ be the set of all nontrivial flag varieties for $G$ and fix an arbitrary connected reductive subgroup $H\subset G$. Given $X\in\mathscr{F}(G)$ such that $X=G/P$ for a parabolic subgroup $P\subset G$, let $\mathscr{N}(X)$ denote the Richardson nilpotent orbit in $\mathfrak{g}=\operatorname{Lie}G$ defined by $P$. We say that two varieties $X_{1},X_{2}\in\mathscr{F}(G)$ are nil-equivalent (notation $X_{1}\sim X_{2}$) if $\mathscr{N}(X_{1})=\mathscr{N}(X_{2})$. Then it turns out that for a given flag variety the condition of being $H$-spherical depends only on its nil-equivalence class: if $X_{1},X_{2}\in\mathscr{F}(G)$ and $X_{1}\sim X_{2}$ then $X_{1}$ is $H$-spherical if and only if $X_{2}$ is so. Next, for every $X\in\mathscr{F}(G)$ let \mbox{[[}X\mbox{]]} denote the image of $X$ in the quotient set $\mathscr{F}(G)/\sim$. The inclusion relation on the set of nilpotent orbits in $\mathfrak{g}$ determines a partial order $\preccurlyeq$ on $\mathscr{F}(G)/\sim$ as follows: we write \mbox{[[}X_{1}\mbox{]]}\preccurlyeq\mbox{[[}X_{2}\mbox{]]} if and only if $\mathscr{N}(X_{1})$ is contained in the closure of $\mathscr{N}(X_{2})$. This partial order has the following remarkable property: if $X_{1},X_{2}\in\mathscr{F}(G)$, \mbox{[[}X_{1}\mbox{[[}\preccurlyeq\mbox{[[}X_{2}\mbox{]]}, and $X_{2}$ is $H$-spherical then $X_{1}$ is also $H$-spherical. In other words, the property of being $H$-spherical ‘‘spreads’’ to smaller nil-equivalence classes in $\mathscr{F}(G)/\sim$. It follows that a natural starting point for solving problem (P1) for a given group $G$ is to determine all the minimal nil-equivalence classes in $\mathscr{F}(G)/\sim$ and classify all spherical actions on the corresponding flag varieties.

If $G$ is one of the classical groups $\operatorname{SL}_{n}$, $\operatorname{Sp}_{2n}$, or $\operatorname{SO}_{n}$ then the nilpotent orbits in $\mathfrak{g}$, the inclusion relation between their closures, and the map $X\mapsto\mathscr{N}(X)$ admit an effective combinatorial description in terms of partitions (see, for instance, [CM]; a summary of these results is presented in [AvPe1, § 3] for $G=\operatorname{SL}_{n}$ and in § 5.4 for $G=\operatorname{Sp}_{2n}$ and $\operatorname{SO}_{n}$). Using this description, it is easy to determine the minimal elements of $\mathscr{F}(G)/\sim$; see details in §§ 5.5–5.7. The results are presented in Table 1 where $\operatorname{SOGr}_{1}(\mathbb{F}^{n})\subset\mathbb{P}(\mathbb{F}^{n})$ is the variety of isotropic lines in $\mathbb{F}^{n}$ and $\operatorname{SOGr}_{\max}^{\pm}(\mathbb{F}^{2n})$ are the two connected components of the variety of isotropic subspaces in $\mathbb{F}^{2n}$ of (maximal possible) dimension $n$. (Isotropic subspaces are taken with respect to the bilinear form defining the orthogonal group.)

For a finite-dimensional $H$-module $U$, it is easy to see that $H$ acts spherically on the projective space $\mathbb{P}(U)$ if and only if $H\times\mathbb{F}^{\times}$ acts spherically on $U$ (where the action of $\mathbb{F}^{\times}$ is by scalar transformations). More generally, given a connected reductive group $K$, a finite-dimensional $K$-module $V$ is said to be spherical if $V$ is spherical as a $K$-variety. There is a complete classification of all spherical modules obtained in [Kac], [BeRa], and [Lea] (see § 3.5 for more details). According to the above discussion, this classification provides a description of all connected reductive subgroups of $\operatorname{SL}_{n}$ that act spherically on $\mathbb{P}(\mathbb{F}^{n})$, which was a starting point for solving problem (P1) for $G=\operatorname{SL}_{n}$ in [AvPe1]. Likewise, in the present paper we apply the classification of spherical modules to determine all connected reductive subgroups of $\operatorname{Sp}_{2n}$ acting spherically on $\mathbb{P}(\mathbb{F}^{2n})$, which provides a starting point for solving problem (P1) in the case $G=\operatorname{Sp}_{2n}$. We note that the list of such subgroups turns out to be very short (see Theorem 6.2), and because of this our classification for $G=\operatorname{Sp}_{2n}$ turns out to be much easier than that for $G=\operatorname{SO}_{n}$.

In the case $G=\operatorname{SO}_{n}$ it is a much more complicated task to describe all spherical actions on flag varieties whose nil-equivalent classes are minimal elements of $\mathscr{F}(G)/\sim$. For an action of $H$ on $X\in\mathscr{F}(G)$ to be spherical, a necessary condition is that $H$ has an open orbit in $X$. If $X$ is a variety of isotropic subspaces of $\mathbb{F}^{n}$ of a fixed dimension (such varieties are often called isotropic Grassmannians or Grassmannians of isotropic subspaces), then the work of Kimelfeld [Kim] provides a classification of connected reductive subgroups $H\subset\operatorname{SO}_{n}$ having an open orbit in $X$ and admitting no proper $H$-stable subspaces in $\mathbb{F}^{n}$ that are nondegenerate with respect to the bilinear form defining the group $\operatorname{SO}_{n}$. Although this classification deals with a rather particular situation, starting from it and using various reductions we ultimately manage to deduce classifications of all spherical actions on the varieties $\operatorname{SOGr}_{1}(\mathbb{F}^{n})$ and $\operatorname{SOGr}_{\max}^{(\pm)}(\mathbb{F}^{n})$ and then complete the whole classification in the case $G=\operatorname{SO}_{n}$.

Our results in solving problem (P1) for $G=\operatorname{Sp}_{2n}$ or $\operatorname{SO}_{n}$ show that the overwhelming majority of spherical actions on flag varieties occurs in the case where the variety acted on is a Grassmannian of isotropic subspaces of dimension 1, or 2, or maximal possible. In particular, this means that the major part of our classification is concentrated in the analysis of flag varieties whose nil-equivalence classes are either minimal elements of $\mathscr{F}(G)/\sim$ or ‘‘close’’ to minimal. It is also worth mentioning that, if $G=\operatorname{Sp}_{2n}$ or $G=\operatorname{SO}_{2k+1}$ and $X$ is not the Grassmannian of isotropic lines, then there are only a few cases of spherical actions on $X$ where $H$ is neither intermediate between a Levi subgroup of $G$ and its derived subgroup nor a symmetric subgroup of $G$, see Theorems 4.6 and 4.10.

Although the general strategy for solving problem (P1) used in this paper is the same as in [AvPe1], for checking $H$-sphericity of a given flag variety $X$ of $G$ we use an effective criterion that is a consequence of a general result of Panyushev [Pan]. Namely, from $H$ and $X$ one computes explicitly a Levi subgroup $M$ of $H$ together with a finite-dimensional $M$-module $U$, and it turns out that $X$ is $H$-spherical if and only if $U$ is a spherical $M$-module (see Proposition 5.17).

For solving problem (P2) we use techniques described in [AvPe2, § 4]. Given an $H$-spherical flag variety $X$ of $G$, the restrictions to $H$ of all irreducible representations of $G$ realized in spaces of sections of homogeneous line bundles on $X$ are encoded in a free monoid of finite rank, which we call the restricted branching monoid. In the cases $G=\operatorname{Sp}_{2n}$, $X=\mathbb{P}(\mathbb{F}^{2n})$ and $G=\operatorname{SO}_{n}$, $X=\operatorname{SOGr}_{1}(\mathbb{F}^{n})$, the description of this monoid follows from well-known facts. For the remaining cases, the restricted branching monoids are determined as follows. First, the rank of the monoid is easily computed from the spherical $M$-module $U$ mentioned in the previous paragraph. Second, to find all the indecomposable elements of the monoid, it suffices to explicitly compute the restrictions to $H$ of several irreducible representations of $G$ with ‘‘small’’ highest weights $\lambda$, where $\lambda$ is usually a fundamental weight or the sum of two (not necessarily distinct) fundamental weights of $G$. Luckily, in the cases that appear in our paper, the computation of such restrictions is rather straightforward because it goes through a chain of successive restrictions to intermediate subgroups and each intermediate restriction is to a Levi subgroup, or to a symmetric subgroup, or a restriction from $\operatorname{SO}_{7}$ to $\mathsf{G}_{2}$. In the Levi subgroup and symmetric subgroup cases, the restrictions are computed using the tables in [AvPe2, § 5.5]. The restrictions from $\operatorname{SO}_{7}$ to $\mathsf{G}_{2}$ can be computed via [AkPa, Theorem 8, part 3] or directly by using the program LiE [LiE1].

As a final remark, we would like to mention that it would be interesting to characterize the spherical actions on flag varieties in terms of the existing combinatorial description of arbitrary spherical varieties (see, for instance, [Tim, §§ 15.1, 30.11]) and/or compute the combinatorial data corresponding to all classified cases of such actions.

This paper is organized as follows. In § 2 we set up some notation and conventions used throughout the paper. In § 3 we introduce basic notions and recall some facts needed to state our main results. In turn, the main results of this paper are presented in § 4. In § 5 we discuss the general strategy for classifying spherical actions on flag varieties of a given group $G$ and analyze in more detail the cases of a symplectic and orthogonal group. The classification itself is carried out in § 6 for the symplectic case and in § 7 for the orthogonal case. In § 8 we explain how to compute the restricted branching monoids in all cases classified in our paper. Finally, Appendix A contains explicit realizations of the algebra $\mathfrak{g}_{2}$ as a subalgebra of $\mathfrak{so}_{7}$ and the algebra $\mathfrak{spin}_{7}$ as a subalgebra of $\mathfrak{so}_{8}$ that are needed in some computations in § 7.

Acknowledgements

A part of this work was done while the first author was visiting the Institut Fourier in Grenoble, France, in October 2018. He thanks this institution for hospitality and excellent working conditions and also expresses his gratitude to Michel Brion for support and useful discussions. Both authors are grateful to the referees for their valuable comments and suggestions on a previous version of this paper.

The results of §§ 6.5, 7.4–7.7 are obtained by the first author supported by the grant RSF–DFG 16-41-01013. The results of §§ 6.2–6.4, 7.2, 7.3 are obtained by the second author supported by the RFBR grant no. 16-01-00818.

2. Notation and conventions

Throughout the paper, all topological terms refer to the Zariski topology. All groups are assumed to be algebraic unless they explicitly appear as character groups. All subgroups of algebraic groups are assumed to be closed. The Lie algebras of groups denoted by capital Latin letters are denoted by the corresponding small Gothic letters. Given a group $K$, a $K$-variety is an algebraic variety equipped with a regular action of $K$.

Notation:

$\mathbb{Z}^{+}=\{z\in\mathbb{Z}\mid z\geq 0\}$;

$|X|$ is the cardinality of a finite set $X$;

$\langle v_{1},\ldots,v_{k}\rangle$ is the linear span of vectors $v_{1},\ldots,v_{k}$ of a vector space $V$;

$V^{*}$ is the vector space of linear functions on a vector space $V$;

$\operatorname{S}^{d}V$ is the $d$th symmetric power of a vector space $V$;

$\wedge^{d}V$ is the $d$th exterior power of a vector space $V$;

$\mathfrak{X}(K)$ is the character group of a group $K$ (in additive notation);

$K_{x}$ is the stabilizer of a point $x$ of a $K$-variety $X$;

$\overline{Y}$ is the closure of a subset $Y$ of a variety $X$;

$\mathbb{F}[X]$ is the algebra of regular functions on an algebraic variety $X$;

$\mathbb{F}(X)$ is the field of rational functions on an irreducible algebraic variety $X$;

$T_{x}X$ is the tangent space of an algebraic variety $X$ at a point $x\in X$;

$V^{(K)}_{\chi}$ is the space of semi-invariants of weight $\chi\in\mathfrak{X}(K)$ for an action of a group $K$ on a vector space $V$.

The simple roots and fundamental weights of simple groups and their Lie algebras are numbered as in [Bou1].

Given two groups $F\subset K$ and a $K$-module $V$, the restriction of $V$ to $F$ is denoted by $\left.V\right|_{F}$.

For every connected reductive group $K$, we choose a Borel subgroup $B_{K}$ and a maximal torus $T_{K}\subset B_{K}$. Let $B_{K}^{-}$ be the Borel subgroup of $K$ opposite to $B_{K}$ with respect to $T_{K}$, so that $B_{K}\cap B_{K}^{-}=T_{K}$. The groups $\mathfrak{X}(B_{K})$ and $\mathfrak{X}(B_{K}^{-})$ are identified with $\mathfrak{X}(T_{K})$ via restricting characters to $T_{K}$. Let $\Lambda^{+}(K)\subset\mathfrak{X}(T_{K})$ be the set of dominant weights of $T_{K}$ with respect to $B_{K}$. For every $\lambda\in\Lambda^{+}(K)$, we denote by $R_{K}(\lambda)$ the simple $K$-module with highest weight $\lambda$. When the group $K$ is clear from the context, we write just $R(\lambda)$.

Given a connected reductive group $K$ and a finite-dimensional $K$-module $V$, by abuse of language the pair $(K,V)$ itself is often referred to as a module.

Throughout the paper (except for the introduction), $G$ denotes a simply connected semisimple group. Let $\pi_{1},\ldots,\pi_{s}\in\Lambda^{+}(G)$ be all the fundamental weights of $G$ and consider the index set $S=\{1,\ldots,s\}$.

For every subset $I\subset S$, we consider the monoid $\Lambda^{+}_{I}(G)=\mathbb{Z}^{+}\{\pi_{i}\mid i\in I\}\subset\Lambda^{+}(G)$. Put $\lambda_{I}=\sum\limits_{i\in I}\pi_{i}$ and let $P_{I}^{-}$ be the stabilizer in $G$ of the line spanned by a lowest weight vector (with respect to $B_{G}$ and $T_{G}$) in $R_{G}(\lambda_{I})^{*}$. Then $P_{I}^{-}$ is a parabolic subgroup of $G$ containing $B_{G}^{-}$. Note that the character group $\mathfrak{X}(P_{I}^{-})$ is canonically identified with $\mathbb{Z}\Lambda^{+}_{I}(G)=\mathbb{Z}\{\pi_{i}\mid i\in I\}$ via restricting characters from $P_{I}^{-}$ to $T_{G}$. At last, we let $X_{I}=G/P_{I}^{-}$ be the flag variety of $G$ corresponding to $I$.

A flag variety $X$ of the group $G$ is said to be trivial if $X$ is a point and nontrivial otherwise. For a subset $I\subset S$, the flag variety $X_{I}$ is nontrivial if and only if $I\neq\varnothing$.

When explicitly describing modules for connected reductive groups, we always use the following conventions:

• •

the groups $\operatorname{GL}_{n}$, $\operatorname{SL}_{n}$, $\operatorname{Sp}_{n}$ (for $n=2m$), $\operatorname{SO}_{n}$ act on $\mathbb{F}^{n}$ via their tautological representations; the actions on $(\mathbb{F}^{n})^{*}$ as well as on symmetric and exterior powers of $\mathbb{F}^{n}$ are induced from this action on $\mathbb{F}^{n}$;

• •

for the group $\operatorname{Sp}_{2m}$ the notation $\wedge^{2}_{0}\mathbb{F}^{2m}$ stands for the module $R(\pi_{2})$ (which is realized as a codimension 1 submodule of $\wedge^{2}\mathbb{F}^{2m}$);

• •

the groups $\operatorname{Spin}_{7}$ and $\operatorname{Spin}_{9}$ act on $\mathbb{F}^{8}$ and $\mathbb{F}^{16}$, respectively, via the spinor representation;

• •

the group $\operatorname{Spin}_{10}$ acts on $\mathbb{F}^{16}$ via a (either of two) half-spin representation;

• •

the group $\mathsf{G}_{2}$ acts on $\mathbb{F}^{7}$ via a faithful representation of minimal dimension.

In this paper, we often fall into a situation where a connected reductive group $K$ acts on a finite-dimensional module $V$ written as a direct sum of several submodules each being a tensor product of several components acted on by different factors of $K$ and we need to specify precisely the action of $K$ on $V$. In such situations, our notation follow the following conventions:

1. (1)

the group $K$ is written as $K=K_{1}\times\ldots\times K_{p}\times\underbrace{\mathbb{F}^{\times}\times\ldots\times\mathbb{F}^{\times}}_{q}$ where each factor $K_{i}$ is visually different from $\mathbb{F}^{\times}$ (for example, some $K_{i}$’s may be written as $\operatorname{GL}_{1}$ or $\operatorname{SO}_{2}$); for short, the product $\underbrace{\mathbb{F}^{\times}\times\ldots\times\mathbb{F}^{\times}}_{q}$ is denoted by $T$ below; 2. (2)

for every $i=1,\ldots,p$ we write the number $i$ right below each component of $V$ on which $K_{i}$ acts nontrivially (exceptions: this notation is omitted when $p=1$ or $V$ is a simple $K$-module); 3. (3)

for every $j=1,\ldots,q$, we denote by $\chi_{j}$ a basis character of the $j$th copy of $\mathbb{F}^{\times}$ in $T$ (if $q=1$ we write just $\chi$ instead of $\chi_{1}$); 4. (4)

if $T$ nontrivially acts on a simple summand $U$ of $V$ via a character $\psi$, we write $[U]_{\psi}$ instead of $U$ (exception: if $U=\mathbb{F}^{1}$ then we write simply $\mathbb{F}^{1}_{\psi}$).

An example of a pair $(K,V)$ written using the above conventions is given by

[TABLE]

For explicit calculations, we use the following realizations of the symplectic and orthogonal groups (where $A$ is the $(n\times n)$-matrix with ones on the antidiagonal and zeros elsewhere):

• •

$\operatorname{Sp}_{2n}$ is the subgroup of $\operatorname{GL}_{2n}$ preserving the skew-symmetric bilinear form with matrix $\begin{pmatrix}0&A\\ -A&0\end{pmatrix};$

• •

$\operatorname{SO}_{n}$ is the subgroup of $\operatorname{SL}_{n}$ preserving the symmetric bilinear form with matrix $A$.

With these realizations, for $K=\operatorname{Sp}_{2n}$ and $K=\operatorname{SO}_{n}$ we choose $B_{K}$ (resp. $B_{K}^{-}$, $T_{K}$) to be the group of all upper-triangular (resp. lower-triangular, diagonal) matrices in $K$. Then the root system of $K$ is identified with a subset of $\mathfrak{X}(T_{K})$, in which we choose the set of simple roots corresponding to $B_{K}$. In the case $K=\operatorname{SO}_{2m}$, due to the symmetry of the Dynkin diagram, we put by convention that the $(m-1)$th (resp. $m$th) simple root takes the value $t_{m-1}t_{m}^{-1}$ (resp. $t_{m-1}t_{m}$) on every diagonal matrix $t\in\nobreak T_{K}$ with diagonal entries $t_{1},\ldots,t_{m},t_{m}^{-1},\ldots,t_{1}^{-1}$.

The vectors of the standard basis of $\mathbb{F}^{n}$ are denoted by $e_{1},\ldots,e_{n}$.

3. Preliminaries

3.1. Homogeneous line bundles on flag varieties

Let $K$ be a subgroup of $G$ and consider the homogeneous space $G/K$.

Given $\chi\in\mathfrak{X}(K)$, consider the one-dimensional $K$-module $\mathbb{F}^{1}_{\chi}$ on which $K$ acts via $\chi$. Let $K$ act on $G$ by right multiplication and let $L(\chi)$ be the quotient $(G\times\mathbb{F}^{1}_{\chi})/K$ with respect to the diagonal action of $K$. Then the natural map $L(\chi)\to G/K$ turns $L(\chi)$ into a line bundle on $G/K$. As this map is $G$-equivariant, $L(\chi)$ is called a homogeneous line bundle on $G/K$. Note that the space of global sections $H^{0}(G/K,L(\chi))$ is a $G$-module.

When $K=P_{I}^{-}$ for some $I\subset S$, one has $G/K=X_{I}$. In this case, a version of the Borel–Weil theorem states that for every $\lambda\in\mathfrak{X}(P_{I}^{-})$ there is a $G$-module isomorphism

[TABLE]

it follows, for instance, from general results discussed in [Jan, Part I, §§ 5.12–5.16 and Part II, § 2.2]; see also [AvPe2, § 4.1] for an explanation in this particular situation. Formula (3.1) shows how irreducible representations $R_{G}(\lambda)$ with $\lambda\in\Lambda_{I}^{+}(G)$ are realized as spaces of sections of homogeneous line bundles on $X_{I}$.

3.2. Finite-dimensional modules with invariant bilinear forms

Let $K$ be a connected reductive group and let $V$ be a finite-dimensional $K$-module. Suppose that $K$ preserves a nondegenerate bilinear form $\omega$ on $V$ that is either symmetric or skew-symmetric.

A $K$-submodule $W\subset V$ is said to be nondegenerate if the restriction of $\omega$ to $W$ is nondegenerate. Clearly, in this situation there is a $K$-module decomposition $V=W\oplus W^{\perp}$ where $W^{\perp}$ is the orthogonal complement of $W$ in $V$ with respect to the form $\omega$.

Given a simple $K$-submodule $W\subset V$, the kernel of the restriction of $\omega$ to $W$ is $K$-stable and hence equal to either $\{0\}$ or the whole $W$. It follows that $W$ is either nondegenerate or isotropic.

The following fact is well known; see, for instance, [Mal, Theorem 4] or [Kim, Theorem 2.2] for a proof.

Proposition 3.1**.**

Suppose that $V$ contains no proper nondegenerate $K$-submodules. Then one of the following two cases occurs:

1. (1)

$V$* is irreducible;* 2. (2)

there are simple $K$-submodules $W_{1},W_{2}\subset V$ such that $V=W_{1}\oplus W_{2}$, both $W_{1},W_{2}$ are isotropic, and $W_{2}\simeq W_{1}^{*}$ as $K$-modules.

Following the terminology of Kimelfeld [Kim], we say that $V$ is weakly reducible (with respect to $\omega$) if it falls into case (2) of Proposition 3.1.

For every $K$-module $W$, we introduce the notation $\Omega(W)=W\oplus W^{*}$. In what follows, we shall regard $\Omega(W)$ as a $K$-module equipped with a $K$-invariant nondegenerate symmetric or skew-symmetric bilinear form such that $\Omega(W)$ is a direct sum of two isotropic subspaces isomorphic to $W$ and $W^{*}$ as $K$-modules.

3.3. Equivalence and BF-equivalence on finite-dimensional modules

Given two connected reductive groups $K_{1},K_{2}$, for $i=1,2$ let $V_{i}$ be a finite-dimensional $K_{i}$-module and consider the corresponding representation $\rho_{i}\colon K_{i}\to\operatorname{GL}(V_{i})$. We say that the pairs $(K_{1},V_{1})$ and $(K_{2},V_{2})$ are equivalent if there exists an isomorphism $V_{1}\xrightarrow{\sim}V_{2}$ identifying the groups $\rho_{1}(K_{1})\subset\operatorname{GL}(V_{1})$ and $\rho_{2}(K_{2})\subset\operatorname{GL}(V_{2})$. In other words, the pairs $(K_{1},V_{1})$ and $(K_{2},V_{2})$ are equivalent if and only if they define the same linear group.

As an important example, every pair $(K,V)$ is equivalent to the pair $(K,V^{*})$.

In Table 2 we list several equivalences for pairs $(\operatorname{SO}_{n},\mathbb{F}^{n})$ with small values of $n$, these equivalences are widely used throughout this paper.

Now let $K_{1},K_{2},V_{1},V_{2},\rho_{1},\rho_{2}$ be as above and suppose that for $i=1,2$ the space $V_{i}$ carries a $K_{i}$-invariant bilinear form $\omega_{i}$. We say that the pairs $(K_{1},V_{1})$ and $(K_{2},V_{2})$ are BF-equivalent if there exists an isomorphism $V_{1}\xrightarrow{\sim}V_{2}$ identifying the group $\rho_{1}(K_{1})\subset\operatorname{GL}(V_{1})$ with $\rho_{2}(K_{2})\subset\operatorname{GL}(V_{2})$ and taking the form $\omega_{1}$ to $\omega_{2}$. In particular, if the pairs $(K_{1},V_{1})$ and $(K_{2},V_{2})$ are BF-equivalent then they are equivalent.

Given a connected reductive group $K$ and a finite-dimensional $K$-module $V$, in this paper we shall often need to specify the pair $(K,V)$ up to BF-equivalence. To this end, we always assume that the corresponding $K$-invariant bilinear form $\omega$ on $V$ is nondegenerate and either symmetric or skew-symmetric (the choice between symmetric and skew-symmetric will always be clear from the context). Further, when $V$ is explicitly written as

[TABLE]

we also assume the following properties:

1. (1)

all direct summands in (3.2) are pairwise orthogonal with respect to the form $\omega$; 2. (2)

for each $j=1,\ldots,q$ the summand $\Omega(W_{j})$ is a direct sum of two $K$-stable isotropic subspaces isomorphic to $W_{j}$ and $W_{j}^{*}$ as $K$-modules.

Throughout this paper, the above conventions will always be enough to uniquely determine the BF-equivalence class of the pair $(K,V)$.

3.4. Spherical varieties

Let $K$ be a connected reductive group. We recall from the introduction that a $K$-variety $X$ is said to be spherical (or $K$-spherical) if the Borel subgroup $B_{K}$ has a dense open orbit in $X$.

Given a spherical $K$-variety $X$, we put

[TABLE]

It is easy to see that $\Lambda_{X}$ is a sublattice of $\mathfrak{X}(T_{K})$, it is called the weight lattice of $X$. The rank of this lattice is said to be the rank of $X$; we denote it by $\operatorname{rk}_{K}X$.

In this paper, we shall need the following general fact on spherical varieties, which follows, for instance, from [Vin, Theorem 1].

Theorem 3.2**.**

Suppose that $X$ is a spherical $K$-variety. Then any $K$-stable irreducible subvariety of $X$ is also spherical.

3.5. Spherical modules

Let $K$ be a connected reductive group and let $V$ be a finite-dimensional $K$-module.

As was already mentioned in the introduction, $V$ is said to be a spherical $K$-module if $V$ is spherical as a $K$-variety. In this case, it is easy to see that every $K$-submodule of $V$ is also spherical. (This also follows from Theorem 3.2.) According to [ViKi, Theorem 2], the condition of $V$ being a spherical $K$-module is equivalent to the fact that the $K$-module $\mathbb{F}[V]$ is multiplicity free. Given a spherical $K$-module $V$, the highest weights of all simple $K$-modules that occur in $\mathbb{F}[V]$ form a submonoid $\mathrm{E}_{K}(V)$ of $\Lambda^{+}(K)$, called the weight monoid of $V$. It is well known that $\mathrm{E}_{K}(V)$ is free (see, for instance, [Kno, Theorem 3.2]) and $\operatorname{rk}\mathrm{E}_{K}(V)=\operatorname{rk}_{K}V$ (see, for instance, [Tim, Proposition 5.14]).

The terminology introduced below follows Knop, see [Kno, § 5].

Let $\rho\colon K\to\operatorname{GL}(V)$ be the representation defining the $K$-module structure on $V$. We say that $V$ is saturated if the dimension of the center of $\rho(K)$ equals the number of irreducible summands of $V$.

We say that $V$ is decomposable if for $i=1,2$ there exist a connected reductive group $K_{i}$ and a finite-dimensional $K_{i}$-module $V_{i}$ such that the pair $(K,V)$ is equivalent to $(K_{1}\times K_{2},V_{1}\oplus V_{2})$. Evidently, in this situation $(K,V)$ is a spherical module if and only if so are both $(K_{1},V_{1})$ and $(K_{2},V_{2})$, in which case $\operatorname{rk}_{K}V=\operatorname{rk}_{K_{1}}V_{1}+\operatorname{rk}_{K_{2}}V_{2}$. We say that $V$ is indecomposable if $V$ is not decomposable.

There is a complete classification of spherical modules. In the case where $V$ is simple the classification was obtained in [Kac]. The case of arbitrary $V$ was settled in the two independent papers [BeRa] and [Lea], it reduces essentially to classifying all indecomposable saturated spherical modules. (In fact, all such modules for which the derived subgroup of the acting group is simple were classified earlier in [Bri].) The weight monoids of all spherical modules are also known thanks to the works [HoUm] (the case of simple $V$) and [Lea] (the general case). A complete list (up to equivalence) of all indecomposable saturated spherical modules can be found in [Kno, § 5] along with various additional data, including the rank and indecomposable elements of the weight monoids. In this paper, for checking sphericity of a given module we find it convenient to use [AvPe1, Theorem 5.3], which is a reformulation of [BeRa, Theorem 7] and [Lea, Theorem 2.6].

Now suppose that $V$ is a spherical $K$-module and fix a decomposition $V=V_{1}\oplus\ldots\oplus V_{k}$ where each direct summand is a simple $K$-module. Let $K^{\prime}$ be the derived subgroup of $K$ and let $Z$ be the subgroup of $\operatorname{GL}(V)$ consisting of all elements that act by scalar transformations on each $V_{i}$, $i=1,\ldots,k$. Then $V$ is a saturated spherical ($K^{\prime}\times Z$)-module, hence the pair $(K,V)$ is equivalent to $(K_{1}\times\ldots\times K_{m},W_{1}\oplus\ldots\oplus W_{m})$ where $K_{i}$ is a connected reductive group and $W_{i}$ is an indecomposable saturated spherical $K_{i}$-module for each $i=1,\ldots,m$. In this situation, it is easy to see that $\operatorname{rk}_{K}V=\operatorname{rk}_{K^{\prime}\times Z}V=\operatorname{rk}_{K_{1}}W_{1}+\ldots+\operatorname{rk}_{K_{m}}W_{m}$. The latter observation will be always used in this paper for computing the ranks of spherical modules.

3.6. Spherical modules with invariant bilinear forms

Retain the notation of § 3.5. Combining the well-known description of invariant bilinear forms on spaces of irreducible representations of semisimple groups (see [Mal, § 2] or [Bou2, Ch. VIII, § 7.5, Proposition 12]) with the classification of spherical modules (see the references in § 3.5) and Proposition 3.1 one obtains the following results.

Proposition 3.3**.**

Suppose that $K$ preserves a nondegenerate skew-symmetric bilinear form $\omega$ on $V$ and $V$ contains no proper $K$-submodules that are nondegenerate with respect to $\omega$. Then $V$ is a spherical $(K\times\mathbb{F}^{\times})$-module (with $\mathbb{F}^{\times}$ acting by scalar transformations) if and only if the pair $(K,V)$ is equivalent to a pair in Table 3.

Proposition 3.4**.**

Suppose that $K$ preserves a nondegenerate symmetric bilinear form $\omega$ on $V$ and $V$ contains no proper $K$-submodules that are nondegenerate with respect to $\omega$. Then $V$ is a spherical $(K\times\mathbb{F}^{\times})$-module (with $\mathbb{F}^{\times}$ acting by scalar transformations) if and only if the pair $(K,V)$ is equivalent to a pair in Table 4.

3.7. Branching monoids and restricted branching monoids

A more detailed discussion of the notions introduced in this subsection can be found in [AvPe2, § 3.4].

Let $H\subset G$ be a connected reductive subgroup.

We put

[TABLE]

Then $\Gamma(G,H)$ is a submonoid of $\Lambda^{+}(G)\times\Lambda^{+}(H)$, it is called the branching monoid for the pair $(G,H)$.

Given any subset $I\subset S$, the monoid

[TABLE]

is called the restricted branching monoid corresponding to the subset $I$.

3.8. Spherical actions on flag varieties and the corresponding restricted branching monoids

Let $H\subset G$ be a connected reductive subgroup and let $I\subset S$ be an arbitrary subset.

The next theorem is a particular case of [ViKi, Corollary 1].

Theorem 3.5**.**

The following conditions are equivalent:

1. (1)

For every $\lambda\in\Lambda^{+}_{I}(G)$, the $H$-module $\left.R_{G}(\lambda)\right|_{H}$ is multiplicity free. 2. (2)

The flag variety $X_{I}$ is $H$-spherical.

Under the conditions of Theorem 3.5, the restriction to $H$ of any simple $G$-module $R_{G}(\lambda)$ with $\lambda\in\Lambda^{+}_{I}(G)$ is uniquely determined by the monoid $\Gamma_{I}(G,H)$ as follows:

[TABLE]

The following result is implied by [AvPe2, Theorem 4.2 and Proposition 4.4].

Theorem 3.6**.**

Under the conditions of Theorem 3.5, the following assertions hold:

1. (a)

the monoid $\Gamma_{I}(G,H)$ is free; 2. (b)

$\operatorname{rk}\Gamma_{I}(G,H)=|I|+\operatorname{rk}_{H}X_{I}$.

4. Statement of the main results

4.1. Reductions

In this subsection, we describe several reductions that simplify the statement of main theorems in this section. Fix a connected reductive subgroup $H\subset G$ along with a subset $I\subset S$ and suppose that the variety $X_{I}$ is $H$-spherical.

Reduction 1. Let $I^{\prime}\subset I$ be an arbitrary subset. Then the variety $X_{I^{\prime}}$ is automatically $H$-spherical and the indecomposable elements of $\Gamma_{I^{\prime}}(G,H)$ are those of $\Gamma_{I}(G,H)$ for which the first component belongs to $\Lambda^{+}_{I^{\prime}}(G)$. Therefore, for a given pair $(G,H)$, it is enough to consider subsets $I\subset S$ that are maximal with the property that $X_{I}$ is $H$-spherical.

Reduction 2. Let $\sigma$ be an automorphism of $G$. Then $(\tau\sigma)(B_{G})=B_{G}$ and $(\tau\sigma)(T_{G})=T_{G}$ for an appropriate inner automorphism $\tau$ of $G$. By abuse of notation, we shall use the same letter $\sigma$ to denote the following objects:

• •

the bijection of $S$ corresponding to the automorphism of the Dynkin diagram of $G$ induced by $\tau\sigma$;

• •

the bijection $\Lambda^{+}(G)\to\Lambda^{+}(G)$ induced by $\tau\sigma$;

• •

the induced bijection $\Lambda^{+}(H)\to\Lambda^{+}(\sigma(H))$.

Now, given $\lambda\in\Lambda^{+}_{I}(G)$, after changing the action of $G$ on $R_{G}(\lambda)$ to $(g,v)\mapsto\sigma^{-1}(g)(v)$ formula (3.3) takes the form

[TABLE]

Then $X_{\sigma(I)}$ is $\sigma(H)$-spherical. Moreover, $(\lambda;\mu)\in\nobreak\Gamma_{I}(G,H)$ if and only if $(\sigma(\lambda);\sigma(\mu))\in\Gamma_{\sigma(I)}(G,\sigma(H))$. In particular, $(\lambda;\mu)$ is an indecomposable element of $\Gamma_{I}(G,H)$ if and only if $(\sigma(\lambda);\sigma(\mu))$ is an indecomposable element of $\Gamma_{\sigma(I)}(G,\sigma(H))$. In this situation, we say that the triple $(G,\sigma(H),\sigma(I))$ is obtained from $(G,H,I)$ by the automorphism $\sigma$.

Reduction 3. Let $H^{\prime}$ denote the derived subgroup of $H$ and suppose $K\subset G$ is a connected reductive subgroup such that $H^{\prime}\subset K\subset H$. Then it follows from Theorems 3.5 and 3.6 that $K$ acts spherically on $X_{I}$ if and only if the restrictions to $\Lambda_{I}^{+}(G)\oplus\Lambda^{+}(K)$ of all the indecomposable elements of $\Gamma_{I}(G,H)$ are linearly independent (in which case these restrictions are all the indecomposable elements of $\Gamma_{I}(G,K)$). Therefore it suffices to classify spherical actions on flag varieties only for groups $H$ that are not intermediate between a bigger connected reductive subgroup of $G$ and its derived subgroup.

We remark that, for completeness of the results obtained in this paper, we use Reduction 3 only to exclude subgroups $H$ that are intermediate between a Levi subgroup of $G$ and its derived subgroup.

4.2. Levi subgroups and symmetric subgroups in $\operatorname{Sp}_{2n}$ and $\operatorname{SO}_{n}$

Let $V$ be a finite-dimensional vector space equipped with a nondegenerate bilinear form $\omega$ that is either symmetric or skew-symmetric. Put $G=\operatorname{Sp}(V)$ if $\omega$ is skew-symmetric and $G=\operatorname{Spin}(V)$ if $\omega$ is symmetric. Let $H\subset G$ be a connected reductive subgroup.

In the statements of our main theorems in §§ 4.3–4.4, we exclude the cases where $H$ is either intermediate between a Levi subgroup of $G$ and its derived subgroup or a symmetric subgroup of $G$. For convenience of the reader, in this subsection we specify explicitly all Levi subgroups and all symmetric subgroups in $G$.

We recall that a subgroup $K$ of $G$ is said to be symmetric if $K$ is the subgroup of fixed points of a nontrivial involutive automorphism of $G$. As $G$ is simply connected, in this case $K$ is reductive and connected by [Stei, Theorem 8.1].

Proposition 4.1**.**

Suppose that $\omega$ is skew-symmetric. Then

1. (a)

$H$* is a Levi subgroup of $G$ if and only if the pair $(H,V)$ is BF-equivalent to*

[TABLE]

for some $k\geq 0$, $m\geq 0$, and $n_{i}\geq 1$; 2. (b)

$H$* is a symmetric subgroup of $G$ if and only if the pair $(H,V)$ is BF-equivalent to either $(\operatorname{Sp}_{2n_{1}}\times\operatorname{Sp}_{2n_{2}},\mathbb{F}^{2n_{1}}\oplus\mathbb{F}^{2n_{2}})$ for some $n_{1},n_{2}\geq 1$ or $(\operatorname{GL}_{n},\Omega(\mathbb{F}^{n}))$ for some $n\geq 1$.*

Proposition 4.2**.**

Suppose that $\omega$ is symmetric. Then

1. (a)

$H$* is a Levi subgroup of $G$ if and only if the pair $(H,V)$ is BF-equivalent to*

[TABLE]

for some $k\geq 0$, $m\geq 0$, and $n_{i}\geq 1$; 2. (b)

$H$* is a symmetric subgroup of $G$ if and only if the pair $(H,V)$ is BF-equivalent to either $(\operatorname{SO}_{n_{1}}\times\operatorname{SO}_{n_{2}},\mathbb{F}^{n_{1}}\oplus\mathbb{F}^{n_{2}})$ for some $n_{1},n_{2}\geq 1$ or $(\operatorname{GL}_{n},\Omega(\mathbb{F}^{n}))$ for some $n\geq 2$.*

4.3. The symplectic case

Let $V$ be a vector space of dimension $2n$ ($n\geq 2$) equipped with a nondegenerate skew-symmetric bilinear form $\omega$. Let $G=\operatorname{Sp}(V)\simeq\operatorname{Sp}_{2n}$ be the subgroup of $\operatorname{GL}(V)$ preserving $\omega$.

First we consider separately the case $I=\{1\}$. Note that $X_{\{1\}}\simeq\mathbb{P}(V)$ (see § 5.3).

Theorem 4.3**.**

The variety $X_{\{1\}}$ is $H$-spherical if and only if $V$ is a spherical $(H\times\mathbb{F}^{\times})$-module where $\mathbb{F}^{\times}$ acts by scalar transformations. Moreover, the above conditions hold if and only if the pair $(H,V)$ is BF-equivalent to one of the pairs in Table 5.

Remark 4.4*.*

The first equivalence in Theorem 4.3 is trivial. The second one is proved in Theorem 6.2.

Recall from § 3.5 the notion of weight monoid $\mathrm{E}_{K}(U)$ of a spherical $K$-module $U$.

Theorem 4.5**.**

In the situation of Theorem 4.3, let $\delta$ denote the character via which $\mathbb{F}^{\times}$ acts on $V$ and let $\mathrm{E}^{0}_{H\times\mathbb{F}^{\times}}(V^{*})$ be the set of indecomposable elements of $\mathrm{E}_{H\times\mathbb{F}^{\times}}(V^{*})$. Identify $\Lambda^{+}(H\times\mathbb{F}^{\times})$ with $\Lambda^{+}(H)\oplus\mathbb{Z}\delta$ and consider the map $\Lambda^{+}(H\times\mathbb{F}^{\times})\to\Lambda^{+}_{\{1\}}(G)\times\Lambda^{+}(H)$ given by $\lambda=\mu+k\delta\mapsto\overline{\lambda}=(k\pi_{1};\mu)$. Then the set of indecomposable elements of $\Gamma_{\{1\}}(G,H)$ is $\{\overline{\lambda}\mid\lambda\in\mathrm{E}^{0}_{H\times\mathbb{F}^{\times}}(V^{*})\}.$

The proof of Theorem 4.5 is given in § 8.1.

We now turn to the case $I\neq\{1\}$.

Theorem 4.6**.**

Suppose that $I\subset S$ is a nonempty subset distinct from $\{1\}$ and $H\subset G$ is a connected reductive subgroup such that

• •

$H$* is not intermediate between a Levi subgroup of $G$ and its derived subgroup;*

• •

$H$* is not a symmetric subgroup of $G$.*

Then the following conditions are equivalent:

1. (1)

The variety $X_{I}$ is $H$-spherical and $I$ is maximal with this property. 2. (2)

the pair $(H,V)$, considered up to BF-equivalence, and the set $I$ fall into the only case in Table 6.

Moreover, Table 6 lists also the rank and indecomposable elements of the monoid $\Gamma_{I}(G,H)$ for that only case.

For notation used in Table 6, see § 4.5.

The equivalence of conditions (1) and (2) in Theorem 4.6 follows from results in §§ 6.3–6.5. The method for computing the monoid $\Gamma_{I}(G,H)$ for the case in Table 6 is described in 8.2.

4.4. The orthogonal case

Let $V$ be a vector space of dimension $d\geq 5$ equipped with a nondegenerate symmetric bilinear form $\omega$. Let $G=\operatorname{Spin}(V)\simeq\operatorname{Spin}_{d}$ be the spinor group determined by $V$ and $\omega$. Let $H\subset G$ be a connected reductive subgroup.

We fix a decomposition $V=V_{1}\oplus\ldots\oplus V_{r}$ into a direct sum of $H$-submodules such that the summands are pairwise orthogonal with respect to the form $\omega$ and each summand is either irreducible or weakly reducible. We denote by $H_{0}$ the image of $H$ in $\operatorname{SO}(V)$. For each $i=1,\ldots,r$ we let $H_{i}$ be the image of $H$ in the group $\operatorname{SO}(V_{i})$.

First we consider separately the case $I=\{1\}$. Note that $X_{\{1\}}$ is isomorphic to the variety $\operatorname{SOGr}_{1}(V)$ mentioned in the introduction (see § 5.3).

Theorem 4.7**.**

The following assertions hold.

1. (a)

Suppose that $r=1$. Then $X_{\{1\}}$ is $H$-spherical if and only if $V$ is a spherical $(H\times\mathbb{F}^{\times})$-module where $\mathbb{F}^{\times}$ acts by scalar transformations. Moreover, the above conditions hold if and only if the pair $(H,V)$ is BF-equivalent to one of the pairs in Table 4. 2. (b)

*Suppose that $r=2$. Then $X_{\{1\}}$ is $H$-spherical if and only if $V$ is a spherical $(H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times})$-module where for $i=1,2$ the *$i$th factor $\mathbb{F}^{\times}$ acts on $V_{i}$ by scalar transformations. Moreover, the above conditions hold if and only if one of the following cases occurs:

1. (1)

$H_{0}=H_{1}\times H_{2}$* and for each $i=1,2$ the pair $(H_{i},V_{i})$ is BF-equivalent to one of the pairs in Table 4;* 2. (2)

the pair $(H,V)$ is BF-equivalent to a pair in Table 7. 3. (c)

If $r\geq 3$ then $X_{\{1\}}$ is not $H$-spherical.

In Case 7 of Table 7 the symbols $\mathbb{F}^{8}_{\pm}$ stand for the spaces of the two half-spin representations of $\operatorname{Spin}_{8}$.

Theorem 4.7 follows from results in § 7.2.

Remark 4.8*.*

In Theorem 4.7(a), the fact that $X_{\{1\}}$ is $H$-spherical if and only if $V$ is a spherical $(H\times\mathbb{F}^{\times})$-module is obtained a posteriori as a result of classification. On the contrary, in Theorem 4.7(b) the fact that $X_{\{1\}}$ is $H$-spherical if and only if $V$ is a spherical $(H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times})$-module is proved by a general argument, see Proposition 7.7.

Recall from § 3.5 the notion of weight monoid $\mathrm{E}_{K}(U)$ of a spherical $K$-module $U$.

Theorem 4.9**.**

The following assertions hold.

1. (a)

In the situation of Theorem 4.7(a), let $\delta$ denote the character via which $\mathbb{F}^{\times}$ acts on $V$ and let $\mathrm{E}^{0}_{H\times\mathbb{F}^{\times}}(V^{*})$ be the set of indecomposable elements of $\mathrm{E}_{H\times\mathbb{F}^{\times}}(V^{*})$. Identify $\Lambda^{+}(H\times\mathbb{F}^{\times})$ with $\Lambda^{+}(H)\oplus\mathbb{Z}\delta$ and consider the map

[TABLE]

Then the set of indecomposable elements of $\Gamma_{\{1\}}(G,H)$ is

[TABLE] 2. (b)

*In the situation of Theorem 4.7(b), for $i=1,2$ let $\delta_{i}$ denote the character via which the *$i$th factor $\mathbb{F}^{\times}$ acts on $V_{i}$ and let $\mathrm{E}^{0}_{H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times}}(V^{*})$ be the set of indecomposable elements of $\mathrm{E}_{H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times}}(V^{*})$. Identify $\Lambda^{+}(H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times})$ with $\Lambda^{+}(H)\oplus\mathbb{Z}\delta_{1}\oplus\mathbb{Z}\delta_{2}$ and consider the map

[TABLE]

Then the set of indecomposable elements of $\Gamma_{\{1\}}(G,H)$ is

[TABLE]

(Note that $2\delta_{1},2\delta_{2}\in\mathrm{E}^{0}_{H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times}}(V^{*})$ and these elements give rise to the same indecomposable element $(2\pi_{1};0)$ of $\Gamma_{\{1\}}(G,H)$.)**

The proof of Theorem 4.9 is given in § 8.1.

We now turn to the case $I\neq\{1\}$.

Theorem 4.10**.**

Suppose that $d=2n+1$ with $n\geq 3$, $I\subset S$ is a nonempty subset distinct from $\{1\}$, and the following properties hold:

• •

$H$* is not intermediate between a Levi subgroup of $G$ and its derived subgroup;*

• •

$H$* is not a symmetric subgroup of $G$.*

Then the following conditions are equivalent:

1. (1)

The variety $X_{I}$ is $H$-spherical and $I$ is maximal with this property. 2. (2)

the pair $(H,V)$, considered up to BF-equivalence, and the set $I$ fall into one of the cases in Table 8.

Moreover, Table 8 lists also the rank and indecomposable elements of the monoid $\Gamma_{I}(G,H)$ for each of the cases.

Theorem 4.11**.**

Suppose that $d=2n$ with $n\geq 4$, $I\subset S$ is a nonempty subset distinct from $\{1\}$, and the following properties hold:

• •

$H$* is not intermediate between a Levi subgroup of $G$ and its derived subgroup;*

• •

$H$* is not a symmetric subgroup of $G$.*

Then the following conditions are equivalent:

1. (1)

The variety $X_{I}$ is $H$-spherical. 2. (2)

Up to an automorphism of $G$, the pair $(H,V)$, considered up to BF-equivalence, and the set $I$ fall into one of the cases in Table 9.

Moreover, Table 9 lists also the rank and indecomposable elements of the monoid $\Gamma_{I}(G,H)$ for each of the cases.

For notation used in Tables 8 and 9, we refer to § 4.5.

In Theorems 4.10 and 4.11, the equivalence of conditions (1) and (2) follows from results in §§ 7.3–7.7. The method for computing the monoid $\Gamma_{I}(G,H)$ for each of the cases in Tables 8 and 9 is described in § 8.2.

4.5. Notation and conventions used in Tables 6, 8, and 9

The symbol $\delta_{i}^{j}$ denotes the Kronecker delta, that is, $\delta_{i}^{j}=1$ for $i=j$ and $\delta_{i}^{j}=0$ otherwise.

Whenever an element $(\lambda;\mu)$ in the last column is followed by a parenthesis containing an inequality on parameters, this means that $(\lambda;\mu)$ is an indecomposable element of $\Gamma_{I}(G,H)$ if and only if the inequality is satisfied.

In all the tables under consideration, the group $H$ contains at most three factors different from $\mathbb{F}^{\times}$; we write $\pi_{i}$ (resp. $\pi^{\prime}_{i}$, $\pi^{\prime\prime}_{i}$) for the $i$th fundamental weight of the first (resp. second, third) factor (for $\operatorname{SO}_{4}$, the fundamental weights have numbers $1$ and $2$). For convenience in certain formulas, we put $\pi_{0}=\pi^{\prime}_{0}=0$. We point out that the usage of the symbols $\pi_{i}$ to denote fundamentals weights of both $G$ and $H$ simultaneously does not cause any ambiguity.

In Cases 9 and 9 of Table 9, the symbol $\chi/2$ stands for the character of the preimage of $\mathbb{F}^{\times}$ in $\operatorname{Spin}(V)$ such that $2\cdot(\chi/2)=\chi$.

In Case 9 of Table 9, there are two conjugacy classes in $G$ of subgroups $H$ having the indicated type, and the variety $X_{I}$ is $H$-spherical only for one of them. See § 7.1 for the convention choosing the right conjugacy class.

5. Main tools

5.1. Nil-equivalence relation on $\mathscr{F}(G)$ and its properties

Let $\mathscr{F}(G)$ denote the set of nontrivial flag varieties of the group $G$.

Given a parabolic subgroup $P\subset G$, consider a Levi decomposition $P=LP^{u}$ where $L$ is a Levi subgroup and $P^{u}$ is the unipotent radical of $P$. A well-known result of Richardson [Rich, Proposition 6(c)] asserts that $\mathfrak{p}^{u}$ has an open orbit for the adjoint action of $P$. Let $O_{P}$ denote this open orbit and put

[TABLE]

Then $\mathscr{N}(G/P)$ is a nilpotent orbit in $\mathfrak{g}$.

We say that two varieties $X_{1},X_{2}\in\mathscr{F}(G)$ are nil-equivalent (notation $X_{1}\sim X_{2}$) if $\mathscr{N}(X_{1})=\mathscr{N}(X_{2})$.

Remark 5.1*.*

If $P,Q\subset G$ are two associated parabolic subgroups (that is, their Levi subgroups are conjugate in $G$) then $\mathscr{N}(G/P)=\mathscr{N}(G/Q)$ by [JoRi, Theorem 2.7]. Hence $G/P$ and $G/Q$ are automatically nil-equivalent in this case.

Now let $K\subset G$ be an arbitrary connected reductive subgroup.

Theorem 5.2** ([AvPe1, Theorem 1.3]).**

Suppose that $X_{1},X_{2}\in\mathscr{F}(G)$ and $X_{1}\sim X_{2}$. Then the following conditions are equivalent:

1. (1)

$X_{1}$* is $K$-spherical.* 2. (2)

$X_{2}$* is $K$-spherical.*

For every $X\in\mathscr{F}(G)$, let \mbox{[[}X\mbox{]]} denote the nil-equivalence class of $X$.

The set $\mathscr{F}(G)/\!\sim$ of all nil-equivalence classes is naturally equipped with a partial order $\preccurlyeq$ defined as follows: \mbox{[[}X_{1}\mbox{]]}\preccurlyeq\mbox{[[}X_{2}\mbox{]]} (or \mbox{[[}X_{2}\mbox{]]}\succcurlyeq\mbox{[[}X_{1}\mbox{]]}) if and only if $\mathscr{N}(X_{1})\subset\overline{\mathscr{N}(X_{2})}$ for all $X_{1},X_{2}\in\mathscr{F}(G)$. We shall also write \mbox{[[}X_{1}\mbox{]]}\prec\mbox{[[}X_{2}\mbox{]]} (or \mbox{[[}X_{2}\mbox{]]}\succ\mbox{[[}X_{1}\mbox{]]}) when \mbox{[[}X_{1}\mbox{]]}\preccurlyeq\mbox{[[}X_{2}\mbox{]]} but \mbox{[[}X_{1}\mbox{]]}\neq\mbox{[[}X_{2}\mbox{]]}.

The following theorem, which traces back to [Pet, Theorem 5.8], is based on a result of Losev [Los].

Theorem 5.3** ([AvPe1, Theorem 1.4]).**

Suppose that $X_{1},X_{2}\in\mathscr{F}(G)$, \mbox{[[}X_{1}\mbox{]]}\prec\mbox{[[}X_{2}\mbox{]]}, and $X_{2}$ is $K$-spherical. Then $X_{1}$ is also $K$-spherical.

Remark 5.4*.*

In [AvPe1], Theorem 5.2 was proved by showing that, for a given $X\in\mathscr{F}(G)$, the $K$-sphericity of $X$ is equivalent to the action of $K$ on $\mathscr{N}(X)$ being coisotropic (see the definition in loc. cit.), and the core of the proof of Theorem 5.3 was the fact that the property of being coisotropic is inherited by all adjoint orbits in $\mathfrak{g}$ lying in the closure of a given one. The latter fact has been recently generalized in [PaYa] to actions of higher corank on adjoint orbits in $\mathfrak{g}$, which provides a more general framework for the results discussed in this subsection.

5.2. Compositions and partitions

Let $d$ be a positive integer.

A tuple $(a_{1},\ldots,a_{p})$ of positive integers satisfying $a_{1}+\nobreak\ldots+a_{p}=d$ is called a composition of $d$. Given a composition $\mathbf{a}=(a_{1},\ldots,a_{p})$ of $d$, each number $a_{i}$ is said to be a part of $\mathbf{a}$. For every part $x$ of $\mathbf{a}$, its multiplicity is the cardinality of the set $\{i\mid a_{i}=x\}$.

We say that a composition $(a_{1},\ldots,a_{p})$ is trivial if $p=1$ and nontrivial if $p\geq 2$.

A composition $(a_{1},\ldots,a_{p})$ of $d$ is said to be symmetric if $a_{i}=a_{p+1-i}$ for all $i=1,\ldots,p$.

A composition $(a_{1},\ldots,a_{p})$ of $d$ is said to be a partition if $a_{1}\geq\ldots\geq a_{p}$. We let $\mathcal{P}(d)$ denote the set of all partitions of $d$.

If $b_{1}>b_{2}>\ldots>b_{l}$ are all parts of a partition $\mathbf{a}\in\mathcal{P}(d)$ and $k_{1},k_{2},\ldots,k_{l}$ are their multiplicities, then $\mathbf{a}$ will be also written as $[b_{1}^{k_{1}},b_{2}^{k_{2}},\ldots,b_{l}^{k_{l}}]$.

Given a partition $(a_{1},\ldots,a_{p})$ of $d$, it is often convenient to assume that $a_{i}=0$ for $i>p$.

The set $\mathcal{P}(d)$ carries a natural partial order defined as follows. For two partitions $\mathbf{a}=(a_{1},\ldots,a_{p})$ and $\mathbf{b}=(b_{1},\ldots,b_{q})$, we write $\mathbf{a}\preccurlyeq\mathbf{b}$ (or $\mathbf{b}\succcurlyeq\mathbf{a}$) if

[TABLE]

We shall also write $\mathbf{a}\prec\mathbf{b}$ (or $\mathbf{b}\succ\mathbf{a}$) if $\mathbf{a}\preccurlyeq\mathbf{b}$ and $\mathbf{a}\neq\mathbf{b}$.

For each $\varepsilon\in\{\pm 1\}$, we define the subset $\mathcal{P}_{\varepsilon}(d)\subset\mathcal{P}(d)$ by the formula

[TABLE]

A partition $\mathbf{a}\in\mathcal{P}_{1}(d)$ is said to be very even if all parts of $\mathbf{a}$ are even. Clearly, very even partitions occur only when $d=4k$ for an integer $k$.

5.3. Flag varieties for the symplectic and orthogonal group

In this subsection, we discuss the description of flag varieties of the symplectic and orthogonal group as varieties of flags of isotropic subspaces in the space of the tautological representation. This description will be widely used in the remaining part of the paper.

Let $V$ be a vector space of dimension $d>0$ equipped with a nondegenerate bilinear form $\omega$ such that $\omega(x,y)=\varepsilon\omega(y,x)$ for all $x,y\in V$, where $\varepsilon\in\{\pm 1\}$. In other words, $\omega$ is symmetric for $\varepsilon=1$ and skew-symmetric for $\varepsilon=-1$. Let $\widetilde{G}_{\varepsilon}=\operatorname{Aut}(V,\omega)$ be the subgroup of $\operatorname{GL}(V)$ consisting of all elements preserving $\omega$, so that $\widetilde{G}_{\varepsilon}=\mathrm{O}(V)$ for $\varepsilon=1$ and $\widetilde{G}_{\varepsilon}=\operatorname{Sp}(V)$ for $\varepsilon=-1$. We also put $G_{\varepsilon}=(\widetilde{G}_{\varepsilon})^{0}$, so that $G_{\varepsilon}=\operatorname{SO}(V)$ for $\varepsilon=1$ and $G_{\varepsilon}=\widetilde{G}_{\varepsilon}=\operatorname{Sp}(V)$ for $\varepsilon=-1$.

For every symmetric composition $\mathbf{a}=(a_{1},\ldots,a_{p})$ of $d$, let $\operatorname{Fl}^{(\varepsilon)}_{\mathbf{a}}(V)$ be the set of all tuples $(V_{0},V_{1},\ldots,V_{[p/2]})$ where $V_{0}=\{0\}$ and $V_{1},\ldots,V_{[p/2]}$ are isotropic subspaces of $V$ such that $V_{1}\subset\ldots\subset V_{[p/2]}$ and $\dim V_{i}=a_{1}+\ldots+a_{i}$ for all $i=1,\ldots,[p/2]$. It is well known that $\operatorname{Fl}^{(\varepsilon)}_{\mathbf{a}}(V)$ is a projective homogeneous $\widetilde{G}_{\varepsilon}$-variety. If $[p/2]=1$ then the points of $\operatorname{Fl}^{(\varepsilon)}_{\mathbf{a}}(V)$ are naturally identified with the $a_{1}$-dimensional isotropic subspaces of $V$; in this situation $\operatorname{Fl}^{(\varepsilon)}_{\mathbf{a}}(V)$ is also called an isotropic Grassmannian.

If $G_{\varepsilon}\simeq\operatorname{Sp}_{2n}$, $n\geq 1$, then for every symmetric composition $\mathbf{a}=(a_{1},\ldots,a_{p})$ of $2n$ the variety $\operatorname{Fl}^{(-1)}_{\mathbf{a}}(V)$ is a single $G_{\varepsilon}$-orbit isomorphic to $X_{I}$ with

[TABLE]

In this case, we shall use the notation $\operatorname{SpFl}_{\mathbf{a}}(V)=\operatorname{Fl}^{(-1)}_{\mathbf{a}}(V)$.

If $G_{\varepsilon}\simeq\operatorname{SO}_{2n+1}$, $n\geq 1$, then for every symmetric composition $\mathbf{a}=(a_{1},\ldots,a_{p})$ of $2n+1$ the variety $\operatorname{Fl}^{(1)}_{\mathbf{a}}(V)$ is a single $G_{\varepsilon}$-orbit isomorphic to $X_{I}$ with

[TABLE]

In this case, we shall use the notation $\operatorname{SOFl}_{\mathbf{a}}(V)=\operatorname{Fl}^{(1)}_{\mathbf{a}}(V)$.

If $G_{\varepsilon}\simeq\operatorname{SO}_{2n}$, $n\geq 2$, then, given a symmetric composition $\mathbf{a}=(a_{1},\ldots,a_{p})$ of $2n$, there are the following possibilities:

• •

if $p$ is odd then $\operatorname{Fl}^{(1)}_{\mathbf{a}}(V)$ is a single $G_{\varepsilon}$-orbit isomorphic to $X_{I}$ with

[TABLE]

in this case we shall use the notation $\operatorname{SOFl}_{\mathbf{a}}(V)=\operatorname{Fl}^{(1)}_{\mathbf{a}}(V)$;

• •

if $p=2q$ is even and $a_{q}=1$ then $\operatorname{Fl}^{(1)}_{\mathbf{a}}(V)$ is a single $\operatorname{SO}_{2n}$-orbit isomorphic to $X_{I}$ with

[TABLE]

in this case we shall use the notation $\operatorname{SOFl}_{\mathbf{a}}(V)=\operatorname{Fl}^{(1)}_{\mathbf{a}}(V)$;

• •

if $p=2q$ is even and $a_{q}\geq 2$ then $\operatorname{Fl}^{(1)}_{\mathbf{a}}(V)$ is a disjoint union of two $\operatorname{SO}_{2n}$-orbits $\operatorname{SOFl}^{+}_{\mathbf{a}}(V)$ and $\operatorname{SOFl}^{-}_{\mathbf{a}}(V)$ such that, by convention in the choice of signs, $\operatorname{SOFl}^{+}_{\mathbf{a}}(V)\simeq X_{I}$ with

[TABLE]

and $\operatorname{SOFl}^{-}_{\mathbf{a}}(V)\simeq X_{I}$ with

[TABLE]

For isotropic Grassmannians, we shall use special notation. Namely, for the variety of $k$-dimensional isotropic subspaces of $V$ we shall write $\operatorname{SpGr}_{k}(V)$ if $\varepsilon=-1$ and $\operatorname{SOGr}_{k}(V)$ if $\varepsilon=1$. If $k=[d/2]$ is maximal possible, we shall also write $\operatorname{SpGr}_{\max}(V)$ and $\operatorname{SOGr}_{\max}(V)$ instead of $\operatorname{SpGr}_{k}(V)$ and $\operatorname{SOGr}_{k}(V)$, respectively. Finally, if $\varepsilon=1$ and $d=2k$ is even then we put $\operatorname{SOGr}_{\max}^{+}(V)=\operatorname{SOGr}_{k}^{+}(V)=\operatorname{SOFl}_{(k,k)}^{+}(V)$ and $\operatorname{SOGr}_{\max}^{-}(V)=\operatorname{SOGr}_{k}^{-}(V)=\operatorname{SOFl}_{(k,k)}^{-}(V)$.

Observe that a flag variety $\operatorname{SpFl}_{\mathbf{a}}(V)$ (or $\operatorname{SOFl}_{\mathbf{a}}(V)$) is nontrivial if and only if so is the corresponding symmetric composition $\mathbf{a}$.

5.4. Nilpotent orbits in the symplectic and orthogonal Lie algebra and their relation to flag varieties

We retain the notation introduced in § 5.3.

For every partition $\mathbf{a}=(a_{1},\ldots,a_{p})\in\mathcal{P}_{\varepsilon}(d)$, let $O_{\mathbf{a}}$ be the set of all matrices in $\mathfrak{g}_{\varepsilon}$ whose Jordan normal form has zeros on the diagonal and the block sizes are $a_{1},\ldots,a_{p}$ up to permutation.

In the next theorem, which provides a parametrization of nilpotent orbits in the symplectic and orthogonal Lie algebra, parts (a) and (b) were obtained in [Ger, Ch. II, § 1] and [SpSt, Ch. IV, 2.27(ii)], respectively; see also [CM, § 5.1].

Theorem 5.5**.**

The following assertions hold.

1. (a)

The map $\mathbf{a}\mapsto O_{\mathbf{a}}$ is a bijection between the set $\mathcal{P}_{\varepsilon}(d)$ and the nilpotent $\widetilde{G}_{\varepsilon}$-orbits in $\mathfrak{g}_{\varepsilon}$. 2. (b)

For every $\varepsilon\in\{\pm 1\}$ and $\mathbf{a}\in\mathcal{P}_{\varepsilon}(d)$, the set $O_{\mathbf{a}}$ is a single $G_{\varepsilon}$-orbit unless $\varepsilon=1$ and $\mathbf{a}$ is very even, in which case $O_{\mathbf{a}}$ is a union of two $G_{\varepsilon}$-orbits $O_{\mathbf{a}}^{+}$ and $O_{\mathbf{a}}^{-}$.

The following theorem was obtained in [Ger, Ch. III, § 3] and [Hes, Theorem 3.10]; see also [CM, Theorem 6.2.5].

Theorem 5.6**.**

Suppose that $\mathbf{a},\mathbf{b}\in\mathcal{P}_{\varepsilon}(d)$ and $N_{\mathbf{a}}$ (resp. $N_{\mathbf{b}}$) is a $G_{\varepsilon}$-orbit in $O_{\mathbf{a}}$ (resp. $O_{\mathbf{b}}$). Then the following conditions are equivalent:

1. (1)

$N_{\mathbf{a}}\subsetneq\overline{N}_{\mathbf{b}}$. 2. (2)

$\mathbf{a}\prec\mathbf{b}$.

Proposition 5.7** ([Kem, § 2.2, Main Lemma]).**

For every $\mathbf{a}\in\mathcal{P}(d)$ and $\varepsilon\in\{\pm 1\}$, there exists a unique $\mathbf{a}^{\sharp}_{\varepsilon}\in\mathcal{P}_{\varepsilon}(d)$ such that $\mathbf{a}^{\sharp}_{\varepsilon}\preccurlyeq\mathbf{a}$ and $\mathbf{b}\preccurlyeq\mathbf{a}^{\sharp}_{\varepsilon}$ for all $\mathbf{b}\in\mathcal{P}_{\varepsilon}(d)$ with $\mathbf{b}\preccurlyeq\mathbf{a}$.

We refer to the partition $\mathbf{a}^{\sharp}_{\varepsilon}$ as the $\varepsilon$-collapse of $\mathbf{a}$.

The proof of the above proposition in [Kem] contains an explicit algorithm for constructing the partition $\mathbf{a}^{\sharp}_{\varepsilon}$ from the initial partition $\mathbf{a}=(a_{1},\ldots,a_{p})\in\mathcal{P}(d)$. We reproduce its steps below.

If $\mathbf{a}\in\mathcal{P}_{\varepsilon}(d)$ then put $\mathbf{a}^{\sharp}_{\varepsilon}=\mathbf{a}$ and exit. 2. 2)

If $\mathbf{a}\notin\mathcal{P}_{\varepsilon}(d)$ then define

[TABLE]

One automatically has $\varepsilon(-1)^{a_{m+1}}=1$ and $a_{m+1}>a_{m+2}$. 3. 3)

Define

[TABLE]

the minimum always exists. 4. 4)

Define a new partition $\mathbf{a}^{\prime}=(a^{\prime}_{1},a^{\prime}_{2},\ldots)$ as follows: $a^{\prime}_{m+1}=a_{m+1}-1$, $a^{\prime}_{l}=a_{l}+1$, $a^{\prime}_{i}=a_{i}$ for $i\neq m+1,l$. 5. 5)

Repeat the algorithm for $\mathbf{a}^{\prime}$.

For each composition $\mathbf{a}=(a_{1},\ldots,a_{p})$ of $d$ one defines the dual partition $\mathbf{a}^{\top}=(\widehat{a}_{1},\ldots,\widehat{a}_{q})$ of $d$ by the following rule:

[TABLE]

The operation $\mathbf{a}\mapsto\mathbf{a}^{\top}$ is an involution on the set $\mathcal{P}(d)$.

Proposition 5.8** ([Kem, § 3.4, Corollary 2]).**

Suppose that $\mathbf{a}$ is a symmetric composition of $d$ and $X$ is a $G_{\varepsilon}$-orbit in $\operatorname{Fl}^{\varepsilon}_{\mathbf{a}}(V)$. Then $\mathscr{N}(X)$ is a $G_{\varepsilon}$-orbit in $O_{(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}}$.

Corollary 5.9**.**

Suppose that $\mathbf{a},\mathbf{b}$ are two symmetric compositions of $d$, $X$ is a $G_{\varepsilon}$-orbit in $\operatorname{Fl}^{\varepsilon}_{\mathbf{a}}(V)$, and $Y$ is a $G_{\varepsilon}$-orbit in $\operatorname{Fl}^{\varepsilon}_{\mathbf{b}}(V)$. Then the following conditions are equivalent:

1. (1)

\mbox{[[}X\mbox{]]}\prec\mbox{[[}Y\mbox{]]}. 2. (2)

$(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}\prec(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$.

Proof.

This follows from Theorem 5.6 and Proposition 5.8. ∎

5.5. An analysis of the partial order on $\mathscr{F}(G)/\sim$ for $G\simeq\operatorname{Sp}_{2n}$

In this subsection, we assume that $G=\operatorname{Sp}(V)$ with $\dim V=2n$ and $n\geq 2$. Throughout this subsection, $\varepsilon=-1$.

Proposition 5.10**.**

Suppose that $X\in\mathscr{F}(G)$. Then \mbox{[[}X\mbox{]]}\succcurlyeq\mbox{[[}\mathbb{P}(V)\mbox{]]}. Moreover, $\mathbb{P}(V)$ is the unique element in \mbox{[[}\mathbb{P}(V)\mbox{]]} for $n\geq 3$ and \mbox{[[}\mathbb{P}(V)\mbox{]]}=\{\mathbb{P}(V),\operatorname{SpGr}_{\max}(V)\} for $n=2$.

Proof.

We apply Corollary 5.9. Put $\mathbf{a}=(1,2n-2,1)$. Then $\operatorname{SpFl}_{\mathbf{a}}(V)=\operatorname{SpGr}_{1}(V)\simeq\mathbb{P}(V)$, $\mathbf{a}^{\top}=[3,1^{2n-3}]$, and $(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}=[2^{2},1^{2n-4}]$. Let $\mathbf{b}$ be the symmetric composition of $2n$ such that $X=\operatorname{SpFl}_{\mathbf{b}}(V)$ and assume $\mathbf{b}\neq\mathbf{a}$. If $|\mathbf{b}|=2$ then $\mathbf{b}=(n,n)$ and $\mathbf{b}^{\top}=(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}=[2^{n}]$, so that $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$ for $n=2$ and $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$ for $n\geq 3$. If $|\mathbf{b}|=3$ then $\mathbf{b}^{\top}$ is of the form $(3,3,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ is also of the form $(3,3,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. If $|\mathbf{b}|=l\geq 4$, then $\mathbf{b}^{\top}$ is of the form $(l,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succcurlyeq[4,1^{2n-4}]\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. ∎

Proposition 5.11**.**

Suppose that $X\in\mathscr{F}(G)$ and $X\neq\mathbb{P}(V)$. Then $X=\operatorname{SpGr}_{\max}(V)$, or $X=\operatorname{SpGr}_{2}(V)$, or \mbox{[[}X\mbox{]]}\succ\mbox{[[}\operatorname{SpGr}_{2}(V)\mbox{]]}.

Proof.

If $n=2$ then \mbox{[[}\mathbb{P}(V)\mbox{]]}=\mbox{[[}\operatorname{SpGr}_{2}(V)\mbox{]]} and the claim is implied by Proposition 5.10, hence in what follows we assume $n\geq 3$. Put $\mathbf{a}=(2,2n-4,2)$. Then $\operatorname{SpFl}_{\mathbf{a}}(V)=\operatorname{SpGr}_{2}(V)$ and $\mathbf{a}^{\top}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}=[3^{2},1^{2n-6}]$. Let $\mathbf{b}$ be the symmetric composition of $2n$ such that $X=\operatorname{SpFl}_{\mathbf{b}}(V)$. If $|\mathbf{b}|=2$ then $X=\operatorname{SpGr}_{\max}(V)$. Now consider the case $|\mathbf{b}|=3$, so that $\mathbf{b}=(b_{1},b_{2},b_{1})$. As $X\neq\mathbb{P}(V)$, we have $b_{1}\neq 1$. If $b_{1}=2$ then $X=\operatorname{SpGr}_{2}(V)$. If $b_{1}\geq 3$ then $\mathbf{b}^{\top}$ is of the form $(3,3,3,\ldots)$ or $(3,3,2,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ is also of the form $(3,3,3,\ldots)$ or $(3,3,2,\ldots)$, which implies $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. If $|\mathbf{b}|\geq 4$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succcurlyeq[4,2,1^{2n-6}]\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. ∎

5.6. An analysis of the partial order on $\mathscr{F}(G)/\sim$ for $G\simeq\operatorname{Spin}_{2n+1}$

In this subsection, we assume that $G=\operatorname{Spin}(V)$ with $\dim V=2n+1$ and $n\geq 1$. Throughout this subsection, $\varepsilon=1$.

Proposition 5.12**.**

Suppose that $X\in\mathscr{F}(G)$. Then \mbox{[[}X\mbox{]]}\succcurlyeq\mbox{[[}\operatorname{SOGr}_{1}(V)\mbox{]]}. Moreover, $\operatorname{SOGr}_{1}(V)$ is the unique element in \mbox{[[}\operatorname{SOGr}_{1}(V)\mbox{]]} for $n\neq 2$ and

[TABLE]

for $n=2$.

Proof.

If $n=1$ then $\mathscr{F}(G)=\{\operatorname{SOGr}_{1}(V)\}$ and the assertion holds trivially, hence in what follows we assume $n\geq 2$. Put $\mathbf{a}=(1,2n-1,1)$. Then $\operatorname{SOFl}_{\mathbf{a}}(V)=\operatorname{SOGr}_{1}(V)$ and $\mathbf{a}^{\top}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}=[3,1^{2n-2}]$. Let $\mathbf{b}$ be the symmetric composition of $2n+1$ such that $X=\operatorname{SOFl}_{\mathbf{b}}(V)$ and assume $\mathbf{b}\neq\mathbf{a}$. First consider the case $|\mathbf{b}|=3$, so that $\mathbf{b}=(b_{1},b_{2},b_{1})$ with $b_{1}\geq 2$. If $b_{2}=1$ then $\mathbf{b}^{\top}=[3,2^{n-1}]$, so that $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$ for $n=2$ and $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$ for $n\geq 3$. If $b_{2}>1$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ is of the form $(3,3,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. In the case $|\mathbf{b}|=l\geq 5$ the partition $\mathbf{b}^{\top}$ has the form $(l,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succcurlyeq[5,1^{2n-4}]\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. ∎

Proposition 5.13**.**

Suppose that $n\geq 2$, $X\in\mathscr{F}(G)$, and $X\neq\operatorname{SOGr}_{1}(V)$. Then $X=\operatorname{SOGr}_{\max}(V)$, or $X=\operatorname{SOGr}_{2}(V)$, or \mbox{[[}X\mbox{]]}\succ\mbox{[[}\operatorname{SOGr}_{2}(V)\mbox{]]}.

Proof.

If $n=2$ then \mbox{[[}\operatorname{SOGr}_{1}(V)\mbox{]]}=\mbox{[[}\operatorname{SOGr}_{\max}(V)\mbox{]]} and the claim is implied by Proposition 5.12, hence in what follows we assume $n\geq 3$. Put $\mathbf{a}=(2,2n-3,2)$. Then $\operatorname{SOFl}_{\mathbf{a}}(V)=\operatorname{SOGr}_{2}(V)$ and $\mathbf{a}^{\top}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}=[3^{2},1^{2n-5}]$. Let $\mathbf{b}$ be the symmetric composition of $2n+1$ such that $X=\operatorname{SOFl}_{\mathbf{b}}(V)$. First consider the case $|\mathbf{b}|=3$, so that $\mathbf{b}=(b_{1},b_{2},b_{1})$. As $X\neq\operatorname{SOGr}_{1}(V)$, we have $b_{1}\neq 1$. If $b_{1}=2$ then $X=\operatorname{SOGr}_{2}(V)$. If $b_{2}=1$ then $X=\operatorname{SOGr}_{\max}(V)$. If $b_{1}\geq 3$ and $b_{2}\geq 2$ then $b_{2}\geq 3$ and $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ has the form $(3,3,3,\ldots)$, which implies $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. In the case $|\mathbf{b}|=l\geq 5$ we have $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succcurlyeq[5,1^{2n-4}]\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. ∎

5.7. An analysis of the partial order on $\mathscr{F}(G)/\sim$ for $G\simeq\operatorname{Spin}_{2n}$

In this subsection, we assume that $G=\operatorname{Spin}(V)$ with $\dim V=2n$ and $n\geq 2$. Throughout this subsection, $\varepsilon=1$.

Proposition 5.14**.**

Suppose that $X\in\mathscr{F}(G)$. Then $X=\operatorname{SOGr}_{1}(V)$, or $X=\operatorname{SOGr}^{+}_{\max}(V)$, or $X=\operatorname{SOGr}^{-}_{\max}(V)$, or \mbox{[[}X\mbox{]]}\succ\mbox{[[}\operatorname{SOGr}_{1}(V)\mbox{]]}.

Proof.

If $n=2$ then $\mathscr{F}(G)=\{\operatorname{SOGr}_{1}(V),\operatorname{SOGr}^{+}_{\max}(V),\operatorname{SOGr}^{-}_{\max}(V)\}$ and the assertion holds trivially, so in what follows we assume $n\geq 3$. Put $\mathbf{a}=(1,2n-2,1)$. Then $\operatorname{SOFl}_{\mathbf{a}}(V)=\operatorname{SOGr}_{1}(V)$ and $\mathbf{a}^{\top}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}=[3,1^{2n-3}]$. Let $\mathbf{b}$ be the symmetric composition of $2n$ corresponding to $X$.

If $|\mathbf{b}|=2$ then $\mathbf{b}=(n,n)$ and $X$ is one of $\operatorname{SOGr}^{+}_{\max}(V)$ or $\operatorname{SOGr}^{-}_{\max}(V)$.

Suppose that $|\mathbf{b}|=3$, so that $\mathbf{b}=(b_{1},b_{2},b_{1})$. If $b_{1}=1$ then $X=\operatorname{SOGr}_{1}(V)$. If $b_{1}\geq 2$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ has the form $(3,3,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$.

Suppose that $|\mathbf{b}|=4$. Then $\mathbf{b}^{\top}$ is of the form $(4,4,\ldots)$ or $(4,2,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ is of the form $(4,4,\ldots)$ or $(3,3,\ldots)$, which implies $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$.

Finally, in the case $|\mathbf{b}|\geq 5$ we have $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succcurlyeq[5,1^{2n-5}]\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. ∎

Proposition 5.15**.**

Suppose that $n\geq 3$, $X\in\mathscr{F}(G)$, $X\neq\operatorname{SOGr}_{1}(V)$, $X\neq\operatorname{SOGr}^{+}_{\max}(V)$, and $X\neq\operatorname{SOGr}^{-}_{\max}(V)\}$. Then \mbox{[[}X\mbox{]]}\succcurlyeq\mbox{[[}\operatorname{SOGr}_{2}(V)\mbox{]]}. Moreover, $\operatorname{SOGr}_{2}(V)$ is the unique element in \mbox{[[}\operatorname{SOGr}_{2}(V)\mbox{]]} for $n\geq 5$,

[TABLE]

for $n=4$, and

[TABLE]

for $n=3$.

Proof.

Put $\mathbf{a}=(2,2n-4,2)$. Then $\operatorname{SOFl}_{\mathbf{a}}(V)=\operatorname{SOGr}_{2}(V)$ and $\mathbf{a}^{\top}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}=[3^{2},1^{2n-6}]$. Let $\mathbf{b}$ be the symmetric composition of $2n$ corresponding to $X$. As $X\neq\operatorname{SOGr}^{+}_{\max}(V)$ and $X\neq\operatorname{SOGr}^{-}_{\max}(V)$, we have $|\mathbf{b}|\geq 3$.

Suppose that $|\mathbf{b}|=3$, so that $\mathbf{b}=(b_{1},b_{2},b_{1})$. As $X\neq\operatorname{SOGr}_{1}(V)$, we have $b_{1}\neq 1$. If $b_{1}=2$ then $X=\operatorname{SOGr}_{2}(V)$. If $b_{1}=3$ and $b_{2}=2$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$, hence $X=\operatorname{SOGr}_{3}(V)\sim\operatorname{SOGr}_{2}(V)$. If either $b_{1}=3$, $b_{2}\geq 3$ or $b_{1}\geq 4$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ has the form $(3,3,3,\ldots)$ or $(3,3,2,2,\ldots)$, which implies $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$.

Suppose that $|\mathbf{b}|=4$, so that $\mathbf{b}=(b_{1},b_{2},b_{2},b_{1})$. If $b_{1}=1$ and $b_{2}=2$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$ and we get X\in\{\operatorname{SOFl}^{\pm}_{(1,2,2,1)}(V)\}\subset\mbox{[[}\operatorname{SOGr}_{2}(V)\mbox{]]}. If $b_{1}=1$ and $b_{2}=3$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}=(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$ and we get X\in\{\operatorname{SOFl}^{\pm}_{(1,3,3,1)}(V)\}\subset\mbox{[[}\operatorname{SOGr}_{2}(V)\mbox{]]}. If $b_{2}=1$ and $b_{1}=2$ then $X=\operatorname{SOGr}_{2}(V)$. If $b_{2}=1$ and $b_{1}=3$ then $X=\operatorname{SOGr}_{3}(V)$. If either $b_{1}=1$, $b_{2}\geq 4$ or $b_{2}=1$, $b_{1}\geq 4$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ has the form $(3,3,2,2,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. If $b_{1},b_{2}\geq 2$ then $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}$ has the form $(4,4,\ldots)$, hence $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$.

In the case $|\mathbf{b}|\geq 5$ we have $(\mathbf{b}^{\top})^{\sharp}_{\varepsilon}\succcurlyeq[5,1^{2n-5}]\succ(\mathbf{a}^{\top})^{\sharp}_{\varepsilon}$. ∎

5.8. Checking $H$-sphericity for a given flag variety

Let $H\subset G$ be a connected reductive subgroup of $G$ and let $I\subset S$ be an arbitrary subset. In this subsection, we present a criterion which enables one to check $H$-sphericity of $X_{I}$ effectively. This criterion is based on results of Panyushev [Pan]; see details in [AvPe2, § 4.3].

Proposition 5.16** ([AvPe2, Proposition 4.6]).**

Let $K$ be a connected reductive group. Suppose that $X$ is a smooth complete irreducible $K$-variety, $Y\subset X$ is a closed $K$-orbit, $y\in Y$, and $M$ is a Levi subgroup of $K_{y}$. Then the following conditions are equivalent:

1. (1)

$X$* is a $K$-spherical variety.* 2. (2)

$T_{y}X/T_{y}Y$* is a spherical $M$-module.*

Moreover, under the above two conditions one has $\operatorname{rk}_{K}X=\operatorname{rk}_{M}(T_{y}X/T_{y}Y)$.

It is well known that, under an appropriate choice of $H$ within its conjugacy class in $G$, one can achieve the inclusion $B_{H}^{-}\subset B_{G}^{-}$. In this situation, Proposition 5.16 combined with Theorem 3.6(b) yield the following result, which is widely used throughout this paper in explicit calculations.

Proposition 5.17** (see [AvPe2, Corollary 4.8]).**

Suppose that $B_{H}^{-}\subset B_{G}^{-}$ and $M$ is a Levi subgroup of $P_{I}^{-}\cap H$. Then the following conditions are equivalent:

1. (1)

$X_{I}$* is an $H$-spherical variety.* 2. (2)

$\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h})$* is a spherical $M$-module.*

Moreover, under the above conditions one has

[TABLE]

6. Classification in the symplectic case

6.1. Preliminary remarks

Throughout this section, we assume that $V$ is a vector space of even dimension $d\geq 4$ equipped with a nondegenerate skew-symmetric bilinear form $\omega$ and $H$ is a connected reductive subgroup of $G=\operatorname{Sp}(V)$.

When referring to the classification of spherical modules, we always use the list in [Kno, § 5] (see also [AvPe1, Theorems 5.1–5.2]) and the general criterion provided by [AvPe1, Theorem 5.3]. One useful consequence of this classification is the following lemma, for which we provide a proof in order to demonstrate application of the above-cited sources in an example.

Lemma 6.1**.**

Consider the group $K=\operatorname{GL}_{n_{1}}\times\ldots\times\operatorname{GL}_{n_{k}}$ with $n_{1}\geq\ldots\geq n_{k}\geq 1$ and the $K$-module

[TABLE]

Then $W$ is $K$-spherical if and only if one of the following two conditions holds:

1. (1)

$k=2$; 2. (2)

$k=3$* and $n_{2}=n_{3}=1$.*

Proof.

If $k=2$ then $W=\mathbb{F}^{n_{1}}\otimes\mathbb{F}^{n_{2}}$, the latter module appearing in the list of [Kno, § 5] hence being $K$-spherical. In what follows we assume $k\geq 3$. If $n_{1}=\ldots=n_{k}=1$ then the pair $(K,W)$ is equivalent to

[TABLE]

By [AvPe1, Theorem 5.3], the latter module is spherical if and only if all characters in the multiset $\{\chi_{i}+\chi_{j}\mid 1\leq i<j\leq k\}$ are linearly independent, which holds only for $k=3$. Now we assume $n_{1}\geq 2$. If $k\geq 4$ then $W$ contains the $K$-submodule $\bigoplus\limits_{2\leq j\leq k}\mathop{\vphantom{|}\mathbb{F}^{n_{1}}}\limits_{1}{\!}\otimes\mathop{\vphantom{|}\mathbb{F}^{n_{j}}}\limits_{j}{\!}$, which is saturated and indecomposable but not present in the list of [Kno, § 5], hence not spherical. It remains to consider the case $k=3$. If $n_{2}\geq 2$ then $W$ is saturated, indecomposable but not present in the list of [Kno, §,5], hence not spherical. Finally, for $n_{2}=n_{3}=1$ the pair $(K,V)$ is equivalent to

[TABLE]

By [AvPe1, Theorem 5.3], the sphericity conditions for the latter module are as follows:

• •

$\chi_{1}+\chi_{2},\ \chi_{1}+\chi_{3},\ \chi_{2}+\chi_{3}$ are linearly independent if $n_{1}=2$;

• •

$\chi_{2}-\chi_{3},\ \chi_{2}+\chi_{3}$ are linearly independent if $n_{1}\geq 3$.

As both conditions hold, the proof is completed. ∎

For explicit calculations involving the subgroup $H$, we use the following conventions. First, we identify $V$ with $\mathbb{F}^{d}$. Second, suppose that $V$ is written as $V=V_{1}\oplus\ldots\oplus V_{m}$ where all direct summands are pairwise orthogonal with respect to $\omega$ and each $V_{i}$ is an $H$-module that is either simple or weakly reducible. If $\dim V_{1}=2k$ then $V_{1}$ is embedded in $V$ as the linear span of the vectors $e_{1},\ldots,e_{k},e_{d-k+1},\ldots,e_{d}$ and $V_{2}\oplus\ldots\oplus V_{m}$ is embedded as the linear span of the vectors $e_{k+1},\ldots,e_{d-k}$. Moreover, if $V_{1}$ is weakly reducible of the form $\Omega(W_{1})$ then we assume in addition that the $H$-submodule $W_{1}\subset V$ is the linear span of the vectors $e_{1},\ldots,e_{k}$ and the $H$-submodule $W_{1}^{*}\subset V$ is the linear span of the vectors $e_{d-k+1},\ldots,e_{d}$. The embeddings of $V_{2}$, …, $V_{m}$ in $V$ are determined by iterating the above procedure.

For checking sphericity of a given $H$-variety we always use Proposition 5.17 with $B_{H}^{-}=B_{G}^{-}\cap H$. The above conventions on $H$ always guarantee that $B_{G}^{-}\cap H$ is a Borel subgroup of $H$. It is well known that in the case $H=G$ every flag variety of $G$ is $H$-spherical, this fact will be used without extra explanation.

In §§ 6.2–6.4, our classification of spherical actions on flag varieties of $G$ involves all possible subgroups $H$ including symmetric subgroups and Levi subgroups. On the contrary, in § 6.5 we exclude symmetric subgroups and Levi subgroups referring to [AvPe2, § 5].

6.2. Spherical actions on $\mathbb{P}(V)$

The starting point of our classification in the symplectic case is the following result.

Theorem 6.2**.**

The variety $\mathbb{P}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to a pair in Table 5.

Proof.

The variety $\mathbb{P}(V)$ is $H$-spherical if and only if $V$ is a spherical $(H\times\mathbb{F}^{\times})$-module, where $\mathbb{F}^{\times}$ acts on $V$ by scalar transformations.

We may assume that the pair $(H,V)$ is BF-equivalent to a pair

[TABLE]

where $p,q\geq 0$ and each $V_{i}$ and $W_{j}$ are simple $H$-modules.

If $V$ is a spherical $(H\times\mathbb{F}^{\times})$-module then each $V_{i}$ and each $W_{i}$ is also a spherical $(H\times\mathbb{F}^{\times})$-module. Then by Proposition 3.3 the image of $H$ in $\operatorname{GL}(V_{i})$ coincides with $\operatorname{Sp}(V_{i})$ for all $i=1,\ldots,p$. Similarly, for each $i=1,\ldots,q$ the pair $(H,W_{i})$ is equivalent to one of $(\operatorname{GL}_{n},\mathbb{F}^{n})$ ($n\geq 1$), $(\operatorname{SL}_{n},\mathbb{F}^{n})$ ($n\geq 3$), or $(\operatorname{Sp}_{2n}\times\mathbb{F}^{\times},[\mathbb{F}^{2n}]_{\chi})$ ($n\geq 2$).

Next we show that $q\leq 1$. Indeed, otherwise $\Omega(W_{1})\oplus\Omega(W_{2})$ would be a spherical $(H\times\mathbb{F}^{\times})$-module, hence $W_{1}\oplus W_{1}^{*}\oplus W_{2}\oplus W_{2}^{*}$ would be a spherical $(\operatorname{GL}(W_{1})\times\operatorname{GL}(W_{2})\times\mathbb{F}^{\times})$-module (where $\mathbb{F}^{\times}$ acts by scalar transformations), which is not the case.

Now suppose there is a simple factor of $H$ that acts nontrivially on some $V_{i}$ and some other summand. According to the classification, this other summand cannot be $W_{1}$; neither can it be $V_{j}$ with $j\neq i$ since otherwise $V_{i}\simeq V_{j}$ as $(H\times\mathbb{F}^{\times})$-modules, in which case $V_{i}\oplus V_{j}$ cannot be a spherical $(H\times\mathbb{F}^{\times})$-module.

It follows from the above arguments that $V$ can be a spherical $(H\times\mathbb{F}^{\times})$-module only if the pair $(H,V)$ is BF-equivalent to a pair in Table 5. On the other hand, for each of these pairs the $(H\times\mathbb{F}^{\times})$-module $V$ is spherical. ∎

6.3. Spherical actions on $\operatorname{SpGr}_{\max}(V)$

Recall from § 5.3 that $\operatorname{SpGr}_{\max}(V)\simeq X_{I}$ with $I=\{d/2\}$. If $\operatorname{SpGr}_{\max}(V)$ is an $H$-spherical variety then $\mathbb{P}(V)$ should be also $H$-spherical in view of Proposition 5.10 and Theorem 5.3. Consequently, by Theorem 6.2 it suffices to consider only the cases listed in Table 5.

Proposition 6.3**.**

Suppose that the pair $(H,V)$ is BF-equivalent to that in Case 5 of Table 5 with $n_{1}\geq n_{2}\geq\ldots\geq n_{k}\geq 1$. Then $\operatorname{SpGr}_{\max}(V)$ is $H$-spherical if and only if one of the following conditions holds:

1. (1)

$k\leq 2$; 2. (2)

$k=3$* and $n_{2}=n_{3}=1$.*

Proof.

If $k=1$ then $\operatorname{SpGr}_{\max}(V)$ is $H$-spherical. If $k\geq 2$ then it is easy to see that the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

By Lemma 6.1, the latter module is spherical if and only if either $k=2$ or condition (2) holds. ∎

Proposition 6.4**.**

Suppose that the pair $(H,V)$ is BF-equivalent to that in Case 5 of Table 5. Then $\operatorname{SpGr}_{\max}(V)$ is $H$-spherical if and only if one of the following two conditions holds:

1. (1)

$k=0$; 2. (2)

$k=m=1$.

Proof.

Without loss of generality we may assume $n_{1}\geq n_{2}\geq\ldots\geq n_{k}\geq 1$. If $H$ acts spherically on $X=\operatorname{SpGr}_{\max}(V)$ then the group $\operatorname{Sp}_{2n_{1}}\times\ldots\times\operatorname{Sp}_{2n_{k}}\times\operatorname{Sp}_{2m}$ also acts spherically on $X$. Then Proposition 6.3 implies that $k\leq 2$ and the following cases may occur.

Case 1: $k=0$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{GL}_{m},\mathrm{S}^{2}\mathbb{F}^{m})$, the latter module being spherical.

Case 2: $k=1$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $m=1$.

Case 3: $k=2$, $n_{1}=n_{2}=1$. If $X$ is $H$-spherical then $X$ is $(\operatorname{Sp}_{4}\times\operatorname{GL}_{m})$-spherical, which implies $m=1$ by the previous case. Then $\dim B_{H}=5<6=\dim X$, hence $X$ is not $H$-spherical.

Case 4: $k=2$, $n_{2}=m=1$. In this case, the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical. ∎

Proposition 6.5**.**

Suppose that the pair $(H,V)$ is BF-equivalent to that in Case 5 of Table 5. Then $\operatorname{SpGr}_{\max}(V)$ is not $H$-spherical.

Proof.

If $H$ acts spherically on $X=\operatorname{SpGr}_{\max}(V)$ then the group $\operatorname{Sp}_{2n_{1}}\times\ldots\times\operatorname{Sp}_{2n_{k}}\times\operatorname{GL}_{m}$ also acts spherically on $X$. Then Proposition 6.4 leaves us with the following two cases.

Case 1: $k=0$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{SL}_{m},\mathrm{S}^{2}\mathbb{F}^{m})$, the latter module being not spherical.

Case 2: $k=m=1$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{GL}_{n_{1}},\mathop{\mathbb{F}^{n_{1}}}\limits_{1}{\!}\oplus\mathbb{F}^{1})$, the latter module being not spherical. ∎

Proposition 6.6**.**

Suppose that the pair $(H,V)$ is BF-equivalent to that in Case 5 of Table 5. Then $\operatorname{SpGr}_{\max}(V)$ is not $H$-spherical.

Proof.

If $H$ acts spherically on $X=\operatorname{SpGr}_{\max}(V)$ then the group $\operatorname{Sp}_{2n_{1}}\times\ldots\times\operatorname{Sp}_{2n_{k}}\times\operatorname{GL}_{2m}$ also acts spherically on $X$. By Proposition 6.4, the latter is possible only if $k=0$. In this case, the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{Sp}_{2m}\times\mathbb{F}^{\times},[\mathrm{S}^{2}\mathbb{F}^{2m}]_{2\chi})$, the latter module being not spherical. ∎

Summarizing the results obtained in Propositions 6.3–6.6 and comparing them with the statements in § 4.2, we arrive at

Corollary 6.7**.**

Suppose that $H$ is neither a symmetric subgroup nor a Levi subgroup of $\operatorname{Sp}(V)$. Then $\operatorname{SpGr}_{\max}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to $(\operatorname{Sp}_{2m}\times\operatorname{SL}_{2}\times\operatorname{SL}_{2},\mathop{\mathbb{F}^{2m}}\limits_{1}{\!}\oplus\mathop{\mathbb{F}^{2}}\limits_{2}{\!}\oplus\mathop{\mathbb{F}^{2}}\limits_{3}{\!})$ with $m\geq 1$.

6.4. Spherical actions on $\operatorname{SpGr}_{2}(V)$

Recall from § 5.3 that $\operatorname{SpGr}_{2}(V)\simeq X_{I}$ with $I=\{2\}$. If $\operatorname{SpGr}_{2}(V)$ is an $H$-spherical variety then $\mathbb{P}(V)$ should be also $H$-spherical in view of Proposition 5.10 and Theorem 5.3. Consequently, by Theorem 6.2 it suffices to consider only the cases listed in Table 5.

Proposition 6.8**.**

Suppose that the pair $(H,V)$ is BF-equivalent to that in Case 5 of Table 5. Then $\operatorname{SpGr}_{2}(V)$ is $H$-spherical if and only if $k\leq 2$.

Proof.

If $k=1$ then $\operatorname{SpGr}_{2}(V)$ is $H$-spherical. In what follows we assume $k\geq 2$ and $n_{1}\geq n_{2}\geq\ldots\geq n_{k}\geq 1$ without loss of generality.

If $n_{1}=\ldots=n_{k}=1$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $k=2$.

If $n_{1}\geq 2$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $k=2$. ∎

Proposition 6.9**.**

Suppose that the pair $(H,V)$ is BF-equivalent to that in Case 5 of Table 5. Then $\operatorname{SpGr}_{2}(V)$ is $H$-spherical if and only if one of the following two conditions holds:

1. (1)

$k=0$, $m=2$. 2. (2)

$k=m=1$.

Proof.

If $H$ acts spherically on $X=\operatorname{SpGr}_{2}(V)$ then the group $\operatorname{Sp}_{2n_{1}}\times\ldots\times\operatorname{Sp}_{2n_{k}}\times\operatorname{Sp}_{2m}$ also acts spherically on $X$. Then Proposition 6.8 leaves us with the following two cases.

Case 1: $k=0$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $m=2$.

Case 2: $k=1$. If $n_{1}=1$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $m=1$. If $n_{1}\geq 2$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\nobreak\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $m=1$. ∎

Proposition 6.10**.**

Suppose that the pair $(H,V)$ is BF-equivalent to that in Case 5 of Table 5. Then $\operatorname{SpGr}_{2}(V)$ is not $H$-spherical.

Proof.

If $H$ acts spherically on $X=\operatorname{SpGr}_{2}(V)$ then the group $\operatorname{Sp}_{2n_{1}}\times\ldots\times\operatorname{Sp}_{2n_{k}}\times\operatorname{GL}_{m}$ also acts spherically on $X$. As $m\geq 3$, the latter is impossible by Proposition 6.9. ∎

Proposition 6.11**.**

Suppose that the pair $(H,V)$ is BF-equivalent to that in Case 5 of Table 5. Then $\operatorname{SpGr}_{2}(V)$ is not $H$-spherical.

Proof.

If $H$ acts spherically on $X=\operatorname{SpGr}_{2}(V)$ then the group $\operatorname{Sp}_{2n_{1}}\times\ldots\times\operatorname{Sp}_{2n_{k}}\times\operatorname{GL}_{2m}$ also acts spherically on $X$. As $m\geq 2$, the latter is impossible by Proposition 6.9. ∎

Summarizing the results obtained in Propositions 6.8–6.11 and comparing them with the statements in § 4.2, we arrive at

Corollary 6.12**.**

If $\operatorname{SpGr}_{2}(V)$ is $H$-spherical then $H$ is either a symmetric subgroup or a Levi subgroup of $\operatorname{Sp}(V)$.

6.5. Completion of the classification

We conclude our classification in the symplectic case by

Proposition 6.13**.**

Let $X$ be a nontrivial flag variety of $\operatorname{Sp}(V)$ different from $\mathbb{P}(V)$, $\operatorname{SpGr}_{2}(V)$, and $\operatorname{SpGr}_{\max}(V)$. If $X$ is $H$-spherical then $H$ is either a symmetric subgroup or a Levi subgroup of $\operatorname{Sp}(V)$.

Proof.

By Proposition 5.11 and Theorem 5.3, $X$ being $H$-spherical implies $\operatorname{SpGr}_{2}(V)$ being $H$-spherical. Then the assertion follows from Corollary 6.12. ∎

7. Classification in the orthogonal case

7.1. Preliminary remarks

Throughout this section, we assume that $V$ is a vector space of dimension $d\geq 3$ equipped with a nondegenerate symmetric bilinear form $\omega$. We assume that $H$ is a connected reductive subgroup of $G=\operatorname{Spin}(V)$.

We fix a decomposition $V=V_{1}\oplus\ldots\oplus V_{r}$ where all direct summands are pairwise orthogonal with respect to $\omega$ and each $V_{i}$ is an $H$-module that is either simple or weakly reducible. For each $i=1,\ldots,r$ we put $d_{i}=\dim V_{i}$.

The image of $H$ in $\operatorname{SO}(V)$ is denoted by $H_{0}$. Note that either $H\simeq H_{0}$ or $H$ is a two-fold covering of $H_{0}$. For every $i=1,\ldots,r$, the projection of $H_{0}$ to $\operatorname{SO}(V_{i})$ is denoted by $H_{i}$.

As in § 6, when referring to the classification of spherical modules, we always use the list in [Kno, § 5] (see also [AvPe1, Theorems 5.1–5.2]) and the general criterion provided by [AvPe1, Theorem 5.3].

Suppose that $d$ is even and there is a decomposition $V=\widetilde{V}_{1}\oplus\widetilde{V}_{2}$ where the direct summands are pairwise orthogonal and $\dim\widetilde{V}_{2}=1$. In this situation, it is well known that the group $\operatorname{SO}(\widetilde{V}_{1})$ acts transitively on both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ and each of these is isomorphic to $\operatorname{SOGr}_{\max}(\widetilde{V}_{1})$ as an $\operatorname{SO}(\widetilde{V}_{1})$-variety. This leads to the following result, which will be used several times in this section.

Proposition 7.1**.**

Under the above notation, suppose that $H_{0}$ is a subgroup of $\operatorname{SO}(\widetilde{V}_{1})$. Then the following conditions are equivalent:

1. (1)

$\operatorname{SOGr}_{\max}^{+}(V)$* is $H$-spherical;* 2. (2)

$\operatorname{SOGr}_{\max}^{-}(V)$* is $H$-spherical;* 3. (3)

$\operatorname{SOGr}_{\max}(\widetilde{V}_{1})$* is $H$-spherical.*

For explicit calculations involving the subgroup $H$, we use the following conventions. First, we identify $V$ with $\mathbb{F}^{d}$. Second, suppose that $V$ is written as $V=\widetilde{V}_{1}\oplus\ldots\oplus\widetilde{V}_{m}$ where all direct summands are pairwise orthogonal with respect to $\omega$ and each $\widetilde{V}_{i}$ is an $H$-module that is either simple or weakly reducible. Unless otherwise specified, we use the following embeddings.

• •

If $\dim\widetilde{V}_{1}=2k$ then $\widetilde{V}_{1}$ is embedded in $V$ as the linear span of the vectors $e_{1},\ldots,e_{k},e_{d-k+1},\ldots,e_{d}$ and $\widetilde{V}_{2}\oplus\ldots\oplus\widetilde{V}_{m}$ is embedded as the linear span of the vectors $e_{k+1},\ldots,e_{d-k}$. Moreover, if $\widetilde{V}_{1}$ is weakly reducible of the form $\Omega(W_{1})$ then we assume in addition that the $H$-submodule $W_{1}\subset V$ is the linear span of the vectors $e_{1},\ldots,e_{k}$ and the $H$-submodule $W_{1}^{*}\subset V$ is the linear span of the vectors $e_{d-k+1},\ldots,e_{d}$.

• •

If $\dim\widetilde{V}_{1}=2k+1$ and $d=2l+1$ then $\widetilde{V}_{1}$ is embedded in $V$ as the linear span of the vectors $e_{l-k+1},\ldots,e_{l+k+1}$ and $\widetilde{V}_{2}\oplus\ldots\oplus\widetilde{V}_{m}$ is embedded as the linear span of the vectors $e_{1},\ldots,e_{l-k},e_{l+k+2},\ldots,e_{d}$.

• •

If $\dim\widetilde{V}_{1}=2k+1$ and $d=2l$ then $\widetilde{V}_{1}$ is embedded in $V$ as the linear span of the vectors $e_{1},\ldots,e_{k},e_{l}+e_{l+1},e_{d-k+1},\ldots,e_{d}$ and $\widetilde{V}_{2}\oplus\ldots\oplus\widetilde{V}_{m}$ is embedded as the linear span of the vectors $e_{k+1},\ldots,e_{l-1},e_{l}-e_{l+1},e_{l+2},\ldots,e_{d-k}$.

The embeddings of $\widetilde{V}_{2}$, …, $\widetilde{V}_{m}$ in $V$ are determined by iterating the above procedure. (In fact, the situation where $\dim\widetilde{V}_{1}$ is odd occurs only for $m=2$.)

The group $\mathsf{G}_{2}$ is always embedded in $\operatorname{SO}_{7}$ according to the embedding of the corresponding Lie algebras described in Appendix A.

The group $\operatorname{Spin}_{7}$ is always embedded in $\operatorname{SO}_{8}$ according to the embedding of the corresponding Lie algebras described in Appendix A. Since there are two conjugacy classes of $\operatorname{Spin}_{7}$ in $\operatorname{SO}_{8}$, to distinguish between them we write $\operatorname{Spin}^{+}_{7}$ and $\operatorname{Spin}^{-}_{7}$ where $\operatorname{Spin}^{+}_{7}$ refers to the above-described subgroup of $\operatorname{SO}_{8}$.

For the module $(\operatorname{Sp}_{2n}\times\operatorname{SL}_{2},\mathbb{F}^{2n}\otimes\mathbb{F}^{2})$, $n\geq 2$, the image of the group $\operatorname{Sp}_{2n}\times\operatorname{SL}_{2}$ in $\operatorname{SO}_{4n}$ is described as follows. Let $e_{1},\ldots,e_{2n}$ (resp. $f_{1},f_{2}$) be the standard basis of $\mathbb{F}^{2n}$ (resp. $\mathbb{F}^{2}$). Then the standard basis of $\mathbb{F}^{2n}\otimes\mathbb{F}^{2}$ providing the required embedding is $e_{1}\otimes f_{1}$, $e_{2}\otimes f_{1}$, $\ldots$, $e_{2n}\otimes f_{1}$, $e_{1}\otimes f_{2}$, $e_{2}\otimes f_{2}$, $\ldots$, $e_{2n}\otimes f_{2}$. For $n=2$, there are two conjugacy classes in $\operatorname{SO}_{8}$ of (the image of) $\operatorname{Sp}_{4}\times\operatorname{SL}_{2}$, and the above-described realization will be referred to as the default one.

For checking sphericity of a given $H$-variety we always use Proposition 5.17 with $B_{H}^{-}=B_{G}^{-}\cap H$. The above conventions on embeddings of $H$, $\mathsf{G}_{2}$, $\operatorname{Spin}_{7}$, and $\operatorname{Sp}_{2n}\times\operatorname{SL}_{2}$ always guarantee that $B_{G}^{-}\cap H$ is a Borel subgroup of $H$. It is well known that any flag variety of $G$ is $H$-spherical if the pair $(H,V)$ is equivalent to either $(\operatorname{SO}_{m},\mathbb{F}^{m})$ or $(\operatorname{Spin}_{7},\mathbb{F}^{8})$ (in the latter case, the subgroup $H\simeq\operatorname{Spin}_{7}$ is symmetric in $G\simeq\operatorname{Spin}_{8}$, and a suitable outer automorphism of $G$ takes $H$ to $\widetilde{H}$ such that the pair $(\widetilde{H},V)$ is equivalent to $(\operatorname{SO}_{7},\mathbb{F}^{7}\oplus\mathbb{F}^{1})$), this will be used without extra explanation.

In §§ 7.2–7.6, our classification of spherical actions on flag varieties of $G$ involves all possible subgroups $H$ including symmetric subgroups and Levi subgroups. On the contrary, in § 7.7 we exclude the cases where $H$ is either a symmetric subgroup or intermediate between a Levi subgroup of $G$ and its derived subgroup; for these cases we refer to [AvPe2, § 5].

We finish this subsection with the following lemma, which allows us to shorten computations in many cases.

Lemma 7.2**.**

Suppose that $\operatorname{SO}(V_{i})\subset H_{0}$ for some $i\in\{1,\ldots,d\}$. Then the following assertions hold.

1. (a)

If $d$ is even then $\operatorname{SOGr}_{\max}^{+}(V)$ is $H$-spherical if and only if so is $\operatorname{SOGr}_{\max}^{-}(V)$. 2. (b)

If $H_{j}\subset H_{0}$ for some $j\neq i$ and the pair $(H_{j},V_{j})$ is BF-equivalent to one of $(\operatorname{Spin}_{7},\mathbb{F}^{8})$ or $(\operatorname{Sp}_{4}\times\operatorname{SL}_{2}$, $\mathbb{F}^{4}\otimes\nobreak\mathbb{F}^{2})$ then for every subset $I\subset S$ the condition of $X_{I}$ being $H$-spherical holds or does not hold simultaneously for both choices of the conjugacy class of $H_{j}$ in $\operatorname{SO}(V_{j})$.

Proof.

For every element $g\in\mathrm{O}(V)$, let $\sigma\colon x\mapsto gxg^{-1}$ be the corresponding automorphism of the group $\operatorname{SO}(V)$. Then, as described in Reduction 2 (see § 4.1), $X_{I}$ is $H$-spherical if and only if $X_{\sigma(I)}$ is $\sigma(H)$-spherical.

(a) If $d=2n$ and $g\in\mathrm{O}(V_{i})\setminus\operatorname{SO}(V_{i})$ then $\sigma(H)=H$, $\operatorname{SOGr}_{\max}^{+}(V)=X_{\{n\}}$, and $\operatorname{SOGr}_{\max}^{-}(V)=X_{\{n-1\}}=X_{\sigma(\{n\})}$.

(b) If $g=g_{1}g_{2}$ with $g_{1}\in\mathrm{O}(V_{i})\setminus\operatorname{SO}(V_{i})$ and $g_{2}\in\mathrm{O}(V_{j})\setminus\operatorname{SO}(V_{j})$ then $g\in\operatorname{SO}(V)$ and hence $\sigma(H)$ is conjugate to $H$ and $\sigma(I)=I$ for every subset $I\subset S$. On the other hand, $\sigma(H)$ differs from $H$ by changing the conjugacy class of $H_{j}$ in $\operatorname{SO}(V_{j})$. ∎

7.2. Spherical actions on $\operatorname{SOGr}_{1}(V)$

Recall from § 5.3 that $\operatorname{SOGr}_{1}(V)\simeq X_{I}$ with $I=\{1\}$. We start with the following auxiliary lemma in which $V$ is not necessarily a simple $H$-module.

Lemma 7.3**.**

Suppose that $v\in V$ is a highest weight vector with respect to the $H$-module structure, $v$ is isotropic in $V$, $Q$ is the stabilizer in $H$ of the point $\langle v\rangle\in\mathbb{P}(V)$, and $M$ is a Levi subgroup of $Q$. Suppose also that $\operatorname{SOGr}_{1}(V)$ is $H$-spherical. Then the $M$-module $T_{\langle v\rangle}\mathbb{P}(V)/T_{\langle v\rangle}(H\langle v\rangle)$ contains a spherical submodule of codimension $1$. Moreover, there are $M$-module isomorphisms

[TABLE]

where $\mathfrak{q}^{u}$ is the nilpotent radical of $\mathfrak{q}$.

Proof.

The hypotheses imply that $\langle v\rangle\in\operatorname{SOGr}_{1}(V)$ and $Q$ is a parabolic subgroup of $H$, hence $H\langle v\rangle\simeq H/Q$ is a closed $H$-orbit in $\operatorname{SOGr}_{1}(V)$. Then it follows from Proposition 5.16 that $T_{\langle v\rangle}\operatorname{SOGr}_{1}(V)/T_{\langle v\rangle}(H\langle v\rangle)$ is a spherical $M$-module. Clearly, the latter module has codimension $1$ in $T_{\langle v\rangle}\mathbb{P}(V)/T_{\langle v\rangle}(H\langle v\rangle)$. The isomorphism in (7.1) follows from the $P$-module isomorphisms $T_{\langle v\rangle}\mathbb{P}(V)\simeq\mathfrak{gl}(V)/\mathfrak{p}\simeq(V/\langle v\rangle)\otimes\langle v\rangle^{*}$ where $P$ is the stabilizer of $\langle v\rangle$ in $\operatorname{GL}(V)$. The isomorphism in (7.2) is obvious. ∎

Remark 7.4*.*

If $V$ is a nontrivial simple $H$-module then every highest weight vector in $V$ is automatically isotropic.

Proposition 7.5**.**

Suppose that $V$ is a simple $H$-module. Then $\operatorname{SOGr}_{1}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to one of $(\operatorname{SO}_{n},\mathbb{F}^{n})$ $(n\geq 3)$, $(\operatorname{Sp}_{2n}\times\operatorname{SL}_{2}$, $\mathbb{F}^{2n}\otimes\mathbb{F}^{2})$ $(n\geq 2)$, $(\mathsf{G}_{2},\mathbb{F}^{7})$, $(\operatorname{Spin}_{7},\mathbb{F}^{8})$, $(\operatorname{Spin}_{9},\mathbb{F}^{16})$.

Proof.

If $H$ acts spherically on $\operatorname{SOGr}_{1}(V)$ then $H$ has an open orbit in $\operatorname{SOGr}_{1}(V)$. According to [Kim, Theorem 2.1], all pairs $(H,V)$ (up to BF-equivalence) for which $H$ has an open orbit in $\operatorname{SOGr}_{1}(V)$ are listed in Table 10.

For each pair $(H,V)$ listed in the statement of the theorem, $V$ is a spherical $(H\times\mathbb{F}^{\times})$-module, hence $H$ acts spherically on $\mathbb{P}(V)$, hence on $\operatorname{SOGr}_{1}(V)$ by Theorem 3.2. As $\dim B_{H}\geq\dim\operatorname{SOGr}_{1}(V)$ is a necessary sphericity condition, a case-by-case check of the remaining entries of Table 10 leaves us with the following three cases, which are treated using Lemma 7.3 and Remark 7.4. In all the cases, $v$ denotes a highest weight vector of $V$ (as an $H$-module) and $M$ is a Levi subgroup of the stabilizer in $H$ of the line $\langle v\rangle$.

Case 1: $H=\operatorname{Sp}_{6}$, $V=\wedge^{2}_{0}\mathbb{F}^{6}$. The pairs $(M,V/\langle v\rangle)$, $(M,\langle v\rangle)$, and $(M,T_{\langle v\rangle}(H\langle v\rangle))$ are equivalent to

[TABLE]

$(\operatorname{SL}_{2}\times\operatorname{SL}_{2}\times\mathbb{F}^{\times},\mathbb{F}^{1}_{-2\chi})$, and

[TABLE]

respectively. Then the pair $(M,T_{\langle v\rangle}\mathbb{P}(V)/T_{\langle v\rangle}(H\langle v\rangle))$ is equivalent to

[TABLE]

The latter module contains no spherical submodules of codimension $1$, hence $\operatorname{SOGr}_{1}(V)$ is not $H$-spherical.

Case 2: $H=\mathsf{F}_{4}$, $V=R(\pi_{4})$. The pairs $(M,V/\langle v\rangle)$, $(M,\langle v\rangle)$, and $(M,T_{\langle v\rangle}(H\langle v\rangle))$ are equivalent to

[TABLE]

$(\operatorname{Spin}_{7}\times\mathbb{F}^{\times},\mathbb{F}^{1}_{-2\chi})$, and

[TABLE]

respectively (the first pair was computed with LiE using the information in [LiE2, § 5.12]). Then the pair $(M,T_{\langle v\rangle}\mathbb{P}(V)/T_{\langle v\rangle}(H\langle v\rangle))$ is equivalent to

[TABLE]

The latter module contains no spherical submodules of codimension $1$, hence $\operatorname{SOGr}_{1}(V)$ is not $H$-spherical.

Case 3: $H=\operatorname{Sp}_{2n}\times\operatorname{Sp}_{4}$, $V=\mathbb{F}^{2n}\otimes\mathbb{F}^{4}$, $n\geq 6$. The pairs $(M,V/\langle v\rangle)$, $(M,\langle v\rangle)$, and $(M,T_{\langle v\rangle}(H\langle v\rangle))$ are equivalent to

[TABLE]

$(\operatorname{Sp}_{2n-2}\times\operatorname{SL}_{2}\times\mathbb{F}^{\times}\times\mathbb{F}^{\times},\mathbb{F}^{1}_{-\chi_{1}-\chi_{2}})$, and

[TABLE]

respectively. Then the pair $(M,T_{\langle v\rangle}\mathbb{P}(V)/T_{\langle v\rangle}(H\langle v\rangle))$ is equivalent to

[TABLE]

The latter module contains no spherical submodules of codimension $1$, hence $\operatorname{SOGr}_{1}(V)$ is not $H$-spherical. ∎

Proposition 7.6**.**

Suppose that $V$ is BF-equivalent to $\Omega(W)$ for a simple $H$-module $W$. Then $\operatorname{SOGr}_{1}(V)$ is $H$-spherical if and only if the pair $(H,W)$ is equivalent to one of $(\operatorname{GL}_{n},\mathbb{F}^{n})$ $(n\geq 2)$, $(\operatorname{SL}_{n},\mathbb{F}^{n})$ $(n\geq 2)$, $(\operatorname{Sp}_{2n}\times\mathbb{F}^{\times},\mathbb{F}^{2n})$ $(n\geq 2)$.

Proof.

If $H$ acts spherically on $\operatorname{SOGr}_{1}(V)$ then $H$ has an open orbit in $\operatorname{SOGr}_{1}(V)$. According to [Kim, Theorem 2.2], all pairs $(H,W)$ (up to equivalence) for which $H$ has an open orbit in $\operatorname{SOGr}_{1}(\Omega(W))$ are listed in Table 11.

For each pair $(H,W)$ mentioned in the statement, $V$ is a spherical $(H\times\mathbb{F}^{\times})$-module (where $\mathbb{F}^{\times}$ acts on $V$ by scalar transformations), hence $H$ acts spherically on $\mathbb{P}(V)$ and hence on $\operatorname{SOGr}_{1}(V)$ by Theorem 3.2. Below we consider all the remaining cases in Table 11 not satisfying the necessary sphericity condition $\dim B_{H}\geq\dim\operatorname{SOGr}_{1}(V)$.

Case 1: $H=\operatorname{Sp}_{2n},W=\mathbb{F}^{2n}$, $n\geq 2$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

Case 2: $H=\operatorname{SO}_{n}\times\mathbb{F}^{\times},W=\mathbb{F}^{n}$, $n\geq 3$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

Case 3: $H=\operatorname{GL}_{n}\times\operatorname{SL}_{2},W=\mathbb{F}^{n}\otimes\mathbb{F}^{2}$, $n\geq 3$. Let $v\in V$ be a highest weight vector for the $H$-submodule $W\subset V$ and let $M$ be the Levi subgroup of the stabilizer of the line $\langle v\rangle$. Computations using (7.1) show that the pair $(M,T_{\langle v\rangle}\mathbb{P}(V))$ is equivalent to

[TABLE]

Using (7.2) we find that the pair $(M,T_{\langle v\rangle}(H\langle v\rangle)$ is equivalent to

[TABLE]

hence the pair $(M,T_{\langle v\rangle}\mathbb{P}(V))/T_{\langle v\rangle}(H\langle v\rangle)$ is equivalent to

[TABLE]

Since the latter module does not contain spherical submodules of codimension $1$, the variety $\operatorname{SOGr}_{1}(V)$ is not $H$-spherical by Lemma 7.3.

Case 4: $H=\operatorname{SL}_{n}\times\operatorname{SL}_{2},W=\mathbb{F}^{n}\otimes\mathbb{F}^{2}$, $n\geq 3$. This is a subgroup of the group in the previous case, which does not act spherically on $\operatorname{SOGr}_{1}(V)$. ∎

Proposition 7.7**.**

Suppose that $V=\widetilde{V}_{1}\oplus\widetilde{V}_{2}$ for two pairwise orthogonal nonzero $H$-submodules $\widetilde{V}_{1},\widetilde{V}_{2}\subset V$. Then $\operatorname{SOGr}_{1}(V)$ is $H$-spherical if and only if $V$ is a spherical module with respect to the action of $H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times}$, where the first (resp. second) factor $\mathbb{F}^{\times}$ acts on $\widetilde{V}_{1}$ (resp. $\widetilde{V}_{2}$) by scalar transformations.

Proof.

Put

[TABLE]

Then $U$ is an open subset of $\operatorname{SOGr}_{1}(V)$. Define a map $\varphi\colon U\to\mathbb{P}(\widetilde{V}_{1})\times\mathbb{P}(\widetilde{V}_{2})$ by $\langle v_{1}+v_{2}\rangle\mapsto(\langle v_{1}\rangle,\langle v_{2}\rangle)$. It is easy to see that this map is $H$-equivariant, its image is

[TABLE]

(which is open in $\mathbb{P}(\widetilde{V}_{1})\times\mathbb{P}(\widetilde{V}_{2})$), and $\varphi$ is a two-fold covering over the image. It follows that $\operatorname{SOGr}_{1}(V)$ is $H$-spherical if and only if $\mathbb{P}(\widetilde{V}_{1})\times\mathbb{P}(\widetilde{V}_{2})$ is $H$-spherical. The latter is equivalent to the fact that $V$ is a spherical $(H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times})$-module. ∎

Corollary 7.8**.**

If $r\geq 3$ then $\operatorname{SOGr}_{1}(V)$ is not $H$-spherical.

Proof.

If $H$ acts spherically on $V$ then so does the group $\operatorname{SO}(V_{1})\times\ldots\times\operatorname{SO}(V_{r})$. By Proposition 7.7, in this situation $V_{2}\oplus\ldots\oplus V_{r}$ should be a spherical module with respect to the action of $\operatorname{SO}(V_{2})\times\ldots\times\operatorname{SO}(V_{r})\times\mathbb{F}^{\times}$ with $\mathbb{F}^{\times}$ acting by scalar transformations. However the latter module is not spherical. ∎

Proposition 7.9**.**

Suppose that $r\geq 2$. Then $\operatorname{SOGr}_{1}(V)$ is $H$-spherical if and only if $r=2$ and one of the following two conditions holds:

1. (1)

$H_{0}=H_{1}\times H_{2}$* and for $i=1,2$ the pair $(H_{i},V_{i})$ is BF-equivalent to a pair in Table 4;* 2. (2)

the pair $(H,V)$ is BF-equivalent to a pair in Table 7.

Proof.

Corollary 7.8 implies $r=2$. In this case, according to Proposition 7.7, $\operatorname{SOGr}_{1}(V)$ is $H$-spherical if and only if $V_{1}\oplus V_{2}$ is a spherical $(H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times})$-module. In particular, for each $i=1,2$ the summand $V_{i}$ is a spherical $(H_{i}\times\mathbb{F}^{\times})$-module, hence by Proposition 3.4 the pair $(H_{i},V_{i})$ is equivalent to one of those in Table 4. If $H_{0}=H_{1}\times H_{2}$ then we get (1). If $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$ then there is a connected normal subgroup $K\subset H_{0}$ that acts nontrivially on both $V_{1}$ and $V_{2}$. If $K$ contains a simple factor then the saturation of the $H$-module $V$ is indecomposable, hence it should be contained in the list of [Kno, § 5]. The latter implies that both $V_{1},V_{2}$ are simple $H$-modules and an easy case-by-case check of the above-cited list yields the first two cases of Table 7. If $K$ contains no simple factors then, inspecting Table 4, we find that $K\simeq\mathbb{F}^{\times}$ and for $i=1,2$ the pair $(H_{i},V_{i})$ is BF-equivalent to either $(\operatorname{SL}_{n_{i}}\times\mathbb{F}^{\times},\Omega([\mathbb{F}^{n_{i}}]_{\chi}))$ ($n_{i}\geq 1$) or $(\operatorname{Sp}_{2n_{i}}\times\mathbb{F}^{\times},\Omega([\mathbb{F}^{2n_{i}}]_{\chi}))$ ($n_{i}\geq 2$). Now an application of [AvPe1, Theorem 5.3] yields the last two cases in Table 7, whence (2). ∎

7.3. Spherical actions on $\operatorname{SOGr}_{\max}^{(\pm)}(V)$ for $r=1$

Throughout §§ 7.3–7.5 we keep in mind the following identifications (see § 5.3):

• •

if $d=2k+1$ then $\operatorname{SOGr}_{\max}(V)\simeq X_{I}$ with $I=\{k\}$;

• •

if $d=2k$ then $\operatorname{SOGr}^{+}_{\max}(V)\simeq X_{I}$ with $I=\{k\}$ and $\operatorname{SOGr}^{-}_{\max}(V)\simeq X_{I}$ with $I=\{k-1\}$.

Proposition 7.10**.**

Suppose that $V$ is a simple $H$-module and $d$ is odd. Then $\operatorname{SOGr}_{\max}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to one of $(\operatorname{SO}_{n},\mathbb{F}^{n})$ $(n\geq 3)$ or $(\mathsf{G}_{2},\mathbb{F}^{7})$.

Proof.

As $d$ is odd, $\operatorname{SOGr}_{\max}(V)$ being $H$-spherical implies that $\operatorname{SOGr}_{1}(V)$ is $H$-spherical by Proposition 5.12 and Theorem 5.3. Then Proposition 7.5 leaves us with the following two cases.

Case 1: $(\operatorname{SO}_{n},\mathbb{F}^{n})$, $n\geq 3$. In this case $\operatorname{SOGr}_{\max}(V)$ is $H$-spherical.

Case 2: $(\mathsf{G}_{2},\mathbb{F}^{7})$. In this case the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\mathbb{F}^{\times},\mathbb{F}^{1}_{\chi})$, the latter module being spherical. ∎

Proposition 7.11**.**

Suppose that $V$ is a simple $H$-module, $d$ is even, and $*\in\{+,-\}$. Then $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to one of $(\operatorname{SO}_{2n},\mathbb{F}^{2n})$ $(n\geq 2)$, $(\operatorname{Sp}_{4}\times\operatorname{SL}_{2},\mathbb{F}^{4}\otimes\mathbb{F}^{2})$, $(\operatorname{Spin}_{7},\mathbb{F}^{8})$.

Proof.

If $H$ acts spherically on $\operatorname{SOGr}_{\max}^{*}(V)$ then $H$ has an open orbit in $\operatorname{SOGr}_{\max}^{*}(V)$. According to [Kim, Theorem 2.1], all pairs $(H,V)$ (up to BF-equivalence) for which $H$ has an open orbit in at least one of the two varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are listed in Table 12.

Taking into account the necessary sphericity condition $\dim B_{H}\geq\dim\operatorname{SOGr}_{\max}^{\pm}(V)$, we are left with the following cases.

Case 1: $(\operatorname{SO}_{2n},\mathbb{F}^{2n})$, $n\geq 2$. In this case both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are $H$-spherical.

Case 2: $(\operatorname{Sp}_{4}\times\operatorname{SL}_{2},\mathbb{F}^{4}\otimes\mathbb{F}^{2})$. Up to isomorphism, it suffices to assume that $H_{0}\subset\operatorname{SO}(V)$ is the default embedding (see § 7.1). For the action on $\operatorname{SOGr}_{\max}^{+}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{SO}_{5}\times\mathbb{F}^{\times},[\mathbb{F}^{5}]_{\chi})$, the latter module being spherical. For the action on $\operatorname{SOGr}_{\max}^{-}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{GL}_{2},\mathbb{F}^{2})$, the latter module being spherical.

Case 3: $(\operatorname{Spin}_{7},\mathbb{F}^{8})$. In this case both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are $H$-spherical. ∎

Proposition 7.12**.**

Suppose that $V$ is BF-equivalent to $\Omega(W)$ for a simple $H$-module $W$ and $*\in\{+,-\}$. Then $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical if and only if the pair $(H,W)$, considered up to equivalence, and $*$ appear in Table 13.

Proof.

If $H$ acts spherically on $\operatorname{SOGr}_{\max}^{*}(V)$ then $H$ has an open orbit in $\operatorname{SOGr}_{\max}^{*}(V)$. According to [Kim, Theorem 2.2], $H$ has an open orbit in at least one of the varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ if and only if the pair $(H,W)$ is equivalent to one of $(\operatorname{GL}_{n},\mathbb{F}^{n})$ ($n\geq 2$), $(\operatorname{SL}_{n},\mathbb{F}^{n})$ ($n\geq 2$), $(\operatorname{SO}_{3}\times\mathbb{F}^{\times},[\mathbb{F}^{3}]_{\chi})$, or $(\operatorname{Sp}_{4}\times\mathbb{F}^{\times},[\mathbb{F}^{4}]_{\chi})$. Now it suffices to compute the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ for each of these pairs $(H,W)$ and each $*$ and conclude whether the resulting module is spherical or not. The results for each of the cases are summarized in Table 14. ∎

7.4. Spherical actions on $\operatorname{SOGr}_{\max}^{(\pm)}(V)$ for $r=2$

We begin with an auxiliary result.

Proposition 7.13**.**

Suppose that $V=\widetilde{V}_{1}\oplus\widetilde{V}_{2}$ for two pairwise orthogonal nonzero $H$-submodules $\widetilde{V}_{1},\widetilde{V}_{2}\subset V$, $\dim\widetilde{V}_{1}\geq 3$, and

• •

$H$* acts spherically on $\operatorname{SOGr}_{\max}(V)$ if $d$ is odd;*

• •

$H$* acts spherically on at least one of $\operatorname{SOGr}_{\max}^{\pm}(V)$ if $d$ is even.*

Then

1. (a)

if $\dim\widetilde{V}_{1}$ is odd then $\operatorname{SOGr}_{\max}(\widetilde{V}_{1})$ is $H$-spherical; 2. (b)

if $\dim\widetilde{V}_{1}$ is even then both varieties $\operatorname{SOGr}_{\max}^{\pm}(\widetilde{V}_{1})$ are $H$-spherical.

Proof.

(a) Case 1: $\dim\widetilde{V}_{2}$ is even. Choose maximal isotropic subspaces $U_{1}\subset\widetilde{V}_{1}$, $U_{2}\subset\widetilde{V}_{2}$ and put $U=U_{1}\oplus U_{2}$. Then $U\in\operatorname{SOGr}_{\max}(V)$. Let $Q$ be the stabilizer of $U$ in the group $\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2})$. It is easy to see that $Q=Q_{1}\times Q_{2}$ where $Q_{i}$ is the stabilizer of $U_{i}$ in $\operatorname{SO}(\widetilde{V}_{i})$ for $i=1,2$. It follows that the $(\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2}))$-orbit $O$ of $U$ in $\operatorname{SOGr}_{\max}(V)$ is closed and isomorphic to $\operatorname{SOGr}_{\max}(\widetilde{V}_{1})\times\operatorname{SOGr}_{\max}^{*}(\widetilde{V}_{2})$ for some choice of $*\in\{+,-\}$. Theorem 3.2 implies that $O$ is $H$-spherical, hence $\operatorname{SOGr}_{\max}(\widetilde{V}_{1})$ is also $H$-spherical.

Case 2: $\dim\widetilde{V}_{2}$ is odd. Choose $X\in\{\operatorname{SOGr}_{\max}^{+}(V),\operatorname{SOGr}_{\max}^{-}(V)\}$ such that $X$ is $H$-spherical. Choose maximal isotropic subspaces $U_{1}\subset\widetilde{V}_{1},U_{2}\subset\widetilde{V}_{2}$. For $i=1,2$ let $U_{i}^{\perp}$ be the orthogonal complement of $U_{i}$ in $\widetilde{V}_{i}$ and choose $u_{i}\in U_{i}^{\perp}\setminus U_{i}$ in such a way that the vector $u=u_{1}+u_{2}$ is isotropic. (Note that for $i=1,2$ the vector $u_{i}$ is nonisotropic and $U_{i}^{\perp}=U_{i}\oplus\langle u_{i}\rangle$.) Then $U=U_{1}\oplus\langle u\rangle\oplus U_{2}$ is a maximal isotropic subspace in $V$. Replacing $u_{2}$ with $-u_{2}$ if necessary we may assume that $U\in X$. Let $Q$ be the stabilizer of $U$ in the group $\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2})$. It is easy to see that for each $i=1,2$ the group $Q$ stabilizes the subspace $U_{i}^{\perp}$ and hence the subspace $U_{i}$, which implies that $Q=Q_{1}\times Q_{2}$ where $Q_{i}$ is the stabilizer of $U_{i}$ in $\operatorname{SO}(\widetilde{V}_{i})$ for $i=1,2$. It follows that the $(\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2}))$-orbit $O$ of $U$ in $\operatorname{SOGr}_{\max}(V)$ is closed and isomorphic to $\operatorname{SOGr}_{\max}(\widetilde{V}_{1})\times\operatorname{SOGr}_{\max}(\widetilde{V}_{2})$. The rest of the argument is as in Case 1.

(b) Case 1: $\dim\widetilde{V}_{2}$ is odd. Fix $*\in\{+,-\}$ and choose maximal isotropic subspaces $U_{1}\subset\widetilde{V}_{1},U_{2}\subset\widetilde{V}_{2}$ such that $U_{1}\in\operatorname{SOGr}_{\max}^{*}(\widetilde{V}_{1})$. Then $U\in\operatorname{SOGr}_{\max}(V)$. Now an argument similar to that in Case 1 of part (a) shows that $\operatorname{SOGr}_{\max}^{*}(\widetilde{V}_{1})$ is $H$-spherical.

Case 2: $\dim\widetilde{V}_{2}$ is even. Choose $X\in\{\operatorname{SOGr}_{\max}^{+}(V),\operatorname{SOGr}_{\max}^{-}(V)\}$ such that $X$ is $H$-spherical and fix $*\in\{+,-\}$. Choose maximal isotropic subspaces $U_{1}\subset\widetilde{V}_{1},U_{2}\subset\widetilde{V}_{2}$ and put $U=U_{1}\oplus U_{2}$. We may assume $U_{1}\in\operatorname{SOGr}_{\max}^{*}(\widetilde{V}_{1})$. Acting on $U_{2}$ by an element of $\mathrm{O}(\widetilde{V}_{2})\setminus\operatorname{SO}(\widetilde{V}_{2})$ if necessary we may also assume that $U\in X$. Now an argument similar to that in Case 1 of part (a) shows that $\operatorname{SOGr}_{\max}^{*}(\widetilde{V}_{1})$ is $H$-spherical. ∎

Proposition 7.14**.**

Suppose that $r=2$ and $d$ is odd. Then $\operatorname{SOGr}_{\max}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to one of $(\operatorname{SO}_{2l}\times\operatorname{SO}_{2m+1}$, $\mathbb{F}^{2l}\oplus\nobreak\mathbb{F}^{2m+1})$ $(l\geq 1,m\geq 0)$, $(\operatorname{Spin}_{7},\mathbb{F}^{8}\oplus\mathbb{F}^{1})$, $(\operatorname{GL}_{n},\Omega(\mathbb{F}^{n})\oplus\mathbb{F}^{1})$ $(n\geq 2)$.

Proof.

As $d$ is odd, $\operatorname{SOGr}_{\max}(V)$ being $H$-spherical implies that $\operatorname{SOGr}_{1}(V)$ is $H$-spherical by Proposition 5.12 and Theorem 5.3.

Without loss of generality we may assume that $\dim V_{1}=2l$ is even and $\dim V_{2}=2m+1$ is odd. Then $V_{2}$ is a simple $H$-module, in which case the pair $(H_{2},V_{2})$ can be BF-equivalent to one of the two pairs $(\operatorname{SO}_{2m+1},\mathbb{F}^{2m+1})$ or $(\mathsf{G}_{2},\mathbb{F}^{7})$ by Proposition 7.9. We first show that the group $\operatorname{SO}_{2l}\times\mathsf{G}_{2}$ does not act spherically on $\operatorname{SOGr}_{\max}(\mathbb{F}^{2l}\oplus\mathbb{F}^{7})$. Indeed, in this case the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

We have proved that a necessary $H$-sphericity condition for $\operatorname{SOGr}_{\max}(V)$ is that the pair $(H_{2},V_{2})$ is BF-equivalent to $(\operatorname{SO}_{2m+1},\mathbb{F}^{2m+1})$. Then according to Proposition 7.9 the pair $(H_{1},V_{1})$ is BF-equivalent to one of those in Table 4. Below we consider all these possibilities for the pair $(H_{1},V_{1})$ up to BF-equivalence.

Case 1: $(\operatorname{SO}_{2l},\mathbb{F}^{2l})$, $l\geq 1$.

If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

If the pair $(H,V)$ is BF-equivalent to $(\operatorname{SL}_{2}\times\operatorname{SL}_{2},\mathop{\mathbb{F}^{2}}\limits_{1}{\!}{\otimes}\mathop{\mathbb{F}^{2}}\limits_{2}{\!}\oplus\mathop{\mathrm{S}^{2}\mathbb{F}^{2}}\limits_{1}{\!})$ then

[TABLE]

hence $\operatorname{SOGr}_{\max}(V)$ is not $H$-spherical.

Case 2: $(\operatorname{Sp}_{2n}\times\operatorname{SL}_{2},\mathbb{F}^{2n}\otimes\mathbb{F}^{2})$, $n\geq 2$. By Proposition 7.13(b), a necessary $H$-sphericity condition for $\operatorname{SOGr}_{\max}(V)$ is that $\operatorname{Sp}_{2n}\times\operatorname{SL}_{2}$ acts spherically on both varieties $\operatorname{SOGr}_{\max}^{\pm}(V_{1})$, which implies $n=2$ by Proposition 7.11. In this case, by Lemma 7.2(b) it suffices to use only the default embedding $H_{1}\subset\operatorname{SO}(V_{1})$ as described in § 7.1.

If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical. It also follows from the above that $\operatorname{SOGr}_{\max}(V)$ is not $H$-spherical if $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

Case 3: $(\operatorname{Spin}_{7},\mathbb{F}^{8})$. By Lemma 7.2(b), it suffices to do the computations only for $\operatorname{Spin}^{+}_{7}$. If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $m=0$. It also follows from the above that $\operatorname{SOGr}_{\max}(V)$ is not $H$-spherical if $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

Case 4: $(\operatorname{Spin}_{9},\mathbb{F}^{16})$. By Proposition 7.13(b), a necessary sphericity condition is that $\operatorname{Spin}_{9}$ acts spherically on both varieties $\operatorname{SOGr}_{\max}^{\pm}(\mathbb{F}^{16})$, which is not the case by Proposition 7.11.

Case 5: $(\operatorname{GL}_{l},\Omega(\mathbb{F}^{l}))$, $l\geq 2$.

If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $m=0$. It also follows from the above that $\operatorname{SOGr}_{\max}(V)$ is not $H$-spherical if $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

Case 6: $(\operatorname{SL}_{l},\Omega(\mathbb{F}^{l}))$ ($l\geq 3$). A necessary $H$-sphericity condition is that $\operatorname{SOGr}_{\max}(V)$ is spherical for the group $\operatorname{GL}_{l}\times\operatorname{SO}_{2m+1}$, which implies $m=0$ by the previous case. Then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{SL}_{l},\mathop{\mathbb{F}^{l}}\limits_{1}{\!}\oplus\mathop{\wedge^{2}\mathbb{F}^{l}}\limits_{1}{\!})$, the latter module being not spherical. It also follows from the above that $\operatorname{SOGr}_{\max}(V)$ is not $H$-spherical if $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

Case 7: $(\operatorname{Sp}_{2n}\times\mathbb{F}^{\times},\Omega([\mathbb{F}^{2n}]_{\chi}))$, $n\geq 2$. By Proposition 7.13(b), a necessary $H$-sphericity condition for $\operatorname{SOGr}_{\max}(V)$ is that $\operatorname{Sp}_{2n}\times\mathbb{F}^{\times}$ acts spherically on both varieties $\operatorname{SOGr}_{\max}^{\pm}(V_{1})$, which is not the case by Proposition 7.12. ∎

Proposition 7.15**.**

Suppose that $r=2$, $d$ is even, and $*\in\{+,-\}$. Then $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical if and only if the pair $(H,V)$, considered up to BF-equivalence, and $*$ are listed in Table 15.

Proof.

Put $p=\dim V_{1}$, $q=\dim V_{2}$. The proof is divided into two parts depending on the parity of $p$ and $q$.

Part 1: $p,q$ are odd.

If $\min(p,q)=1$ then applying Propositions 7.1 and 7.10 yields that for each $*\in\{+,-\}$ the variety $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to one of $(\operatorname{SO}_{2m+1},\mathbb{F}^{2m+1}\oplus\mathbb{F}^{1})$ ($m\geq 2$) or $(\mathsf{G}_{2},\mathbb{F}^{7}\oplus\mathbb{F}^{1})$.

Now assume $\min(p,q)\geq 3$. By Proposition 7.13(a), a necessary $H$-sphericity condition for $\operatorname{SOGr}_{\max}^{\pm}(V)$ is that $H$ acts spherically on both varieties $\operatorname{SOGr}_{\max}(V_{1})$ and $\operatorname{SOGr}_{\max}(V_{2})$, hence by Proposition 7.10 each pair $(H_{1},V_{1})$ and $(H_{2},V_{2})$ is BF-equivalent to either $(\operatorname{SO}_{2m+1},\mathbb{F}^{2m+1})$ for some $m\geq 1$ or $(\mathsf{G}_{2},\mathbb{F}^{7})$.

If the pair $(H,V)$ is BF-equivalent to $(\operatorname{SO}_{2m+1}\times\operatorname{SO}_{2l+1},\mathop{\mathbb{F}^{2m+1}}\limits_{1}{\!}\oplus\mathop{\mathbb{F}^{2l+1}}\limits_{2}{\!})$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{GL}_{m}\times\operatorname{GL}_{l},\mathbb{F}^{m}\otimes\mathbb{F}^{l})$, the latter module being spherical.

If the pair $(H,V)$ is BF-equivalent to $(\operatorname{SO}_{2m+1},\mathbb{F}^{2m+1}\oplus\mathbb{F}^{2m+1})$ then

[TABLE]

whence $\operatorname{SOGr}_{\max}^{\pm}(V)$ is not spherical.

If $(H,V)$ is BF-equivalent to $(\mathsf{G}_{2}\times\operatorname{SO}_{2l+1},\mathop{\mathbb{F}^{7}}\limits_{1}{\!}\oplus\mathop{\mathbb{F}^{2l+1}}\limits_{2}{\!})$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $l\leq 1$. The latter also implies that none of the two varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ is $H$-spherical if both pairs $(H_{1},V_{1})$ and $(H_{2},V_{2})$ are BF-equivalent to $(\mathsf{G}_{2},\mathbb{F}^{7})$.

Part 2: $p,q$ are even.

By Proposition 7.13(b), a necessary $H$-sphericity condition for $\operatorname{SOGr}_{\max}^{\pm}(V)$ is that for any $i=1,2$ with $\dim V_{i}>2$ the group $H$ acts spherically on both varieties $\operatorname{SOGr}_{\max}^{\pm}(V_{i})$. Then Propositions 7.11 and 7.12 imply that for $i=1,2$ the pair $(H_{i},V_{i})$ can be BF-equivalent to one of $(\operatorname{SO}_{2m},\mathbb{F}^{2m})$ ($m\geq 1$), $(\operatorname{Sp}_{4}\times\operatorname{SL}_{2},\mathbb{F}^{4}\otimes\mathbb{F}^{2})$, $(\operatorname{Spin}_{7},\mathbb{F}^{8})$, $(\operatorname{GL}_{m},\Omega(\mathbb{F}^{m}))$ $(m\geq 2)$, $(\operatorname{SL}_{2m+1},\Omega(\mathbb{F}^{2m+1}))$ ($m\geq 1$), $(\operatorname{SO}_{3}\times\mathbb{F}^{\times},\Omega([\mathbb{F}^{3}]_{\chi}))$.

In Cases 7.4–7.4 below we assume that the pair $(H_{2},V_{2})$ is BF-equivalent to $(\operatorname{SO}_{2l},\mathbb{F}^{2l})$ with $l\geq 1$ and consider the various possibilities for the pair $(H_{1},V_{1})$ up to BF-equivalence. By Lemma 7.2(a), for the situation $H_{0}=H_{1}\times H_{2}$ in all these cases it suffices to check $H$-sphericity only for $\operatorname{SOGr}_{\max}^{+}(V)$.

Case 1: $(\operatorname{SO}_{2m},\mathbb{F}^{2m})$, $m\geq 1$.

If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{GL}_{m}\times\operatorname{GL}_{l},\mathbb{F}^{m}\otimes\mathbb{F}^{l}),$ the latter module being spherical.

If the pair $(H,V)$ is BF-equivalent to $(\operatorname{SO}_{2l},\mathbb{F}^{2l}\oplus\mathbb{F}^{2l})$, $l\geq 2$, then $\dim B_{H}=l^{2}<2l^{2}-l=\dim\operatorname{SOGr}_{\max}^{\pm}(V)$, hence both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical.

If the pair $(H,V)$ is BF-equivalent to $(\operatorname{SL}_{2}\times\operatorname{SL}_{2}\times\operatorname{SL}_{2},\mathop{\mathbb{F}^{2}}\limits_{1}{\!}\otimes\mathop{\mathbb{F}^{2}}\limits_{3}{\!}\oplus\mathop{\mathbb{F}^{2}}\limits_{2}{\!}\otimes\mathop{\mathbb{F}^{2}}\limits_{3}{\!})$ and $H_{0}\subset\operatorname{Spin}_{7}^{+}$ then:

• •

for $\operatorname{SOGr}_{\max}^{+}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical;

• •

for $\operatorname{SOGr}_{\max}^{-}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

Case 2: $(\operatorname{Sp}_{4}\times\operatorname{SL}_{2},\mathbb{F}^{4}\otimes\mathbb{F}^{2})$. By Lemma 7.2(b), it suffices to assume that $H_{1}\subset\operatorname{SO}(V_{1})$ is the default embedding, see § 7.1.

If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

It also follows that both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical if $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

Case 3: $(\operatorname{Spin}_{7},\mathbb{F}^{8})$. By Lemma 7.2(b), it suffices to do the computations only for $\operatorname{Spin}_{7}^{+}$.

The only possibility is $H_{0}=H_{1}\times H_{2}$, in which case the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

Case 4: $(\operatorname{GL}_{m},\Omega(\mathbb{F}^{m}))$, $m\geq 2$.

If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $m\leq 3$ or $l=1$.

If the pair $(H,V)$ is BF-equivalent to $(\operatorname{SL}_{m}\times\mathbb{F}^{\times},\mathop{\Omega([\mathbb{F}^{m}]_{a\chi})}\limits_{1}{\!}\oplus\Omega(\mathbb{F}^{1}_{b\chi}))$, $m\geq 2$, $a,b\in\mathbb{Z}\setminus\{0\}$, then:

• •

for $\operatorname{SOGr}_{\max}^{+}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if either $m$ is even or $m$ is odd and $b\neq-ma$;

• •

for $\operatorname{SOGr}_{\max}^{-}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if either $m$ is even or $m$ is odd and $b\neq ma$.

If the pair $(H,V)$ is BF-equivalent to $(\operatorname{SL}_{2}\times\operatorname{SL}_{2}\times\mathbb{F}^{\times},\mathop{\Omega([\mathbb{F}^{2}]_{\chi})}\limits_{1}{\!}\oplus\mathop{\vphantom{|}\mathbb{F}^{2}}\limits_{1}{\!}\otimes\mathop{\vphantom{|}\mathbb{F}^{2}}\limits_{2}{\!})$ then $\dim B_{H}=5<6=\dim\operatorname{SOGr}_{\max}^{\pm}(V)$, hence both $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical in this case.

If the pair $(H,V)$ is BF-equivalent to $(\operatorname{SL}_{4}\times\mathbb{F}^{\times},\mathop{\Omega([\mathbb{F}^{4}]_{\chi})}\limits_{1}{\!}\oplus\mathop{\vphantom{|}\wedge^{2}\mathbb{F}^{4}}\limits_{1}{\!})$ then $\dim B_{H}=10<21=\dim\operatorname{SOGr}_{\max}^{\pm}(V)$, hence both $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical in this case.

Case 5: $(\operatorname{SL}_{2m+1},\Omega(\mathbb{F}^{2m+1}))$, $m\geq 1$.

If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $l=1$. This shows in particular that both $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical when $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

Case 6: $(\operatorname{SO}_{3}\times\mathbb{F}^{\times},\Omega([\mathbb{F}^{3}]_{\chi}))$.

If $H_{0}=H_{1}\times H_{2}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

It also follows that both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical if $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

The results obtained in Cases 7.4–7.4 show that it now suffices to consider only the cases where for $i=1,2$ the pair $(H_{i},V_{i})$ is BF-equivalent to either $(\operatorname{Spin}_{7},\mathbb{F}^{8})$ or $(\operatorname{GL}_{m},\Omega(\mathbb{F}^{m}))$ ($m\geq 2$). Next we consider the remaining cases for the pairs $(H_{1},V_{1})$ and $(H_{2},V_{2})$ up to BF-equivalence.

Case 7: $(\operatorname{Spin}_{7},\mathbb{F}^{8})$, $(\operatorname{Spin}_{7},\mathbb{F}^{8})$.

If $H_{0}=H_{1}\times H_{2}$ then $\dim B_{H}=24<28=\dim\operatorname{SOGr}_{\max}^{\pm}(\mathbb{F}^{8}\oplus\mathbb{F}^{8})$, hence both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical. Clearly, the latter also holds if $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

Case 8: $(\operatorname{Spin}_{7},\mathbb{F}^{8})$, $(\operatorname{GL}_{m},\Omega(\mathbb{F}^{m}))$, $m\geq 2$.

Up to automorphism, it suffices to do the computations only for $\operatorname{Spin}_{7}^{+}$. If $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical for some $*\in\{+,-\}$ then $\operatorname{SOGr}_{\max}^{*}(V)$ would be spherical for $\operatorname{SO}_{8}\times\operatorname{GL}_{m}$, which implies $m\leq 3$ by the results in Case 7.4.

If $m=3$ and $H_{0}=H_{1}\times H_{2}$ then $\dim B_{H}=18<21=\dim\operatorname{SOGr}_{\max}^{\pm}(V)$, hence both $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical. Clearly, the latter also holds if $m=3$ and $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$.

If $m=2$ and $H_{0}=H_{1}\times H_{2}$ then:

• •

for $\operatorname{SOGr}_{\max}^{+}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical;

• •

for $\operatorname{SOGr}_{\max}^{-}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

Case 9: $(\operatorname{GL}_{m},\mathbb{F}^{m})$, $(\operatorname{GL}_{l},\mathbb{F}^{l})$, $m,l\geq 2$.

If $H_{0}=H_{1}\times H_{2}$ then:

• •

for $\operatorname{SOGr}_{\max}^{+}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical;

• •

for $\operatorname{SOGr}_{\max}^{-}(V)$ the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

It also follows that both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical if $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}$. ∎

7.5. Spherical actions on $\operatorname{SOGr}_{\max}^{(\pm)}(V)$ for $r\geq 3$

Proposition 7.16**.**

Suppose that $r\geq 3$ and $d$ is odd. Then $\operatorname{SOGr}_{\max}(V)$ is not $H$-spherical.

Proof.

As $d$ is odd, $\operatorname{SOGr}_{\max}(V)$ being $H$-spherical implies that $\operatorname{SOGr}_{1}(V)$ is $H$-spherical by Proposition 5.12 and Theorem 5.3. Then Corollary 7.8 yields $r\leq 2$. ∎

In what follows we assume that $d$ is even.

Proposition 7.17**.**

Suppose that $V=\widetilde{V}_{1}\oplus\widetilde{V}_{2}\oplus\widetilde{V}_{3}$ for three pairwise orthogonal nonzero $H$-submodules $\widetilde{V}_{1},\widetilde{V}_{2},\widetilde{V}_{3}\subset V$ of dimensions $n_{1},n_{2},n_{3}$, respectively. If $H$ acts spherically on $\operatorname{SOGr}_{\max}^{*}(V)$ for some $*\in\{+,-\}$ then one of the following possibilities holds:

1. (1)

$\min(n_{1},n_{2},n_{3})=1$; 2. (2)

at least two of $n_{1},n_{2},n_{3}$ equal $2$.

Proof.

It suffices to prove the assertion for the case $H_{0}=\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2})\times\operatorname{SO}(\widetilde{V}_{3})$. Then by Lemma 7.2(a) it suffices to consider the case $*=+$. Without loss of generality we may assume that $n_{1}$ is even.

Case 1: $n_{2},n_{3}$ are odd. We may assume $n_{2}\geq n_{3}\geq 1$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

If $n_{3}=1$ then the above module is spherical. If $n_{3}\geq 3$ then the submodule consisting of all summands except the first one is not spherical by Lemma 6.1, hence the whole module is not spherical.

Case 2: $n_{2},n_{3}$ are even. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

By Lemma 6.1, the latter module is spherical if and only if at least two of $n_{1},n_{2},n_{3}$ equal $2$. ∎

According to Proposition 7.17, the analysis of the case $r=3$ is completed by Propositions 7.18 and 7.19 below.

Proposition 7.18**.**

Suppose that $r=3$, $d_{1}\geq d_{2}\geq d_{3}=1$, and $*\in\{+,-\}$. Then $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to one of $(\operatorname{SO}_{2l}\times\operatorname{SO}_{2m+1},\mathbb{F}^{2l}\oplus\mathbb{F}^{2m+1}\oplus\mathbb{F}^{1})$ $(l\geq 1,m\geq 0)$, $(\operatorname{Spin}_{7},\mathbb{F}^{8}\oplus\mathbb{F}^{1}\oplus\mathbb{F}^{1})$, $(\operatorname{GL}_{n},\Omega(\mathbb{F}^{n})\oplus\nobreak\mathbb{F}^{1}\oplus\nobreak\mathbb{F}^{1})$ $(n\geq 2)$.

Proof.

By Proposition 7.1, $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical if and only if $\operatorname{SOGr}_{\max}(V_{1}\oplus V_{2})$ is $H$-spherical. Then the claim follows from Proposition 7.14. ∎

Proposition 7.19**.**

Suppose that $r=3$, $d_{2}=d_{3}=2$, and $*\in\{+,-\}$. Then $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to $(\operatorname{SO}_{2l}\times\mathbb{F}^{\times}\times\mathbb{F}^{\times}$, $\mathbb{F}^{2l}\oplus\Omega(\mathbb{F}^{1}_{\chi_{1}})\oplus\Omega(\mathbb{F}^{1}_{\chi_{2}}))$.

Proof.

It follows from the hypothesis that for $i=2,3$ the pair $(H_{i},V_{i})$ is BF-equivalent to $(\mathbb{F}^{\times},\Omega(\mathbb{F}_{\chi}))$. For $i=2,3$ let $V_{i}=V^{\prime}_{i}\oplus V^{\prime\prime}_{i}$ be a decomposition of $V_{i}$ into a direct sum of two $H$-stable isotropic lines. Let $\widetilde{H}$ be the subgroup of $\operatorname{SO}(V_{1})\times\operatorname{SO}(V_{2}\oplus V_{3})$ stabilizing both subspaces $V^{\prime}_{2}\oplus V^{\prime}_{3}$ and $V^{\prime\prime}_{2}\oplus V^{\prime\prime}_{3}$. Then the pair $(\widetilde{H},V)$ is BF-equivalent to $(\operatorname{SO}_{d_{1}}\times\operatorname{GL}_{2},\mathop{\mathbb{F}^{d_{1}}}\limits_{1}{\!}\oplus\Omega(\mathop{\mathbb{F}^{2}}\limits_{2}{\!}))$.

If $\operatorname{SOGr}_{\max}^{*}(V)$ is $H$-spherical then it is $\widetilde{H}$-spherical, hence by Proposition 7.15 the pair $(H_{1},V_{1})$ should be BF-equivalent to one of $(\operatorname{SO}_{2l},\mathbb{F}^{2l})$ ($l\geq 1$) or $(\operatorname{Spin}_{7},\mathbb{F}^{8})$. In what follows we treat these two cases separately.

Case 1: $(\operatorname{SO}_{2l},\mathbb{F}^{2l})$, $l\geq 1$.

If $H_{0}=H_{1}\times H_{2}\times H_{3}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

If $l=1$ and $H_{0}$ is a proper subgroup of $H_{1}\times H_{2}\times H_{3}$ then $\dim B_{H}\leq 2<3=\dim\operatorname{SOGr}_{\max}^{*}(V)$, hence $\operatorname{SOGr}_{\max}^{*}(V)$ is not $H$-spherical.

If $l\geq 2$ and the pair $(H,V)$ is equivalent to $(\operatorname{SO}_{2l}\times\mathbb{F}^{\times},\mathbb{F}^{2l}\oplus\Omega(\mathbb{F}^{1}_{a\chi})\oplus\Omega(\mathbb{F}^{1}_{b\chi})$ for some $a,b\in\mathbb{Z}\setminus\{0\}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is BF-equivalent to

[TABLE]

the latter module being not spherical.

Case 2: $(\operatorname{Spin}_{7},\mathbb{F}^{8})$. If $H_{0}=H_{1}\times H_{2}\times H_{3}$ then $\dim B_{H}=14<15=\dim\operatorname{SOGr}_{\max}^{*}(V)$, hence $\operatorname{SOGr}_{\max}^{*}(V)$ is not $H$-spherical. It also follows that $\operatorname{SOGr}_{\max}^{*}(V)$ is not $H$-spherical if $H$ is a proper subgroup of $H_{1}\times H_{2}\times H_{3}$. ∎

Proposition 7.20**.**

Suppose that $V=\widetilde{V}_{1}\oplus\widetilde{V}_{2}\oplus\widetilde{V}_{3}\oplus\widetilde{V}_{4}$ for four pairwise orthogonal nonzero $H$-submodules $\widetilde{V}_{1},\widetilde{V}_{2},\widetilde{V}_{3},\widetilde{V}_{4}\subset V$. Then both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical.

Proof.

It suffices to prove the assertion for $H=\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2})\times\operatorname{SO}(\widetilde{V}_{3})\times\operatorname{SO}(\widetilde{V}_{4})$, which is assumed in what follows. Then by Lemma 7.2(a) it suffices to consider $\operatorname{SOGr}_{\max}^{+}(V)$ only. Assume that $\operatorname{SOGr}_{\max}^{+}(V)$ is $H$-spherical and put $n_{i}=\dim\widetilde{V}_{i}$ for $i=1,2,3,4$.

If one of the numbers $n_{i}$ is odd, say $n_{4}$, then $H$ acts spherically on $\operatorname{SOGr}_{\max}(\widetilde{V}_{1}\oplus\widetilde{V}_{2}\oplus\widetilde{V}_{3})$ by Proposition 7.13(a), which is impossible by Proposition 7.16. Thus $n_{i}$ are even for all $i=1,2,3,4$.

Clearly, the group $\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2})\times\operatorname{SO}(\widetilde{V}_{3}\oplus\widetilde{V}_{4})$ also acts spherically on $\operatorname{SOGr}_{\max}^{+}(V)$. As $n_{3}+n_{4}\geq 4$, Proposition 7.17 yields $n_{1}=n_{2}=2$. Likewise, $n_{3}=n_{4}=2$. But then $\dim B_{H}=4<6=\dim\operatorname{SOGr}_{\max}^{+}(V)$, hence $\operatorname{SOGr}_{\max}^{+}(V)$ is not $H$-spherical. ∎

Corollary 7.21**.**

If $r\geq 4$ then both varieties $\operatorname{SOGr}_{\max}^{\pm}(V)$ are not $H$-spherical.

7.6. Spherical actions on $\operatorname{SOGr}_{2}(V)$

As $\operatorname{SOGr}_{2}(V)=\operatorname{SOGr}_{\max}^{+}(V)\cup\operatorname{SOGr}_{\max}^{-}(V)$ for $d=4$ and $\operatorname{SOGr}_{2}(V)=\operatorname{SOGr}_{\max}(V)$ for $d=5$, for classifying $H$-spherical actions on $\operatorname{SOGr}_{2}(V)$ it suffices to assume $d\geq 6$. (Although the result of Proposition 7.22 will be needed later for $d\geq 4$). Recall from § 5.3 that $\operatorname{SOGr}_{2}(V)\simeq X_{I}$ with $I=\{2\}$ for $d\geq 5$.

Proposition 7.22**.**

Suppose that $V$ is a simple $H$-module and $d\geq 4$. Then

1. (a)

if $d\geq 5$ then $\operatorname{SOGr}_{2}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to one of $(\operatorname{SO}_{d},\mathbb{F}^{d})$, $(\mathsf{G}_{2},\mathbb{F}^{7})$, $(\operatorname{Spin}_{7},\mathbb{F}^{8})$; 2. (b)

if $d=4$ and $*\in\{+,-\}$ then $\operatorname{SOGr}_{2}^{*}(V)$ is $H$-spherical if and only if the pair $(H,V)$ is BF-equivalent to $(\operatorname{SO}_{4},\mathbb{F}^{4})$.

Proof.

(a) If $H$ acts spherically on $\operatorname{SOGr}_{2}(V)$ then it acts spherically on $\operatorname{SOGr}_{1}(V)$ by Propositions 5.12 and 5.14 and Theorem 5.3, hence Proposition 7.5 leaves us with the following cases (up to BF-equivalence).

Case 1: $H=\operatorname{SO}_{n}$, $V=\mathbb{F}^{n}$, $n\geq 6$. In this case $\operatorname{SOGr}_{2}(V)$ is spherical.

Case 2: $H=\operatorname{Sp}_{2n}\times\operatorname{SL}_{2}$, $V=\mathbb{F}^{2n}\otimes\mathbb{F}^{2}$, $n\geq 2$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

Case 3: $H=\mathsf{G}_{2}$, $V=\mathbb{F}^{7}$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{GL}_{2},\mathbb{F}^{2})$, the latter module being spherical.

Case 4: $H=\operatorname{Spin}_{7}$, $V=\mathbb{F}^{8}$. In this case $\operatorname{SOGr}_{2}(V)$ is spherical.

Case 5: $H=\operatorname{Spin}_{9}$, $V=\mathbb{F}^{16}$. We have $\dim B_{H}=20<25=\dim\operatorname{SOGr}_{2}(V)$, hence $\operatorname{SOGr}_{2}(V)$ is not $H$-spherical.

(b) This follows directly from Proposition 7.11. ∎

Proposition 7.23**.**

Suppose that $V$ is BF-equivalent to $\Omega(W)$ for a simple $H$-module $W$ with $\dim W\geq 3$. Then $\operatorname{SOGr}_{2}(V)$ is $H$-spherical if and only if the pair $(H,W)$ is equivalent to one of $(\operatorname{GL}_{n},\mathbb{F}^{n})$ $(n\geq 3)$ or $(\operatorname{SL}_{n},\mathbb{F}^{n})$ $(n\geq 3,n\neq 4)$.

Proof.

If $\operatorname{SOGr}_{2}(V)$ is $H$-spherical then $\operatorname{SOGr}_{1}(V)$ is $H$-spherical by Proposition 5.14 and Theorem 5.3. Then Proposition 7.6 leaves us with the following cases (up to equivalence).

Case 1: $H=\operatorname{GL}_{n}$, $W=\mathbb{F}^{n}$, $n\geq 3$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

Case 2: $H=\operatorname{SL}_{n}$, $W=\mathbb{F}^{n}$, $n\geq 3$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical if and only if $n\neq 4$.

Case 3: $H=\operatorname{Sp}_{2n}\times\mathbb{F}^{\times}$, $W=[\mathbb{F}^{2n}]_{\chi}$, $n\geq 2$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical. ∎

To proceed with the case $r\geq 2$, we shall need the following lemma.

Lemma 7.24**.**

Suppose that $V=\widetilde{V}_{1}\oplus\widetilde{V}_{2}$ for two pairwise orthogonal nonzero $H$-submodules $\widetilde{V}_{1},\widetilde{V}_{2}\subset V$, $\dim\widetilde{V}_{1}\geq 4$, and $\operatorname{SOGr}_{2}(V)$ is $H$-spherical. Then

1. (a)

if $\dim\widetilde{V}_{1}\geq 5$ then $\operatorname{SOGr}_{2}(\widetilde{V}_{1})$ is $H$-spherical; 2. (b)

if $\dim\widetilde{V}_{1}=4$ then both varieties $\operatorname{SOGr}_{2}^{\pm}(\widetilde{V}_{1})$ are $H$-spherical.

Proof.

Put $Y=\operatorname{SOGr}_{2}(\widetilde{V}_{1})$ if $\dim\widetilde{V}_{1}\geq 5$ and choose $Y\in\{\operatorname{SOGr}_{2}^{+}(\widetilde{V}_{1}),\operatorname{SOGr}_{2}^{-}(\widetilde{V}_{1})\}$ if $\dim\widetilde{V}_{1}=4$. Choose a two-dimensional isotropic subspace $U\subset\widetilde{V}_{1}$ such that $U\in\nobreak Y$. Clearly, the $(\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2}))$-orbit of $U$ in $\operatorname{SOGr}_{2}(V)$ is closed and isomorphic to $Y$. Then $Y$ is $H$-spherical by Theorem 3.2. ∎

Proposition 7.25**.**

Suppose that $d\geq 5$, $V=\widetilde{V}_{1}\oplus\widetilde{V}_{2}$ for two pairwise orthogonal nonzero $H$-submodules $\widetilde{V}_{1},\widetilde{V}_{2}\subset V$ of dimensions $n_{1},n_{2}$, respectively, and $\operatorname{SOGr}_{2}(V)$ is $H$-spherical. Then $\min(n_{1},n_{2})\leq 2$.

Proof.

It suffices to prove that $\operatorname{SOGr}_{2}(V)$ is not $H$-spherical when $H_{0}=\operatorname{SO}(\widetilde{V}_{1})\times\operatorname{SO}(\widetilde{V}_{2})$ and $n_{1}\geq n_{2}\geq 3$, which is assumed in what follows.

Choose realizations $\widetilde{V}_{1}=\langle e_{1},e_{d}\rangle\oplus W_{1}$, $\widetilde{V}_{2}=\langle e_{2},e_{d-1}\rangle\oplus W_{2}$ where $W_{1},W_{2}$ are nondegenerate subspaces such that $W_{1}\oplus W_{2}=\langle e_{3},e_{4},\ldots,e_{d-2}\rangle$. With these realizations, the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical. ∎

Proposition 7.26**.**

Suppose that $r\geq 2$ and $d\geq 6$. Then $\operatorname{SOGr}_{2}(V)$ is $H$-spherical if and only if $r=2$ and the pair $(H,V)$ is BF-equivalent to one of $(\operatorname{SO}_{n},\mathbb{F}^{n}\oplus\mathbb{F}^{1})$ $(n\geq 5)$, $(\operatorname{SO}_{n}\times\mathbb{F}^{\times},\mathbb{F}^{n}\oplus\Omega(\mathbb{F}^{1}_{\chi}))$ $(n\geq 4)$, $(\operatorname{Spin}_{7},\mathbb{F}^{8}\oplus\mathbb{F}^{1})$ or $(\operatorname{Spin}_{7}\times\mathbb{F}^{\times},\mathbb{F}^{8}\oplus\Omega(\mathbb{F}^{1}_{\chi}))$.

Proof.

If $H$ acts spherically on $\operatorname{SOGr}_{2}(V)$ then it acts spherically on $\operatorname{SOGr}_{1}(V)$ by Propositions 5.12 and 5.14 and Theorem 5.3. Then Corollary 7.8 yields $r=2$. By Proposition 7.25, in what follows we may assume $d_{2}\leq 2$.

We now show that the $H$-module $V_{1}$ cannot be weakly reducible. To this end, it suffices to prove that $\operatorname{SOGr}_{2}(V)$ is not $H$-spherical if the pair $(H,V)$ is BF-equivalent to $(\operatorname{GL}_{n},\Omega(\mathbb{F}^{n})\oplus\mathbb{F}^{1})$ ($n\geq 3$) or $(\operatorname{GL}_{n}\times\mathbb{F}^{\times},\Omega(\mathbb{F}^{n})\oplus\Omega(\mathbb{F}^{1}_{\chi}))$ ($n\geq 2$).

Indeed, in the first case the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical, and in the second case the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

Thus it remains to analyze the situation where $V_{1}$ is a simple $H$-module. In this situation, Lemma 7.24 and Proposition 7.22 imply that the pair $(H_{1},V_{1})$ can be BF-equivalent to one of $(\operatorname{SO}_{d_{1}},\mathbb{F}^{d_{1}})$, $(\operatorname{Spin}_{7},\mathbb{F}^{8})$, or $(\mathsf{G}_{2},\mathbb{F}^{7})$. Below we consider all the corresponding cases for the pair $(H,V)$ up to BF-equivalence.

Case 1: $(\operatorname{SO}_{d_{1}},\mathbb{F}^{d_{1}}\oplus\mathbb{F}^{1})$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{GL}_{2},\mathbb{F}^{2})$, the latter module being spherical.

Case 2: $(\operatorname{SO}_{d_{1}}\times\mathbb{F}^{\times},\mathbb{F}^{d_{1}}\oplus\Omega(\mathbb{F}^{1}_{\chi}))$. Choose the realizations $V_{2}=\langle e_{1},e_{d}\rangle$ and $V_{1}=\langle e_{2},\ldots,e_{d-1}\rangle$. Then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

Case 3: $(\operatorname{Spin}_{7},\mathbb{F}^{8}\oplus\mathbb{F}^{1})$. By Lemma 7.2(b), it suffices to do the calculations for $\operatorname{Spin}_{7}^{+}$. Then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

Case 4: $(\operatorname{Spin}_{7}\times\mathbb{F}^{\times},\mathbb{F}^{8}\oplus\Omega(\mathbb{F}^{1}_{\chi}))$. By Lemma 7.2(b), it suffices to do the calculations for $\operatorname{Spin}_{7}^{+}$. Then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

Case 5: $(\mathsf{G}_{2},\mathbb{F}^{7}\oplus\mathbb{F}^{1})$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

Case 6: $(\mathsf{G}_{2}\times\mathbb{F}^{\times},\mathbb{F}^{7}\oplus\Omega(\mathbb{F}^{1}_{\chi}))$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical. ∎

7.7. Completion of the classification

Suppose that $d\geq 5$ and $X$ is a nontrivial flag variety of $G$ different from $\operatorname{SOGr}_{1}(V)$, $\operatorname{SOGr}_{2}(V)$, $\operatorname{SOGr}_{\max}(V)$ (for $d$ odd), and $\operatorname{SOGr}_{\max}^{\pm}(V)$ (for $d$ even). By Propositions 5.13, 5.15 and Theorem 5.3, $X$ being $H$-spherical implies that $\operatorname{SOGr}_{2}(V)$ is $H$-spherical, hence the pair $(H,V)$ is BF-equivalent to one of those listed in Propositions 7.22, 7.23, and 7.26. Excluding the cases where $H$ is either a symmetric subgroup of $G$ or intermediate between a Levi subgroup of $G$ and its derived subgroup (see § 4.2), we arrive at the pairs $(\mathsf{G}_{2},\mathbb{F}^{7})$, $(\operatorname{Spin}_{7},\mathbb{F}^{8}\oplus\mathbb{F}^{1})$, $(\operatorname{Spin}_{7}\times\mathbb{F}^{\times},\mathbb{F}^{8}\oplus\Omega(\mathbb{F}^{1}_{\chi}))$, which remain to be considered.

Proposition 7.27**.**

Suppose that the pair $(H,V)$ is BF-equivalent to $(\mathsf{G}_{2},\mathbb{F}^{7})$. Then $X_{I}$ is $H$-spherical if and only if $|I|\leq 2$.

Proof.

First suppose that $|I|=3$. Then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being not spherical.

Now suppose that $|I|=2$.

If $I=\{1,2\}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\mathbb{F}^{\times}\times\mathbb{F}^{\times},\mathbb{F}^{1}_{\chi_{1}}\oplus\mathbb{F}^{1}_{\chi_{1}+\chi_{2}})$, the latter module being spherical.

If $I=\{1,3\}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\operatorname{SL}_{2}\times\mathbb{F}^{\times},[\mathbb{F}^{2}]_{\chi}\oplus\mathbb{F}^{1}_{2\chi})$, the latter module being spherical.

If $I=\{2,3\}$ then the pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to $(\mathbb{F}^{\times}\times\mathbb{F}^{\times},\mathbb{F}^{1}_{\chi_{1}}\oplus\mathbb{F}^{1}_{\chi_{1}+\chi_{2}})$, the latter module being spherical. ∎

Proposition 7.28**.**

Suppose that the pair $(H,V)$ is BF-equivalent to $(\operatorname{Spin}_{7},\mathbb{F}^{8}\oplus\mathbb{F}^{1})$. Then $X_{I}$ is $H$-spherical if and only if either $|I|=1$ or $I=\{1,2\}$.

Proof.

By Lemma 7.2(b), it suffices to do the calculations for $\operatorname{Spin}_{7}^{+}$.

Case $I=\{3\}$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

Case $I=\{1,2\}$. The pair $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is equivalent to

[TABLE]

the latter module being spherical.

Case $I=\{1,4\}$ or $\{3,4\}$. As $\dim B_{H}=12<13=\dim X_{I}$, the variety $X_{I}$ is not spherical.

Case $I=\{1,3\}$ or $\{2,3\}$ or $\{2,4\}$. As $\dim B_{H}=12<14=\dim X_{I}$, the variety $X_{I}$ is not spherical. ∎

Proposition 7.29**.**

Suppose that $(H,V)$ is BF-equivalent to $(\operatorname{Spin}_{7}\times\mathbb{F}^{\times},\mathbb{F}^{8}\oplus\Omega(\mathbb{F}^{1}_{\chi}))$. Then $X_{I}$ is $H$-spherical if and only if $I$ equals one of $\{1\}$, $\{2\}$, $\{4\}$, or $\{5\}$.

Proof.

By Lemma 7.2(b), it suffices to do the calculations for $\operatorname{Spin}_{7}^{+}$.

Case $I=\{3\}$. As $\dim B_{H}=13<15=\dim X_{I}$, the variety $X_{I}$ is not $H$-spherical.

Case $I=\{1,2\}$ or $\{1,4\}$ or $\{1,5\}$ or $\{4,5\}$. As $\dim B_{H}=13<14=\dim X_{I}$, the variety $X_{I}$ is not $H$-spherical.

Case $I=\{2,5\}$. As $\dim B_{H}=13<16=\dim X_{I}$, the variety $X_{I}$ is not $H$-spherical. ∎

8. Remarks on computing the restricted branching monoids

In this section we explain how to compute the restricted branching monoids corresponding to all spherical actions on flag varieties appearing in our theorems in §§ 4.3, 4.4.

8.1. Case $I=\{1\}$

Proof of Theorem 4.5.

It is well known that for all $k\geq 0$ there is the $G$-module isomorphism $R_{G}(k\pi_{1})\simeq\mathrm{S}^{k}V$. Then it follows from definitions that the map

[TABLE]

is an isomorphism, which immediately implies the required result. ∎

Proof of Theorem 4.9.

We shall need the following well-known $G$-module isomorphism that holds for all $k\geq 0$:

[TABLE]

(a) Take any $\lambda=\mu+k\delta\in\mathrm{E}^{0}_{H\times\mathbb{F}^{\times}}(V^{*})\setminus\{2\delta\}$. As $V^{*}$ is a spherical $(H\times\mathbb{F}^{\times})$-module, by (8.1) there is a unique $l\in\{0,\ldots,[k/2]\}$ such that $R_{H}(\mu)$ is a submodule of $\left.R_{G}((k-2l)\pi_{1})\right|_{H}$. If $l>0$ then again by (8.1) $R_{H}(\mu)$ is a submodule of $\left.\mathrm{S}^{k-2l}V\right|_{H}$ and hence $\lambda-2l\delta\in\mathrm{E}_{H\times\mathbb{F}^{\times}}(V^{*})$, a contradiction. Thus $l=0$ and $\overline{\lambda}\in\Gamma_{\{1\}}(G,H)$.

Conversely, take any element $(k\pi_{1};\mu)\in\Gamma_{\{1\}}(G,H)$ and put $\lambda=\mu+k\delta\in\Lambda^{+}(H\times\mathbb{F}^{\times})$ so that $(k\pi_{1};\mu)=\overline{\lambda}$. Then by (8.1) $R_{H}(\mu)$ is a submodule of $\left.\mathrm{S}^{k}V\right|_{H}$, which yields $\lambda\in\mathrm{E}_{H\times\mathbb{F}^{\times}}(V^{*})$. If $\lambda-2\delta\in\mathrm{E}_{H\times\mathbb{F}^{\times}}(V^{*})$ then (8.1) implies that $R_{H}(\mu)$ is a submodule of $\left.\mathrm{S}^{k-2}V\right|_{H}$ and hence $\left.\mathrm{S}^{k}V\right|_{H}$ contains at least two copies of $R_{H}(\mu)$, which is impossible as $V^{*}$ is a spherical ($H\times\mathbb{F}^{\times}$)-module. Thus $\lambda$ is a linear combination of elements in $\mathrm{E}^{0}_{H\times\mathbb{F}^{\times}}(V^{*})\setminus\{2\delta\}$.

(b) Take any $\lambda=\mu+k_{1}\delta_{1}+k_{2}\delta_{2}\in\mathrm{E}^{0}_{H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times}}(V^{*})$. If $\lambda=2\delta_{1}$ or $\lambda=2\delta_{2}$ then $\overline{\lambda}=(2\pi_{1};0)$. As $\mathrm{S}^{2}V\simeq R_{G}(2\pi_{1})\oplus\mathbb{F}^{1}$ as $G$-modules and each of $\mathrm{S}^{2}V_{1}$ and $\mathrm{S}^{2}V_{2}$ contains a trivial $G$-submodule, it follows that $R_{G}(2\pi_{1})$ also contains a trivial submodule, hence $\overline{\lambda}\in\Gamma_{\{1\}}(G,H)$. Now assume $\lambda\notin\{2\delta_{1},2\delta_{2}\}$. Then $R_{H}(\mu)$ is a submodule of $\left.\mathrm{S}^{k_{1}+k_{2}}V\right|_{H}$. By (8.1), there is the minimal $l\in\{0,\ldots,[(k_{1}+k_{2})/2]\}$ such that $R_{H}(\mu)$ is a submodule of $\left.R_{G}((k_{1}+k_{2}-2l)\pi_{1})\right|_{H}$. If $l>0$ then again by (8.1) $R_{H}(\mu)$ is a submodule of $\left.\mathrm{S}^{k_{1}+k_{2}-2l}V\right|_{H}$ and hence $\lambda-s_{1}\delta_{1}-s_{2}\delta_{2}\in\mathrm{E}_{H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times}}(V^{*})$ for some $s_{1},s_{2}\in\mathbb{Z}$ with $s_{1}+s_{2}=2l$, which is impossible. Thus $l=0$ and $\overline{\lambda}\in\Gamma_{\{1\}}(G,H)$.

Conversely, take any element $(k\pi_{1};\mu)\in\Gamma_{\{1\}}(G,H)$. Then by (8.1) $R_{H}(\mu)$ is a submodule of $\left.\mathrm{S}^{k}V\right|_{H}$, hence there are $k_{1},k_{2}\geq 0$ with $k_{1}+k_{2}=k$ such that $\lambda=\mu+k_{1}\delta_{1}+k_{2}\delta_{2}\in\mathrm{E}_{H\times\mathbb{F}^{\times}\times\mathbb{F}^{\times}}(V^{*})$. As $(k\pi_{1};\mu)=\overline{\lambda}$, the proof is completed. ∎

8.2. Case $I\neq\{1\}$

For Case 6 in Table 6 as well as for all cases appearing in Theorems 4.10 and 4.11 the monoids $\Gamma_{I}(G,H)$ are computed as follows.

The rank of $\Gamma_{I}(G,H)$ is calculated using formula (5.1). For each triple $(G,H,I)$, the spherical module $(M,\mathfrak{g}/(\mathfrak{p}_{I}^{-}+\mathfrak{h}))$ is computed in the corresponding part of § 6 or § 7 and the rank of this module is calculated as described in § 3.5.

Once the rank of $\Gamma_{I}(G,H)$ has been determined, the indecomposable elements of this monoid are found by using the following straightforward observation generalizing [AvPe2, Propositions 4.9 and 4.10].

Proposition 8.1**.**

Let $I=\{i_{1},\ldots,i_{k}\}\subset S$ and choose a nonzero tuple $(a_{1},\ldots,a_{k})$ of nonnegative integers. Let $(\lambda_{0};\mu_{0})\in\Gamma_{I}(G,H)$ be an element such that $\lambda_{0}=a_{1}\pi_{i_{1}}+\ldots+a_{k}\pi_{i_{k}}\in\Lambda^{+}(G)$. Let $J$ be the set of indecomposable elements of $\Gamma_{I}(G,H)$ having the form $(b_{1}\pi_{i_{1}}+\ldots+b_{k}\pi_{i_{k}};*)$ for a nonzero tuple $(b_{1},\ldots,b_{k})\neq(a_{1},\ldots,a_{k})$ satisfying $b_{1}\leq a_{1},\ldots,b_{k}\leq a_{k}$. Suppose that $(\lambda_{0};\mu_{0})\notin\mathbb{Z}^{+}J$. Then $(\lambda_{0};\mu_{0})$ is an indecomposable element of $\Gamma_{I}(G,H)$.

In the situation of the above proposition, one successively computes all indecomposable elements of $\Gamma_{I}(G,H)$ of the form $(a_{1}\pi_{i_{1}}+\ldots+a_{k}\pi_{i_{k}};*)$ first with $a_{1}+\ldots+a_{k}=1$, then with $a_{1}+\ldots+a_{k}=2$, and so on until the required number of indecomposable elements has been found.

To implement the above algorithm for computing the indecomposable elements of $\Gamma_{I}(G,H)$, one should be able to compute explicitly the restriction to $H$ of any given representation $R_{G}(\lambda)$ with $\lambda\in\Lambda^{+}_{I}(G)$. For each of the cases in Tables 6, 8, and 9, the inclusion $G\supset H$ fits into a chain $G=H_{0}\supset H_{1}\supset\ldots\supset H_{k}=H$ where for each $i=1,\ldots,k$ one of the following possibilities holds:

• •

$H_{i}$ is a symmetric subgroup of $H_{i-1}$;

• •

$H_{i}$ is a Levi subgroup of $H_{i-1}$;

• •

$H_{i-1}=\operatorname{SO}_{7}\times K$ for some group $K$ and $H_{i}=\mathsf{G}_{2}\times K$.

In the former two cases, the restrictions are computed using the information in [AvPe2, §§ 5.2, 5.3]. In the latter case, the restrictions are computed either via [AkPa, Theorem 8, part 3] or directly by using the program LiE [LiE1].

Appendix A Explicit embeddings $\mathfrak{g}_{2}\subset\mathfrak{so}_{7}$ and $\mathfrak{spin}_{7}\subset\mathfrak{so}_{8}$

In this appendix, we present explicit realizations of the algebra $\mathfrak{g}_{2}$ as a subalgebra of $\mathfrak{so}_{7}$ and also of the algebra $\mathfrak{spin}_{7}$ as a subalgebra of $\mathfrak{so}_{8}$. These realizations are widely used in § 7 for explicit calculations.

The algebra $\mathfrak{g}_{2}$ is realized as the subalgebra of $\mathfrak{so}_{7}$ consisting of all matrices of the form

[TABLE]

The algebra $\mathfrak{spin}_{7}$ is realized as the subalgebra of $\mathfrak{so}_{8}$ consisting of all matrices of the form

[TABLE]

As a crucial property of these realizations, in both cases $\mathfrak{k}=\mathfrak{g}_{2}\subset\mathfrak{so}_{7}$ and $\mathfrak{k}=\mathfrak{spin}_{7}\subset\mathfrak{so}_{8}$ the set $\mathfrak{b}^{+}$ of all upper-triangular (and also the set $\mathfrak{b}^{-}$ of all lower-triangular) matrices in $\mathfrak{k}$ is a Borel subalgebra of $\mathfrak{k}$ and the set $\mathfrak{t}$ of all diagonal matrices in $\mathfrak{k}$ is a Cartan subalgebra of $\mathfrak{k}$.

In the case $\mathfrak{k}=\mathfrak{g}_{2}\subset\mathfrak{so}_{7}$, if $\alpha_{1},\alpha_{2}\in\mathfrak{t}^{*}$ are the two simple roots with respect to $\mathfrak{b}^{+}$ then for every positive root $i\alpha_{1}+j\alpha_{2}$ the corresponding root subspace in $\mathfrak{k}$ is spanned by the matrix for which $x_{ij}=1$ and all the other coordinates equal [math]. Similarly, the root subspace in $\mathfrak{k}$ corresponding to the negative root $-(i\alpha_{1}+j\alpha_{2})$ is spanned by the matrix for which $y_{ij}=1$ and all the other coordinates equal [math].

In the case $\mathfrak{k}=\mathfrak{spin}_{7}\subset\mathfrak{so}_{8}$, if $\alpha_{1},\alpha_{2},\alpha_{3}\in\mathfrak{t}^{*}$ are the three simple roots with respect to $\mathfrak{b}^{+}$ then for every positive root $i\alpha_{1}+j\alpha_{2}+k\alpha_{3}$ the corresponding root subspace in $\mathfrak{k}$ is spanned by the matrix for which $x_{ijk}=1$ and all the other coordinates equal [math]. Similarly, the root subspace in $\mathfrak{k}$ corresponding to the negative root $-(i\alpha_{1}+j\alpha_{2}+k\alpha_{3})$ is spanned by the matrix for which $y_{ijk}=1$ and all the other coordinates equal [math].