Representations of abstract copolarity one and two

TL;DR

This paper investigates the cohomogeneity of reducible representations with copolarity 1 and 2, extending previous research by notable mathematicians in the field.

Contribution

It generalizes existing results on the cohomogeneity of representations with copolarity 1 and 2, broadening the understanding of their structure.

Findings

Extended classification of reducible representations with copolarity 1 and 2.

Provided new insights into the cohomogeneity of these representations.

Built upon and generalized prior results by Gorodski, Olmos, Tojeiro, and Lytchak.

Abstract

We study the cohomogeneity of possibly reducible representations of copolarity 1 and 2, generalizing results obtained by Claudio Gorodski, Carlos Olmos, Ruy Tojeiro and by Claudio Gorodski, Alexander Lytchak.

Equations

$$W/H=X_1\times X_2$$

$$W/H=X_0\times X_1\times \cdots \times X_n,$$

$$(H_0\times H_1\times \cdots \times H_n,W_0\oplus W_1\oplus \cdots \oplus W_n),$$

$$\dim G\le \dim G_1+\dim G_2\le \dim G_1^{\prime }+\dim G_2^{\prime }.$$

$$\dim \text{Str}(p)=\dim W^{H_p}+\dim H-\dim N_H(H_p),$$

$$V/G^{\circ }=X_0\times X_1\times \cdots \times X_k.$$

$$\frac{V}{G}=\frac{X_{\text{flat}}}{\Gamma }\times \frac{X_1}{\Gamma }\times \cdots \times \frac{X_m}{\Gamma }.$$

$$\dim V^{\omega }+\dim G-\dim Z_G(\omega )=\dim V-1,$$

$$V=V_1\oplus \cdots \oplus V_m,$$

$$\bar{t}\cdot u=c_{\omega }(t)\cdot u=t\cdot u,\text{i.e.}{t}^2\cdot u=u.$$

$$V=V_1\oplus \cdots \oplus V_{k+1}$$

$$\sum {\text{codim}}_{V_i}{V}_i^{{\omega }_i}={\text{codim}}_V{V}^{\omega }=k+1,$$

$$V=V_{+}\oplus V_{-}\oplus \bar{V}.$$

$$\varphi (t)w=t\cdot w=\omega t\omega \cdot w=c_{\omega }(t)\cdot w=\varphi (c_{\omega }(t))w,$$

$${\text{codim}}_{V_{+}}{V}_{+}^{{\omega }_{+}}+{\text{codim}}_{V_{-}}{V}_{-}^{{\omega }_{-}}+{\text{codim}}_{\bar{V}}{\bar{V}}^{\bar{\omega }}={\text{codim}}_V{V}^{\omega }=2$$

$$W_1/H=(V_1\oplus V_2)/G,W_2/H=V_{-}/G.$$

$$\rho =(H,W),{\rho }_1=(H,W_1),{\rho }_2=(H,W_2),$$

$$\tilde{\rho }:=(H^{\prime }\times H^{\prime \prime },W^{\prime }\oplus W^{\prime \prime }),$$

$$W^{\prime }=W_1\oplus {\bar{W}}_1.$$

$$\rho {\simeq }_{o.e.}(H^{\prime }\times H^{\prime \prime },W^{\prime }\oplus W^{\prime \prime }){\simeq }_{o.e.}((\text{SO}(2)\times \text{SO}(n))\times S^1,({\mathbb{R}}^2\otimes {\mathbb{R}}^n)\oplus W_2)$$

Full text

Representations of abstract copolarity one and two

Francesco Panelli and Carolin Pomrehn

• Abstract. We study the cohomogeneity of possibly reducible representations of copolarity $1$ and $2$, generalizing results obtained by Claudio Gorodski, Carlos Olmos, Ruy Tojeiro and by Claudio Gorodski, Alexander Lytchak.

1   Introduction

††2010 Mathematics Subject Classification. Primary 57S15, 20G05.††The first author was supported by GNSAGA of INdAM – Roma (Italy).

Consider an orthogonal representation $\rho:H\rightarrow\text{\bf O}(W)$, written $\rho=(H,W)$ for short, of a compact Lie group $H$ on a real vector space $W$. A subspace $\Sigma$ of $W$ is called a generalized section, or, more precisely, a $k$-section, if it intersects all orbits, and contains the normal space to principal orbits it meets at each intersection point with codimension $k$ (cf. [7]). The minimum of all such $k$, c$(\rho)$, is called the copolarity of $\rho$. The copolarity of a representation has been related to its cohomogeneity in some particular cases: for instance, it is proved that an irreducible representation of copolarity $1$ has cohomogeneity $3$ (cf. [7, Theor. 1.1]). In [6, Cor. 1.6] this result was generalized showing that an irreducible representation of copolarity $k$, $1\leq k\leq 6$, has cohomogeneity $k+2$.

In this paper we analyze the question whether these results continue to hold for non-necessarily irreducible representations in the special case $k=1,2$, giving another partial answer to the question stated by Claudio Gorodski and Alexander Lytchak in [6, Question 1.15], asking for general relationships between copolarty and cohomogeneity of representations. In the reducible case, there is a standard method to construct counterexamples (cf. [7]). Indeed consider a product representation $\rho:=(H_{1}\times H_{2},W_{1}\oplus W_{2})$ (here $H_{1}\times H_{2}$ acts componentwisely) where $(H_{2},W_{2})$ is polar. Then $\text{c}(\rho)=\text{c}(H_{2},W_{2})$, but the cohomogeneity of $\rho$ is grater than that of $(H_{1},W_{1})$. In this way we have a family of representations with constant copolarity, but arbitrarily large cohomogeneity. Therefore, the question is interesting only for a smaller class of representations, namely those which are indecomposable; roughly speaking, they are representations that cannot be written as a product of subrepresentations (cf. Definition 2.1 below). In the case c$(\rho)=1$ we show that the counterexamples described above are the only ones that can occour.

Theorem 1.1**.**

Let $\rho:H\rightarrow\text{\bf O}(W)$ be an indecomposable representation of a connected, compact Lie group $H$ which is not the $1$-dimensional torus $\mathbb{T}^{1}$. If $\rho$ has copolarity $1$, then it has cohomogeneity $3$.

The case c$(\rho)=2$ is slightly more complicated, and in fact there are some exceptions which will be completely described. Actually our result will be proved in a more general setting, and before stating it we need to recall some definitions from [6, Section 1].

Definition 1.2**.**

Let $\rho:H\rightarrow\text{\bf O}(W)$ and $\tau:G\rightarrow\text{\bf O}(V)$ be representations of two compact Lie groups $G$, $H$.

1. i.

We say that $\rho$ and $\tau$ are quotient-equivalent if their orbit spaces $W/H$ and $V/G$ are isometric. If moreover $\dim G<\dim H$, we say that $\tau$ is a reduction of $\rho$. 2. ii.

We say that $\tau$ is reduced if $\dim G$ is minimal in the quotient-equivalence class of $\tau$. In this case $\text{ac}(\tau):=\dim G$ is called the abstract copolarity of $\tau$ (or of any representation quotient-equivalent to $\tau$). 3. iii.

A reduction of $\rho$ which is reduced is called a minimal reduction of $\rho$.

The simplest example of quotient-equivalence is orbit-equivalence: two representations $\rho_{1}$, $\rho_{2}$ are called orbit-equivalent if they have the same orbits up to an isometry between the representation spaces; in this case we write $\rho_{1}\simeq_{o.e.}\rho_{2}$.

For later use, we shall now briefly describe two simple ways to obtain a reduction from a given representation $\rho:H\rightarrow\text{\bf O}(W)$.

First consider the principal isotropy group $K$ of $\rho$, and let $W^{K}$ denote the subspace of all fixed points of $K$. Clearly the normalizer $N_{H}(K)$ acts on $W^{K}$. The inclusion map $W^{K}\rightarrow W$ induces then an isometry $W^{K}/\bar{N}\rightarrow W/H$, where $\bar{N}:=N_{H}(K)/K$ (cf. [8, 11, 14]). If $K$ is not trivial, then $(\bar{N},W^{K})$ is a reduction of $(H,W)$, which is called Luna-Richardson-Straume reduction, LRS-reduction in the sequel.

Let now $\Sigma$ be a $k$-section for $\rho$, where $k$ is minimal (so that $k=\text{c}(\rho)$). The group $N_{H}(\Sigma)$ of all elements $h\in H$ preserving $\Sigma$ acts on $\Sigma$ with a kernel which we denote by $Z_{H}(\Sigma)$. Then the inclusion map $\Sigma\rightarrow W$ yields an isometry $\Sigma/\bar{N}_{\Sigma}\rightarrow W/H$, where $\bar{N}_{\Sigma}:=N_{H}(\Sigma)/Z_{H}(\Sigma)$ (cf. [7]). If $\Sigma$ is a proper subspace of $W$ (in this case, following [7], we say that $\rho$ has non-trivial copolarity), then $(\bar{N},\Sigma)$ is a reduction of $(H,W)$. Notice that abstract copolarity is bounded above by copolarity; however it is not known whether these two invariants coincide in general (cf. [6, p. 69]).

We may now state our second main result:

Theorem 1.3**.**

Let $\rho:H\rightarrow\text{\em\bf O}(W)$ be a non-reduced, indecomposable representation of a compact connected Lie group $H$ of abstract copolarity $2$. Then either

1. i.

$\rho$* has cohomogeneity $4$, or* 2. ii.

$\rho=\rho_{1}\oplus\rho_{2}$* where*

1. (a)

$\rho_{1}:H\rightarrow\text{\bf O}(W_{1})$* is orbit-equivalent to the isotropy representation of a rank $2$ real grassmannian,* 2. (b)

$\rho_{2}:H\rightarrow\text{\bf O}(W_{1}^{\perp})$* is orbit-equivalent to a non-polar $\mathbb{T}^{1}$-representation without non-trivial fixed points.*

Conversely, let $\rho_{1}$ be the isotropy representation of a rank $2$-real grasmannian, $\rho_{2}$ be a non-polar $\mathbb{T}^{1}$-representation without non-trivial fixed points and set $\rho:=\rho_{1}\oplus\rho_{2}$. Then $\rho$ is indecomposable, has cohomogeneity $\neq 4$, and both its copolarity and abstract copolarity are equal to $2$.

We remark here that Theorem 1.1 may be expressed in terms of abstract copolarity as well, and in fact it will be proved in such a setting; we refer to Section 4, Theorem 4.1, for its precise statement. From the latter result we shall be able to derive the following:

Corollary 1.4**.**

A representation $\rho:H\rightarrow\text{\bf O}(W)$ of a connected compact Lie group $H$ has abstract copolarity $1$ if and only if it has copolarity $1$.

This means in particular that Theorems 1.1 and 4.1 are equivalent. We don’t know whether the analogous of Corollary 1.4 when copolarity and abstract copolarity are equal to $2$ holds; however note that a representation of a connected compact Lie group which has copolarity $2$ must have abstract copolarity $2$.

Another motivation for Theorems 1.1, 1.3 comes from Theorem 1.7 in [6]. Indeed, an easy consequence of such a result asserts that if $\rho=(H,W)$ is an irreducible representation of a connected compact group $H$ admitting a minimal reduction $\tau=(G,V)$ where $G$ is a finite extension of a $k$-dimensional torus, $k\geq 1$, then the cohomogeneity of $\rho$ equals $k+2$.

As a technical tool we shall need the following observation, which, we think, is interesting on its own, and might be useful in other contexts:

Proposition 1.5**.**

Let $\rho:H\rightarrow\text{\bf O}(W)$ be a non-polar indecomposable representation of a connected, compact Lie group $H$. If $G\rightarrow\text{\bf O}(V)$ is quotient-equivalent to $\rho$, then the induced representation $G^{\circ}\rightarrow\text{\bf O}(V)$ is indecomposable.

The paper is organized as follows. In Section 2 we formally define decomposable representations and give a useful criterion for decomposability of toric representations. In Section 3 we prove Proposition 1.5 and recall the notion of nice involution from [6], adapting it to our particular setting. Finally in Section 4 we prove our main results and Corollary 1.4.

Acknowledgements. The first author is very grateful to Prof. A. Lytchak for kindly introducing him into the topic of this paper; he wishes also to thank Prof. C. Gorodski for very useful conversations and many valuable remarks and Prof. F. Podestà for several suggestions and his constant support. Part of this work was completed while the first author was visiting the University of Köln, which he wishes to thank for hospitality.

The second author would like to thank Prof. G. Thorbergsson, Prof. A. Lytchak and Dr. S. Wiesendorf for their support and many helpful discussions.

2   Decomposability of representations

In this section we rigorously define the notion of decomposability for representations and develop some properties and consequences of the definition that will be needed later.

Definition 2.1**.**

Let $\rho=(H,W)$ be a representation of a compact Lie group $H$. We say that $\rho$ is decomposable if the metric space $W/H$ splits as a product

[TABLE]

for suitable metric spaces $X_{1}$, $X_{2}$ neither of which is isometric to a single point.

Note that the notion of decomposability has already been used in the more general setting of singular Riemannian foliations by Alexander Lytchak in [12] and Marco Radeschi in [13].

Lemma 2.2**.**

Let $(H,W)$ be a decomposable representation of the compact Lie group $H$, and set $W/H=X_{1}\times X_{2}$. Then there exists an $H$-invariant splitting $W=W_{1}\oplus W_{2}$ of $W$ such that $X_{i}$ is isometric to the quotient space of the induced representation $\rho_{i}:=(H,W_{i})$, $i=1,2$. In particular $(H,W)$ is orbit-equivalent to the product representation $(H_{1}\times H_{2},W_{1}\oplus W_{2})$, where $H_{i}:=\rho_{i}(H)$.

Proof.

Let $SW$ denote the unit sphere in $W$, and set $S:=SW/H$, $X:=W/H$. Notice that $S$ is the unit sphere in $X$, which in turn coincides with the cone $CS$ over the metric space $S$. So we have a splitting of the cone $CS$ as a product $X_{1}\times X_{2}$. Let $O$ be the vertex of $CS$, and use the same letter to denote both the projections of $O$ in $X_{1}$ and $X_{2}$. Identify then $X_{i}$ with the $X_{i}$-fibre through $O$, and set $Y_{i}:=X_{i}\cap S$, $i=1,2$. It is easily seen that $X_{i}$ is isometric to the cone $CY_{i}$ over $Y_{i}$, so $S$ is isometric to the spherical join $Y_{1}\ast Y_{2}$ of $Y_{1}$ and $Y_{2}$ (cf. [2, p. 63] and note that diam$(S)\leq\pi$). Since, for $i,j=1,2$, $i\neq j$, $Y_{i}$ is the set of all points in $S$ having dinstance $\pi/2$ from each point in $Y_{j}$, we deduce, using the discussion in [6, p. 76], that we may find two $H$-invariant complementary subspaces $W_{1}$, $W_{2}$ of $W$ such that $Y_{i}=SW_{i}/H$, i.e. $X_{i}=W_{i}/H$.

Now we show that the product representation $(H_{1}\times H_{2},W_{1}\oplus W_{2})$ and $(H,W)$ have the same orbits. Indeed, if $w\in W$, it is clear that $H\cdot w\subseteq(H_{1}\times H_{2})\cdot w$. On the other hand, the representations above are quotient-equivalent, hence there exists $w^{\prime}\in W$ such that $(H_{1}\times H_{2})\cdot w^{\prime}=H\cdot w$. Thus $(H_{1}\times H_{2})\cdot w^{\prime}=H\cdot w\subseteq(H_{1}\times H_{2})\cdot w$ which yields $H\cdot w=(H_{1}\times H_{2})\cdot w$, as claimed. ∎

By the main theorem in [5], given a representation $(H,W)$ the quotient metric space $W/H$ splits as a product of metric spaces

[TABLE]

where $X_{0}$ is euclidean (possibly trivial), and where each $X_{i}$, $i\geq 1$, is a non-trivial non-euclidean metric space which is irreducible, in the sense that it cannot be further decomposed as a non-trivial product of metric spaces. Moreover decomposition (1) is unique up to a permutiation of the isometric factors. Lemma 2.2 yields then:

Proposition 2.3**.**

Let $(H,W)$ be a representation of a compact Lie group $H$. Then there exists an $H$-invariant splitting $W=W_{0}\oplus W_{1}\oplus\cdots\oplus W_{n}$ of $W$ such that:

1. i.

the induced representation $\rho_{0}:=(H,W_{0})$ is trivial and $\dim W_{0}\geq 0$; 2. ii.

the induced representation $\rho_{i}:=(H,W_{i})$ is indecomposable, non-trivial and $\dim W_{i}\geq 1$ for $i=1,\dots,n$; 3. iii.

$(H,W)$* is orbit-equivalent to the product representation*

[TABLE]

where $H_{0}:=\{1\}$, $H_{i}:=\rho_{i}(H)$, $i=1,\dots,n$.

Remark 2.4*.*

The abstract copolarity of a product representation $\rho$ of the form $(H_{1}\times H_{2},W_{1}\oplus W_{2})$ equals the sum of the abstract copolarities of $\rho_{i}:=(H_{i},W_{i})$, $i=1,2$. Indeed, let $(G,V)$ be a minimal reduction of $\rho$, and let $(G_{i},V_{i})$ be a minimal reduction of $\rho_{i}$. By Lemma 2.2 $(G,V)$ is orbit-equivalent to a product representation $(G_{1}^{\prime}\times G_{2}^{\prime},V_{1}^{\prime}\oplus V_{2}^{\prime})$, where $V=V_{1}^{\prime}\oplus V_{2}^{\prime}$. Clearly we have

[TABLE]

The first inequality means that $\text{ac}(\rho)\leq\text{ac}(\rho_{1})+\text{ac}(\rho_{2})$. If the two sides were not equal, than $\dim G<\dim G_{1}^{\prime}+\dim G_{2}^{\prime}$, showing that $(G,V)$ is a minimal reduction of $(G_{1}^{\prime}\times G_{2}^{\prime},V_{1}^{\prime}\oplus V_{2}^{\prime})$; so $\dim V<\dim V_{1}^{\prime}+\dim V_{2}^{\prime}$, a contradiction.

We shall use now Lemma 2.2 to characterize decomposability of representations of the torus $\mathbb{T}^{k}$. This result will be needed several times later on.

It is well-known that (finite-dimensional) irreducible complex representations of $\mathbb{T}^{k}$ are in 1-1 correspondence with Lie groups homomorphisms $\mathbb{T}^{k}\rightarrow S^{1}$. Moreover, given any complex representation $(\mathbb{T}^{k},U)$ and any such homomorphism $\phi$, we denote by $U_{\phi}$ the isotypical component of $\phi$, i.e. the generalized eigenspace of $U$ with generalized eigenvalue $\phi$. If $U_{\phi}\neq\{O\}$, we say that $\phi$ is a character of the representation. Clearly $U$ can be uniquely decomposed as a direct sum of isotypical components.

In this paper we are interested only in real representations $\rho=(\mathbb{T}^{k},V)$, therefore we apply the theory to the complexified representation $(\mathbb{T}^{k},V^{\mathbb{C}})$. Let $V_{0}$ be the set of fixed points of $\rho$; then $V$ can be uniquely decomposed as $V=V_{0}\oplus V_{1}\oplus\cdots\oplus V_{n},$ where, $V_{i}$, $i\geq 1$, are the (real) isotypical components of $V$, i.e. each $V_{i}$ has the form $(V^{\mathbb{C}}_{\phi}\oplus V^{\mathbb{C}}_{\bar{\phi}})\cap V$ for a suitable non-trivial character $\phi$ of $\rho$.

We identify the Lie algebra of $S^{1}$ with the real line, and associate to any character $\phi$ of $\rho$ (and so to any isotypical component of $\rho$) a $1$-form $\theta_{\phi}$ on the Lie algebra $\mathfrak{t}^{k}$ of $\mathbb{T}^{k}$, namely $\theta_{\phi}:=\text{d}\phi_{e}$. Such $1$-forms are called weights of $\rho$.

Proposition 2.5**.**

Let $\rho=(\mathbb{T}^{k},V)$ be a representation of the torus $\mathbb{T}^{k}$, and let $\Theta\subseteq(\mathfrak{t}^{k})^{\ast}$ denote the set of all weights of $\rho$. Assume that there exist two non-empty subsets $\Theta_{1}$, $\Theta_{2}\subseteq\Theta$ so that $\Theta_{1}\cup\Theta_{2}=\Theta$ and $\langle\Theta_{1}\rangle\cap\langle\Theta_{2}\rangle=\{O\}$. Then $\rho$ is decomposable.

Proof.

If $\Theta_{i}$, $i=1,2$, are as in the statement, we denote by $V_{i}$ the sum of all isotypical components of $\rho$ corresponding to weights in $\Theta_{i}$, and by $\mathfrak{a}_{i}\subseteq\mathfrak{t}^{k}$ the annihilator of $\langle\Theta_{i}\rangle$. From $\langle\Theta_{1}\rangle\cap\langle\Theta_{2}\rangle=\{O\}$ we get $\mathfrak{a}_{1}+\mathfrak{a}_{2}=\mathfrak{t}^{k}$ (direct sum if and only if $\rho$ has discrete kernel), while from $\Theta_{1}\cup\Theta_{2}=\Theta$ we get $V_{1}\oplus V_{2}=V$. Let $T_{i}$ be the subtorus of $\mathbb{T}^{k}$ with Lie algebra $\mathfrak{a}_{i}$. The exponential map of $\mathbb{T}^{k}$ is surjective, hence any element $t\in\mathbb{T}^{k}$ can be written as $t=t_{1}t_{2}$ for suitable $t_{i}\in T_{i}$. Since the induced representation $(T_{i},V_{i})$ is trivial, we deduce that $\rho$ and the product representation $(T_{2}\times T_{1},V_{1}\oplus V_{2})$ have the same orbits, so $\rho$ is decomposable. ∎

Remark 2.6*.*

Using notation of Proposition 2.5, we see that $\rho$ has discrete kernel if and only if $\langle\Theta\rangle=(\mathfrak{t}^{k})^{\ast}$. Indeed, an element $x\in\mathfrak{t}^{k}$ belongs to the Lie algebra of $\ker\rho$ if and only if it is annihilated by all weights of $\rho$.

In order to state the main Corollary of Proposition 2.5 we need to introduce some terminology. Precisely, if $\rho=(\mathbb{T}^{k},V)$ is a representation of the torus $\mathbb{T}^{k}$, we say that a line $\mathfrak{s}\subseteq(\mathfrak{t}^{k})^{\ast}$ is induced by $\rho$ if $\rho$ has a character $\phi$ such that $\langle\theta_{\phi}\rangle=\mathfrak{s}$.

Corollary 2.7**.**

Let $\rho=(\mathbb{T}^{k},V)$ be a faithful representation of the torus $\mathbb{T}^{k}$, $k\geq 2$. If $\rho$ is indecomposable, then it induces at least $k+1$ lines in $(\mathfrak{t}^{k})^{\ast}$.

Proof.

Let $\mathfrak{s}_{1},\dots$, $\mathfrak{s}_{\ell}$, $\ell\leq k$, be the lines induced by $\rho$ in $(\mathfrak{t}^{k})^{\ast}$, and let $\Theta$ be the set of all weights of $\rho$. Clearly $\langle\mathfrak{s}_{1},\dots,\mathfrak{s}_{\ell}\rangle=\langle\Theta\rangle=(\mathfrak{t}^{k})^{\ast}$, where the latter equality holds because $\rho$ is faithful, hence $\ell=k\geq 2$, and the $\mathfrak{s}_{i}$’s are linearly independent. Now define $\Theta_{1}$ to be the subset of $\Theta$ consisting of all weights $\theta$ such that $\langle\theta\rangle=\mathfrak{s}_{1}$, and similarly define $\Theta_{2}$ to be the subset of $\Theta$ consisting of all weights $\theta$ such that $\langle\theta\rangle=\mathfrak{s}_{j}$, for some $j\geq 2$. Then $\Theta_{1}$ and $\Theta_{2}$ satisfy the hypotesis of Proposition 2.5 therefore $\rho$ is decomposable. ∎

Especially for toric representations, the absence of proper generalized sections can be read of the quotient. In order to state precisely this fact, recall that, given an orthogonal representation $(H,W)$ of a compact Lie group $H$, the stratum through a point $p\in W$, Str$(p)$, is, by definition, the connected component through $p$ of the set of points whose isotropy groups are conjugated to the isotropy at $p$. We have

[TABLE]

where $H_{p}$ denotes the isotropy at $p$, while $W^{H_{p}}$ denotes the set of fixed points of $H_{p}$. The boundary (in the sense of Alexandrov) of $W/H$, $\partial(W/H)$, is defined as the closure of the union of all strata in $W/H$ that have codimension $1$.

Proposition 2.8**.**

Let $\rho=(\mathbb{T}^{k},V)$ be a representation of the $k$-dimensional torus $\mathbb{T}^{k}$. Then $\partial(V/\mathbb{T}^{k})=\varnothing$ if and only if $\rho$ has trivial copolarity.

Proof.

We may assume without loss of generality that $\rho$ is faithful and has no non-trivial fixed points. Since $\mathbb{T}^{k}$ is abelian, this in particular implies that $\rho$ has trivial principal isotropy group.

If $\partial(V/\mathbb{T}^{k})=\varnothing$, $\rho$ is reduced by [6, Prop. 1.1], so it has trivial copolarity.

Conversely, assume $\partial(V/\mathbb{T}^{k})\neq\varnothing$, and let $p\in V$ be a point projecting onto a stratum of codimension $1$ in the quotient. The non-trivial part of the slice representation at $p$ has trivial principal isotropy group and cohomogeneity $1$, therefore $\mathbb{T}^{k}_{p}$ is a sphere $S^{a}$. Since $\mathbb{T}^{k}$ is abelian we get $a\in\{0,1\}$ and, by (2), $\dim V^{\mathbb{T}^{k}_{p}}=\dim\text{Str}(p)=\dim V-a-1$, where $V^{\mathbb{T}^{k}_{p}}$ denotes the fixed point set of $\mathbb{T}^{k}_{p}$.

Assume first $a=0$, and let $\omega$ be the generator of $\mathbb{T}^{k}_{p}\simeq S^{0}\simeq\mathbb{Z}_{2}$. Then $V^{\omega}$ has codimension $1$ and $\omega$ is a reflection. On the other hand $\omega\in S^{1}\subseteq\text{\bf SO}(V)$, a contradiction.

Then $a=1$, and we have a $\mathbb{T}^{k}$-invariant decomposition $V=V^{\mathbb{T}^{k}_{p}}\oplus\bar{V},$ where $\dim\bar{V}=2$; we shall write accordingly $\rho=\tilde{\rho}\oplus\bar{\rho}$. Notice that $\bar{\rho}$ is irreducible since $\rho$ has no non-trivial fixed points. Let $\tilde{\Theta}$ be the set of weights corresponding to isotypical components in $V^{\mathbb{T}^{k}_{p}}$, and denote by $\bar{\theta}$ the weight of $\bar{\rho}$. Clearly any weight of $\rho$ either belongs to $\tilde{\Theta}$, or it is equal to $\bar{\theta}$; moreover all weights in $\tilde{\Theta}$ vanish on the $1$-dimensional Lie algebra of $\mathbb{T}^{k}_{p}$, $\mathfrak{t}^{k}_{p}$. Faithfulness of $\rho$ implies that $\bar{\theta}$ cannot vanish on $\mathfrak{t}^{k}_{p}$, so $\langle\tilde{\Theta}\rangle\cap\langle\bar{\theta}\rangle=\{O\}$. By (the proof of) Proposition 2.5 $\rho$ is orbit-equivalent to the product representation $(\tilde{\rho}(\mathbb{T}^{k})\times S^{1},V^{\mathbb{T}^{k}_{p}}\oplus\bar{V})$; since $(S^{1},\bar{V})$ is polar $\rho$ cannot have trivial copolarity. ∎

3   Nice involutions

In this paper an important role is played by representations $(G,V)$ of a compact (possibly disconnected) Lie group $G$ such that $\Gamma:=G/G^{\circ}$ acts on $V/G^{\circ}$ as a reflection group, i.e. $\Gamma$ is generated by elements which act on $V/G^{\circ}$ as reflections. Notice that the representation $(G,V)$ appearing in the statement of Proposition 1.5 is of this kind, since it is quotient-equivalent to a representation of a connected group (cf. [6, Prop. 1.2]).

Proof of Proposition 1.5.

By hypotesis and what we observed above, $(G,V)$ is non-polar, indecomposable and the quotient group $\Gamma:=G/G^{\circ}$ is generated by reflections. Decompose the quotient $V/G^{\circ}$ of the induced representation $(G^{\circ},V)$ as in (1):

[TABLE]

We may assume that $X_{1},\dots$, $X_{m}$ are the non-flat factors appearing in (3), and set $X_{\text{flat}}:=X_{0}\times X_{m+1}\times\cdots\times X_{k}$. Since a representation is polar if and only if the principal part of its orbit space is flat (cf. [1, 9]), and since non-polarity of $(G,V)$ is equivalent to non-polarity of $(G^{\circ},V)$, we deduce that $m\geq 1$. Now, any reflection in $\Gamma$ either preserves all $X_{i}$, being a reflection on one of them and fixing pointwisely the others, or interchanges $X_{j}$, $X_{m}$, $j\neq m$, and fixes pointwisely the others. In the latter case $X_{j}$, $X_{m}$ have dimension $1$, so they are flat; therefore

[TABLE]

Since $(G,V)$ is indecomposable and $m\geq 1$ we deduce that $X_{\text{flat}}$ is trivial and that $m=1$; this finishes the proof. ∎

Among all representations $\rho=(G,V)$ where the quotient group $\Gamma:=G/G^{\circ}$ is generated by reflections, those with trivial principal isotropy group will be of particular importance to us; indeed, in this case, the discussion in [6, Section 4.3] implies that there exists a set $\Omega\subseteq\Gamma$ of reflections generating $\Gamma$ all of whose elements $\omega^{\prime}$ have a lift $\omega$ which is an involution and satisfies the following dimension formula:

[TABLE]

where $V^{\omega}$ is the fixed-point set of $\omega$, and $Z_{G}(\omega)$ is the centralizer of $\omega$ in $G$. Following [6], we shall call involutions of this kind nice involutions.

From now on $(G,V)$ will be a minimal reduction of a non-reduced indecomposable representation $(H,W)$ with $H$ connected. Note that, in this case, the orbit space $V/G=W/H$ has non-empty boundary (cf. [6, Prop. 1.1]). Of course, moreover, $(G,V)$ has trivial copolarity and trivial principal isotropy group. We shall denote by $\Gamma$ the finite group $G/G^{\circ}$.

In the following sections $G$ will be assumed to have dimension $1$ or $2$, so $G^{\circ}$ will be a $1$ or a $2$-dimensional torus; however in the remaining part of this Section we shall prove some preliminary results which don’t require the above restriction on $\dim G$, so we shall only suppose that $G$ is a finite extension of a $k$-dimensional torus $\mathbb{T}^{k}$, $k\geq 1$.

We begin collecting a few simple remarks in the following:

Lemma 3.1**.**

Under the above assumptions, the group $G$ is disconnected, and contains a set of nice involutions whose projection in $\Gamma$ is a set of generators. Moreover the induced representation $(\mathbb{T}^{k},V)$ of the identity component of $G$ is indecomposable and reduced, and its orbit space $V/\mathbb{T}^{k}$ has empty boundary.

Proof.

First we observe that $G$ cannot be connected by Proposition 2.8. Since $G$ has trivial principal isotropy group and $\Gamma$ acts on $V/\mathbb{T}^{k}$ as a reflection group (cf. [6, Prop. 1.2]), $\Gamma$ is generated by a set $\Omega$ all of whose elements can be lifted to a nice involution in $G$. Since $(G,V)$ is non-polar and indecomposable, Proposition 1.5 implies moreover that the induced representation $(\mathbb{T}^{k},V)$ is indecomposable as well. We now claim that $V/\mathbb{T}^{k}$ has no boundary, so that $(\mathbb{T}^{k},V)$ is reduced (cf. [6, Prop. 1.1]). Indeed, if $\partial(V/\mathbb{T}^{k})\neq\varnothing$, then Propositions 2.8 would yield a proper generalized section for $(\mathbb{T}^{k},V)$, and hence a proper generalized section for $(G,V)$, contradiction. ∎

We now decompose the space $V$ into its isotypical components with respect to the induced representation $(\mathbb{T}^{k},V)$:

[TABLE]

Observe that the induced action of $\mathbb{T}^{k}$ on each $V_{i}$ cannot be trivial, since otherwise $(\mathbb{T}^{k},V)$ would be decomposable, and this is impossible by Lemma 3.1.

Lemma 3.2**.**

We have that $\dim V$ is even and $\geq 2k+2$.

Proof.

What we noticed above implies that $\dim V=2m$ is even. Moreover, by Corollary 2.7 and indecomposability of $(\mathbb{T}^{k},V)$ (cf. Lemma 3.1) we get that $V$ has $m\geq k+1$ $\mathbb{T}^{k}$-isotypical components, hence $\dim V=2m\geq 2k+2$. ∎

Consider now a nice involution $\omega\in G$. Since we have $0\leq\dim Z_{G}(\omega)\leq k$, formula (4) yields $1\leq\text{codim}_{V}V^{\omega}\leq k+1$.

Lemma 3.3**.**

If $\omega\in G$ is a nice involution, then $\text{\em codim}_{V}V^{\omega}\neq 1$ or, equivalently, $\dim Z_{G}(\omega)\neq k$.

Proof.

Assume that $\omega\in G$ is a nice involution such that $\dim Z_{G}(\omega)=k$; then the identity component of $G$, $\mathbb{T}^{k}$, is contained in $Z_{G}(\omega)$, and $\omega$ preserves all $\mathbb{T}^{k}$-invariant subspaces of $V$. Since $\omega$ is a reflection, there exists a $\mathbb{T}^{k}$-invariant subspace $U$ of $V$ such that $\omega(U)=U$ and $\omega|_{U}$ is a reflection. If $\dim U=2$, then $(\mathbb{T}^{k},U)$ is equivalent to an irreducible $S^{1}$-representation, which is given by rotations in $U\simeq\mathbb{R}^{2}$; this is impossible since $Z_{G}(\omega)\supseteq\mathbb{T}^{k}$. Hence $\dim U=1$, and $\mathbb{T}^{k}$ fixes $U$ pointwisely, a contradiction. ∎

As a corollary we immediately deduce the following:

Lemma 3.4**.**

Let $\omega\in G$ be a nice involution. Then $0\leq\dim Z_{G}(\omega)\leq k-1$ and $2\leq\text{\em codim}_{V}V^{\omega}\leq k+1$.

We now assume that $\omega\in G$ is a nice involution satisfying $\text{codim}_{V}V^{\omega}=k+1$ or, equivalently, $\dim Z_{G}(\omega)=0$. Then $\text{Ad}(\omega):\mathfrak{t}^{k}\rightarrow\mathfrak{t}^{k}$ is an involution with no non-trivial fixed points and conjugation $c_{\omega}$ in $\mathbb{T}^{k}$ with respect to $\omega$ coincides with the inversion map $t\mapsto t^{-1}$. If $\phi:\mathbb{T}^{k}\rightarrow S^{1}$ is a homomorphism, we have then $\phi\circ c_{\omega}=\bar{\phi}$; this implies that $\omega$ preserves all real isotypical components of $V$ (i.e. $\omega$ preserves decomposition (5)).

Lemma 3.5**.**

If $\omega\in G$ is a nice involution such that $\text{\em codim}_{V}V^{\omega}=k+1$, and $U\subseteq V$ is a non-trivial $\mathbb{T}^{k}$-invariant subspace of $V$ which is preserved by $\omega$, then the action of $\omega$ on $U$ is not trivial.

Proof.

If $\omega$ acts trivially on $U$ we have, for any $t\in\mathbb{T}^{k}$ and $u\in U$,

[TABLE]

Thus $\mathbb{T}^{k}$ acts trivially on $U$, which implies $U=\{O\}$. ∎

Proposition 3.6**.**

If $G$ contains a nice involution $\omega$ so that $\text{\em codim}_{V}V^{\omega}=k+1$, then $V$ decomposes as

[TABLE]

where the $V_{i}$’s are the irreducible isotypical components of $V$; moreover $\omega$ preserves each $V_{i}$ and is a reflection on it. In particular we have $\dim V=2k+2$ and $\text{\em chm}(G,V)=k+2$.

Proof.

Let $V_{i}$, $i=1,\dots,m$, be the $\mathbb{T}^{k}$-isotypical components of $V$, and let $\omega_{i}$ be the restriction of $\omega$ to $V_{i}$. Since

[TABLE]

Lemma 3.5 implies $m\leq k+1$, so $m=k+1$ by Corollary 2.7. Using again Lemma 3.5 we get $\dim V_{i}=2$ for all $i=1,\dots,k+1$, so $\dim V=2k+2$ and $\text{chm}(G,V)=k+2$, as claimed. ∎

4   Proof of the main results

In this Section we are going to prove our main results, beginning form Theorem 1.1. As mentioned in the introduction, we shall restate such a Theorem in the setting of abstract copolarity:

Theorem 4.1**.**

Let $\rho:H\rightarrow\text{\bf O}(W)$ be a non-reduced, indecomposable representation of a connected, compact Lie group $H$ of abstract copolarity $1$. Then $\rho$ has cohomogeneity $3$.

Proof.

Let $(G,V)$ be a minimal reduction of $\rho$; then $G$ is a finite extension of the $1$-dimensional torus $\mathbb{T}^{1}$, and by Lemma 3.1 it contains a nice involution $\omega$. Applying Lemma 3.4 we deduce $\text{codim}_{V}V^{\omega}=2$, so $\text{chm}(\rho)=\text{chm}(G,V)=3$ by Proposition 3.6. ∎

Now Corollary 1.4 easily follows:

Proof of Corollary 1.4.

Since a representation has abstract copolarity [math] if and only if it is polar (cf. [6, p. 69]), it is clear that if $\rho:=(H,W)$ has copolarity $1$, than it has abstract copolarity $1$.

Conversely, assume that $\rho$ has abstract copolarity $1$. If it is indecomposable, then either it is reduced and there is nothing to prove, or it is not reduced and $\text{chm}(\rho)=3$ by Theorem 4.1. Now any non-polar representation of cohomogeneity $3$ has copolarity $1$ (cf. [14]), so we are done in this case.

Assume next that $\rho$ is decomposable. Exploiting Remark 2.4, we may suppose that $\rho$ is orbit equivalent to a product representation $(H_{1}\times H_{2},W_{1}\oplus W_{2})$, where $(H_{1},W_{1})$ is polar and $(H_{2},W_{2})$ is indecomposable with abstract copolarity $1$. By the above discussion $(H_{2},W_{2})$ has copolarity $1$, so the same is true for $\rho$ since the copolarity of a product representation is the sum of the copolarities of the factors. ∎

We now begin the work that will lead us to the proof of Theorem 1.3; in particular from now on $\rho:=(H,W)$ will denote a non-reduced, indecomposable representation of a connected, compact Lie group $H$ of abstract copolarity $2$, and $(G,V)$ a minimal reduction of $\rho$. Clearly $G$ is a finite extension of the $2$-dimensional torus $\mathbb{T}^{2}$, and contains a nice involution $\omega$ thank to Lemma 3.1. By Lemma 3.4 we know moreover that $\text{codim}_{V}V^{\omega}\in\{2,3\}$. If $\text{codim}_{V}V^{\omega}=3$, Proposition 3.6 implies $\text{chm}(\rho)=\text{chm}(G,V)=4$ and we are done; so we may assume that all nice involutions $\omega\in G$ satisfy $\text{codim}_{V}V^{\omega}=2$.

In what follows $\omega$ will denote an arbitrarily fixed nice involution in $G$. Our previous assumption implies then that $\text{Ad}(\omega):\mathfrak{t}^{2}\rightarrow\mathfrak{t}^{2}$ is an involution with eigenvalues $\pm 1$. Denote by $U_{\pm}\subseteq\mathfrak{t}^{2}$ the $1$-dimensional eigenspace corresponding to the eigenvalue $\pm 1$, and let $\mathfrak{s}^{\pm}$ be the annihilator of $U_{\pm}$, which is a line in $(\mathfrak{t}^{2})^{\ast}$. Finally denote by $V_{\pm}$ the sum of all $\mathbb{T}^{2}$-isotypical components of $V$ which induce the line $\mathfrak{s}^{\pm}$. If $\bar{V}:=(V_{+}\oplus V_{-})^{\perp}$, $V$ can be written as

[TABLE]

Observe that decomposition (6) depends on the choice of $\omega$.

Proposition 4.2**.**

With respect to decomposition (6), $V_{+}=\{O\}$ and $\bar{V}$ contains exactly two isotypical components $V_{1}$, $V_{2}$ which have dimension $2$. Moreover $(\mathbb{T}^{2},V)$ induces exactly $3$ lines in $(\mathfrak{t}^{2})^{\ast}$ and $\omega$ acts as the identity on $V_{-}$ and interchanges $V_{1}$, $V_{2}$.

The proof of Proposition 4.2 requires the following:

Lemma 4.3**.**

Assume $W\subseteq V$ is a $\mathbb{T}^{2}$-invariant subspace on which $\omega$ acts as the identity. Then $W\subseteq V_{-}$.

Proof.

We may suppose without loss of generality that $W$ is irreducible; denoting by $\phi$ the corresponding character, we need to prove that $\text{d}\phi_{e}(x)=0$ whenever $x\in U_{-}$. Since such an $x$ must be of the form $\text{Ad}(\omega)(y)-y$ for a suitable $y\in\mathfrak{t}^{2}$, this corresponds to show that $\text{d}(\phi\circ c_{\omega})_{e}(y)=\text{d}\phi_{e}(y)$ for any $y\in\mathfrak{t}^{2}$. On the other hand if $t\in\mathbb{T}^{2}$ and $w\in W$ we have

[TABLE]

so that $\phi=\phi\circ c_{\omega}$ on $\mathbb{T}^{2}$ and our claim follows. ∎

We will denote by $\omega_{\pm}$, $\bar{\omega}$, respectively, the restriction of $\omega$ to $V_{\pm}$, $\bar{V}$.

Proof of Proposition 4.2.

First we note that $\bar{V}\neq\{O\}$; indeed, otherwise $(\mathbb{T}^{2},V)$ would induce only two lines in $(\mathfrak{t}^{2})^{\ast}$, and so it would be decomposable by Corollary 2.7, contradicting Lemma 3.1. Now, let $V_{1}$ be a $\mathbb{T}^{2}$-isotypical component contained in $\bar{V}$; clearly $V_{2}:=\omega(V_{1})$ is a $\mathbb{T}^{2}$-isotypical component contained in $\bar{V}$ different from $V_{1}$, so by [6, Lemma 6.2] we deduce that the action $(\mathbb{T}^{2},V_{i})$, $i=1,2$, has cohomogeneity $1$. Since principal orbits of such representations are $1$-dimensional, we get $\dim V_{i}=2$ and $\text{codim}_{\bar{V}}\bar{V}^{\bar{\omega}}\geq\dim V_{i}=2$. From

[TABLE]

we obtain then $\text{codim}_{V_{+}}V_{+}^{\omega_{+}}=0=\text{codim}_{V_{-}}V_{-}^{\omega_{-}}$ so $\omega$ acts as the identity both on $V_{+}$ and $V_{-}$. This implies $V_{+}=\{O\}$ by Lemma 4.3. Now set $U:=V_{1}\oplus V_{2}$ and denote by $\omega^{\prime}$ the restriction $\omega|_{U}$. Since $\text{codim}_{U}U^{\omega^{\prime}}=2$, a splitting formula for $\text{codim}_{V}V^{\omega}$ similar to (7) shows that $\omega$ acts trivially on the orthogonal complement $U^{\perp}$ of $U$ in $\bar{V}$, so $U^{\perp}=\{O\}$ by Lemma 4.3 and $\bar{V}=V_{1}\oplus V_{2}$. ∎

Let $\omega_{1}\in G$ be a nice involution, and decompose $V=V_{-}\oplus V_{1}\oplus V_{2}$ as in Proposition 4.2 with respect to $\omega_{1}$. We denote by $\mathfrak{s}^{-}$, $\mathfrak{s}_{1}$, $\mathfrak{s}_{2}$ the lines in $(\mathfrak{t}^{2})^{\ast}$ induced by $(\mathbb{T}^{2},V_{-})$, $(\mathbb{T}^{2},V_{1})$, $(\mathbb{T}^{2},V_{2})$ respectively. Any nice involution $\omega_{2}\in G$ permutes the $\mathbb{T}^{2}$-isotypical components of $V$, so, via its natural action on $(\mathfrak{t}^{2})^{\ast}$, it permutes the lines $\mathfrak{s}^{-}$, $\mathfrak{s}_{1}$, $\mathfrak{s}_{2}$. If $\omega_{2}$ does not fix $\mathfrak{s}^{-}$, then $\dim V_{-}=2$, $\dim V=6$ and $\text{chm}(G,V)=4$, so Theorem 1.3 holds. Note that in this case all $\mathbb{T}^{2}$-isotypical components of $V$ are irreducible and $G$ acts as the full permutation group on them, so $(G,V)$, as well as the original representation $(H,W)$, is irreducible. Henceforth we shall suppose that all nice involutions $\omega\in G$ fix $\mathfrak{s}^{-}$ and interchange $\mathfrak{s}_{1}$, $\mathfrak{s}_{2}$.

Remark 4.4*.*

Under the above assumption it is not hard to prove that any two nice involutions in $G$ project onto the same element in the quotient $G/G^{\circ}$, thus $G/G^{\circ}=\mathbb{Z}_{2}$ in this case.

Lemma 3.2 implies now that $\dim V_{-}\geq 2$; if $\dim V_{-}=2$, then $\dim V=6$ and $\text{chm}(H,W)=4$, so we shall assume $\dim V_{-}\geq 3$ and show that we are in case (ii) of Theorem 1.3.

Notice that $V_{1}\oplus V_{2}$ and $V_{-}$ are $G$-invariant subspaces of $V$; by [6, Lemma 5.1], we can then find two $H$-invariant orthogonal complementary subspaces $W_{1}$, $W_{2}\subseteq W$ such that

[TABLE]

Let $\rho_{i}$ be the induced representaton $(H,W_{i})$; recalling that $\rho:=(H,W)$ in the notation of Theorem 1.3, we get $\rho=\rho_{1}\oplus\rho_{2}$.

First observe that $(G,V_{-})$ has no non-trivial fixed points by Lemma 3.2; so $\rho_{2}$ has no non-trivial fixed points either (cf. [6, Section 5]). Moreover, since any nice involution $\omega\in G$ acts trivially on $V_{-}$, $(G,V_{-})$ has a $1$-dimensional kernel (the line $\mathfrak{s}^{-}$), and so it has the same orbits as $(S^{1},V_{-})$. Since $\dim V_{-}\geq 3$, $(S^{1},V_{-})$ cannot be polar and is a minimal reduction of $(H,W_{2})$. Moreover, applying Proposition 2.8, we deduce that $V_{-}/S^{1}=W_{2}/H$ does not have boundary. This implies, by [6, Prop. 1.1], that $\rho_{2}$ is itself reduced, so $\rho_{2}(H)=S^{1}\subseteq\text{\bf SO}(W_{2})$ is $1$-dimensional.

We now study $(G,V_{1}\oplus V_{2})$. Clearly it is irreducible, has cohomogeneity $2$ and so is polar. This is of course true for $\rho_{1}$ as well, so the latter is orbit-equivalent to the isotropy representation of a rank $2$ symmetric space (cf. [3]). The proof that this is in fact a real grassmannian will require a few remarks.

We begin with the observation that $\rho_{2}(H)=S^{1}$, hence $H$ cannot be semisimple; we shall write then $H=Z^{\circ}\cdot H_{s}$, where $Z^{\circ}$, the identity component of the centre $Z(H)$ of $H$, is a torus $\mathbb{T}^{a}$, $a\geq 1$, and $H_{s}$ is the semisimple part of $H$. The same argument implies moreover that the induced representation $(H_{s},W_{2})$ is trivial.

Lemma 4.5**.**

$\rho_{1}(H)$* has $1$-dimensional centre.*

Proof.

Clearly $\rho_{1}(Z^{\circ})\subseteq Z(\rho_{1}(H))^{\circ}$, and the latter is either [math] or $1$-dimensional. Assume by contradiction that $\dim Z(\rho_{1}(H))=0$. Then $Z^{\circ}\subseteq\ker\rho_{1}$, which implies that $(H,W)$ is decomposable; indeed in this case $(H,W)$ is orbit-equivalent to the product representation $(H_{s}\times Z^{\circ},W_{1}\oplus W_{2})$. ∎

Lemma 4.6**.**

Set $K:=\rho_{1}(H)$, and write $K=S^{1}\cdot K_{s}$, where $K_{s}$ denotes the semisimple part of $K$. Then $(K,W_{1})$ is not orbit-equivalent to the induced representation $(K_{s},W_{1})$.

Proof.

Representations $(K,W_{1})$ and $\rho_{1}=(H,W_{1})$ are orbit-equivalent, and so are $(K_{s},W_{1})$ and $\rho_{s}:=(H_{s},W_{1})$ since $K_{s}=\rho_{1}(H_{s})$. It is then enough to prove that $\rho_{1}$ and $\rho_{s}$ are not orbit-equivalent. This is easily done, since it is not hard to show that if this were not the case, then $(H,W)$ and the product representation $(H_{s}\times Z^{\circ},W_{1}\oplus W_{2})$ would have the same orbits. ∎

Lemma 4.5 and the main theorem in [4] imply that $\rho_{1}$ is orbit-equivalent to the isotropy representation of a rank $2$ irreducible Hermitian symmetric space, i.e. one of the following (cf. [10, p.518-520]):

• •

AIII($p\geq 3$, $q=2$): $\text{\bf SU}(p+2)/\text{S}(\text{\bf U}(p)\times\text{\bf U}(q))$,

• •

DIII($n=5$): $\text{\bf SO}(10)/\text{\bf U}(5)$,

• •

BDI($p\geq 3,\;q=2$): $\text{\bf SO}(p+2)/(\text{\bf SO}(p)\times\text{\bf SO}(2))$,

• •

EIII: $\text{\bf E}_{6}/\text{\bf SO}(10)\cdot\mathbb{T}^{1}$.

Only the isotropy representation of BDI($p\geq 3,\;q=2$) satisfies condition in Lemma 4.6 (cf. [4]), and the first part of Theorem 1.3 is finally proved.

Remark 4.7*.*

Assuming $\rho$ faithful, we have $a=\dim Z(H)=1$. Indeed, since $\rho_{1}(H)=K=K_{s}\cdot S^{1}$ and $\rho_{2}(H)=S^{1}$, if $a\geq 3$ then $Z^{\circ}\simeq\mathbb{T}^{a}$ contains a subgroup of positive dimension acting trivially on $W$. So $a\leq 2$. Suppose now $a=2$. Since $\rho_{1}(Z^{\circ})\simeq\rho_{2}(Z^{\circ})\simeq S^{1}$ and $\rho$ is faithful, we may write $Z^{\circ}=S^{1}_{1}\times S^{1}_{2}$ for suitable subgroups $S^{1}_{i}$ of $Z^{\circ}$ isomorphic to $S^{1}$ in such a way that $S^{1}_{1}$ acts trivially on $W_{2}$ and $S^{1}_{2}$ acts trivially on $W_{1}$. Hence $H_{s}\cdot S^{1}_{1}$ acts trivially on $W_{2}$ and we conclude that $\rho$ is orbit-equivalent to the natural product representation $((H_{s}\times S^{1}_{1})\times S^{1}_{2},W_{1}\oplus W_{2})$. Once again this is a contradiction.

We now turn to the proof of the second part of Theorem 1.3; in what follows we shall use notation introduced in its statement. We set moreover $H:=\text{\bf SO}(2)\times\text{\bf SO}(n)$, $W_{1}:=\mathbb{R}^{2}\otimes\mathbb{R}^{n}$, and we define $W_{2}$ to be the representation space of $\rho_{2}$. In this way we can write

[TABLE]

where $W:=W_{1}\oplus W_{2}$. Here are some preliminary results:

Lemma 4.8**.**

Copolarity and abstract copolarity of $\rho$ are both $\leq 2$.

Proof.

The proof consists in computing the Luna-Richardson-Straume reduction of $\rho=\rho_{1}\oplus\rho_{2}$. It is easily seen that the principal isotropy group $K_{1}$ of $\rho_{1}$ is isomorphic to $\mathbb{Z}_{2}\times\text{\bf SO}(n-2)$, so the principal isotropy group $K$ of $\rho$ is isomorphic either to $\text{\bf SO}(n-2)$ or to $\mathbb{Z}_{2}\times\text{\bf SO}(n-2)$. In both cases $N_{H}(K)^{\circ}\simeq\mathbb{T}^{2}\times\text{\bf SO}(n-2)$ and $\left(\!\frac{N_{H}(K)}{K}\!\right)^{\!\circ}\simeq\mathbb{T}^{2}$, as claimed. ∎

Lemma 4.9**.**

$\rho$* is indecomposable.*

Proof.

Suppose this is not true; we may assume without loss of generality that $\rho$ is orbit-equivalent to

[TABLE]

where $W_{1}\oplus W_{2}=W^{\prime}\oplus W^{\prime\prime}$ and $W^{\prime}$, $W^{\prime\prime}$ are $G$-invariant with positive dimension. Since $W_{1}$ is an irreducible isotypical component of $\rho$, we may assume also that $W_{1}\subseteq W^{\prime}$, and that

[TABLE]

Here $\bar{W}_{1}$ is a sum of irreducible components of $\rho$ and is $H^{\prime}$-invariant, while $\bar{W}_{1}\oplus W^{\prime\prime}=W_{2}$ is a sum of isotypical components of $\rho$. Using that $\rho$ and $\tilde{\rho}$ have the same orbits, we easily conclude that $(H,W_{2})$ and the product representation $(H^{\prime}\times H^{\prime\prime},\bar{W}_{1}\oplus W^{\prime\prime})$ are orbit-equivalent; since principal orbits of $(H,W_{2})$ have dimension $1$, this forces one of the two representations $(H^{\prime},\bar{W}_{1})$, $(H^{\prime\prime},W^{\prime\prime})$ to be trivial. Since $\dim W^{\prime\prime}\geq 1$ and $(H,W_{2})$ has no non-trivial fixed points, this implies $\bar{W}_{1}=\{O\}$, and thus $(H^{\prime},W^{\prime})$, $(H^{\prime\prime},W^{\prime\prime})$ are orbit-equivalent to $\rho_{1}$, $\rho_{2}$ respectively. It follows that

[TABLE]

which is absurd since principal orbits of $((\text{\bf SO}(2)\times\text{\bf SO}(n))\times S^{1},(\mathbb{R}^{2}\otimes\mathbb{R}^{n})\oplus W_{2})$ and of $\rho$ don’t have the same dimension. ∎

Lemma 4.10**.**

We have $\text{\em chm}(\rho)\geq 6$.

Proof.

Clearly (cf. [6, p.92]) $\text{chm}(\rho)=2+\dim W_{2}$. Since $\rho_{2}$ is not polar and has no non-trivial fixed points, $\dim W_{2}\geq 4$ and we are done. ∎

We may finally prove the second part of Theorem 1.3:

Proof.

Using Lemmas 4.9 and 4.10 it is enough to prove that c$(\rho)=\text{ac}(\rho)=2$.

First we observe that $\rho$ is not polar (since otherwise both $\rho_{1}$ and $\rho_{2}$ would be polar, [3]), and its abstract copolarity cannot be $1$ (otherwise we would have $\text{chm}(\rho)=3$ by Theorem 4.1 and Lemma 4.9). Hence, the abstract copolarity of $\rho$ has to be at least $2$, which, together with Lemma 4.8, implies that the abstract copolarity (and of course the copolarity) of $\rho$ is exactly $2$. ∎