Algorithms for the Tits alternative and related problems

TL;DR

This paper introduces algorithms to determine key structural properties of finitely generated linear groups, such as solvability, nilpotency, and abelian characteristics, with implementations available in MAGMA.

Contribution

It provides the first computationally effective algorithms for the Tits alternative and related properties in finitely generated linear groups.

Findings

Algorithms successfully decide group properties like solvable-by-finite and nilpotent-by-finite.

Implementation in MAGMA is publicly available for practical use.

The algorithms advance computational group theory by making theoretical results algorithmically accessible.

Abstract

We present an algorithm that decides whether a finitely generated linear group over an infinite field is solvable-by-finite: a computationally effective version of the Tits alternative. We also give algorithms to decide whether the group is nilpotent-by-finite, abelian-by-finite, or central-by-finite. Our algorithms have been implemented in MAGMA and are publicly available.

Equations

$${\psi }_{2,p}:{\sum }_{i=0}^{k-1}{b}_i{\alpha }^i\mapsto {\sum }_{i=0}^{k-1}{\bar{b}}_i{\bar{\alpha }}^i$$

$${\psi }_{4,\alpha }:{\sum }_{i=0}^{e-1}{c}_i{\beta }^i\mapsto {\sum }_{i=0}^{e-1}{\tilde{c}}_i{\tilde{\beta }}^i$$

$$K^G=\{h_1^g,\dots ,h_k^g\mid g\in G\}.$$

$$X_d=\{x_d\mid x\in X\}\text{and}{X}_u=\{x_u\mid x\in X\}.$$

$$f_d:h\mapsto h_d,f_u:h\mapsto h_u$$

$$H_d=⟨f_d(K^G)⟩\text{and}{H}_u=⟨f_u(K^G)⟩.$$

$$H_d=⟨K_d^G⟩\text{and}{H}_u=⟨K_u^G⟩.$$

Full text

1. Introduction

The Tits alternative, established by Tits [27], states that a finitely generated linear group over a field either is solvable-by-finite, or it contains a non-cyclic free subgroup. This theorem partitions finitely generated linear groups into two very different classes, which require separate treatment. Consequently, one of the first questions that must be settled for such a group is to determine the class of the Tits alternative to which it belongs. In the class of groups with non-cyclic free subgroups, some basic computational problems are undecidable in general; whereas solvable-by-finite groups are more amenable to computation (see [16, Section 3]). For further discussion of the Tits alternative, and its influence on other areas of group theory, we refer to [18].

Algorithms to decide the Tits alternative over the rational field $\mathbb{Q}$ were proposed in [6, 7]. Drawing on results of [17], a different approach was considered in [23]. Another algorithm for the Tits alternative in $\mathrm{GL}(n,\mathbb{Q})$, as well as practical algorithms to test solvability and polycyclicity of rational matrix groups, appeared in [1, 2, 3]. We are not aware of implementations of these algorithms to decide the Tits alternative over $\mathbb{Q}$.

This paper gives a practical algorithm to decide whether a finitely generated linear group over an arbitrary field is solvable-by-finite. Additionally, we can test whether the group is solvable. Our method uses congruence homomorphism techniques (see [16, Section 4]), which were applied previously to special cases of the problems mentioned above; namely, deciding finiteness and nilpotency [11, 12, 13, 14]. We also rely on two other recent developments. The first is a description by Wehrfritz [29] of congruence subgroups of solvable-by-finite linear groups. The second is the development of effective algorithms to construct presentations of matrix groups over finite fields (see [4, 22]).

If the field is $\mathbb{Q}$, our algorithm to test virtual solvability is a refinement and extension of that in [1]. However, we consider finitely generated linear groups defined over an arbitrary field (albeit possibly with a finite number of exceptions in positive characteristic). We also solve the related problems of deciding whether a group defined over a field of characteristic zero is virtually nilpotent, virtually abelian, or central-by-finite. The resulting algorithms are practical, and implementations are publicly available in Magma [8].

We emphasize that this paper demonstrates that the various problems of testing virtual properties are decidable for finitely generated groups over a wide range of fields. Solvability testing was previously known to be decidable for groups over number fields [21].

Section 2 sets up the background theory for our congruence homomorphism techniques. In Section 3 we present an algorithm to decide virtual solvability. Section 4 deals with the special case where the group is completely reducible. In Section 5 we outline algorithms to decide whether a group in characteristic zero is nilpotent-by-finite, abelian-by-finite, or central-by-finite. Finally, we report on the Magma implementation of our algorithms.

2. Congruence homomorphisms and computing in solvable-by-finite groups

We start by fixing some notation. Let $G=\langle\hskip 0.56905ptS\hskip 0.7113pt\rangle\leq\mathrm{GL}(n,\mathbb{F})$, where $S=\{g_{1},\ldots,g_{r}\}$ and $\mathbb{F}$ is an infinite field. Denote the integral domain generated by the entries of the matrices in $S\cup S^{-1}$ by $R$. Recall that $R/\rho$ is a finite field if $\rho$ is a maximal ideal of $R$ [28, 4.1, p. 50]. Let $\rho$ be a (proper) ideal of a subring $\Delta$ of $\mathbb{F}$; then natural projection $\Delta\rightarrow\Delta/\rho$ extends to a group homomorphism $\mathrm{GL}(n,\Delta)\rightarrow\mathrm{GL}(n,\Delta/\rho)$ and a ring homomorphism $\mathrm{Mat}(n,\Delta)\rightarrow$ $\mathrm{Mat}(n,\Delta/\rho)$. We denote all these homomorphisms by $\psi_{\rho}$. The kernel of $\psi_{\rho}$ on $G$ is denoted $G_{\rho}$, and is called a congruence subgroup of $G$.

2.1. Congruence subgroups of solvable-by-finite groups

Each solvable-by-finite linear group has a triangularizable normal subgroup of finite index [26, Theorem 7, p. 135]; in particular, its Zariski connected component is unipotent-by-abelian. Proving that $G$ is solvable-by-finite is therefore equivalent to proving that $G$ has a unipotent-by-abelian normal subgroup of finite index. So to apply congruence homomorphism techniques to computing in the first class of the Tits alternative, we should first answer the following question: if $G$ is solvable-by-finite, for which ideals $\rho\subseteq R$ is $G_{\rho}$ unipotent-by-abelian? We summarize recent results of Wehrfritz [29, Theorems 1–3] that describe such ideals (as usual, $H^{\prime}$ is the commutator subgroup $[H,H]$ of a group $H$).

Theorem 2.1**.**

Suppose that $G\leq\mathrm{GL}(n,\Delta)$ is solvable-by-finite, where $\Delta$ is an integral domain.

• (i)

Let $\rho$ be an ideal of $\Delta$. If $\mathrm{char}\,\Delta=p>n$, or $\mathrm{char}\,\Delta=0$ and $\mathrm{char}(\Delta/\rho)=$ $p>n$, then $G^{\prime}_{\rho}$ is unipotent.

• (ii)

Suppose that $\Delta$ is a Dedekind domain of characteristic zero, and $\rho$ is a maximal ideal of $\Delta$. If $p\in\mathbb{Z}$ is an odd prime such that $p\in\rho\setminus\rho^{p-1}$, then $G_{\rho}$ is connected; hence $G^{\prime}_{\rho}$ is unipotent.

We call $\psi_{\rho}:\mathrm{GL}(n,\Delta)\rightarrow\mathrm{GL}(n,\Delta/\rho)$ a W-homomorphism if $\Delta/\rho$ is finite and $G_{\rho}^{\prime}$ is unipotent whenever $G\leq\mathrm{GL}(n,\Delta)$ is solvable-by-finite.

2.2. Construction of W-homomorphisms

We may assume that $\mathbb{F}$ is finitely generated over its prime subfield, and is the field of fractions of $R$. Then it suffices to let $\mathbb{F}$ be one of

• I.

the rationals $\mathbb{Q}$,

• II.

a number field,

• III.

a function field $\mathbb{P}(x_{1},\ldots,x_{m})$, or

• IV.

a finite extension of $\mathbb{P}(x_{1},\ldots,x_{m})$,

where $\mathbb{P}$ is a number field or finite field in III–IV. See [16, Section 4] for more details.

In each case I–IV we explain below how to construct W-homomorphisms on $\mathrm{GL}(n,R)$. Note that if $\mathbb{F}$ has positive characteristic at most $n$, then in general we cannot construct a W-homomorphism. For a subring $\Delta$ of a field, $\frac{1}{\mu}\Delta$ denotes the localization $\{x\mu^{-i}\mid x\in\Delta,i\geq 0\}$ of $\Delta$ at a non-zero element $\mu$.

2.2.1. The rational field

(Cf. [17, Lemma 9].) Let $\mathbb{F}=\mathbb{Q}$. Then $R=\frac{1}{\mu}\mathbb{Z}$ for some $\mu\in\mathbb{Z}\setminus\{0\}$ determined by the denominators of entries in the elements of $S\cup S^{-1}$. By Theorem 2.1 (ii), if $p\in\mathbb{Z}$ is an odd prime not dividing $\mu$, then reduction mod $p$ is a W-homomorphism from $\mathrm{GL}(n,R)$ onto $\mathrm{GL}(n,p)$. We denote this homomorphism by $\Psi_{1}=\Psi_{1,p}$.

2.2.2. Number fields

Let $\mathbb{F}=\mathbb{Q}(\alpha)$ where $\alpha$ is an algebraic integer. We may take $R=\frac{1}{\mu}\mathbb{Z[\alpha]}$, $\mu\in\mathbb{Z}\setminus\{0\}$. Let $f(t)=a_{0}+\cdots+a_{k-1}t^{k-1}+t^{k}\in\mathbb{Z}[t]$ be the minimal polynomial of $\alpha$. For a prime $p\in\mathbb{Z}$ not dividing $\mu$, define $\psi_{2,p}:R\rightarrow\mathbb{Z}_{p}(\bar{\alpha})$ by

[TABLE]

where $\bar{b}_{i}$ denotes the reduction of $b_{i}$ mod $p$, and $\bar{\alpha}$ is a root of $\bar{f}(t)=\bar{a}_{0}+\cdots+\bar{a}_{k-1}t^{k-1}+t^{k}$.

Lemma 2.2**.**

• (i)

Let $p\in\mathbb{Z}$ be an odd prime dividing neither $\mu$ nor the discriminant of $f(t)$. Then $\psi_{2,p}$ is a W-homomorphism.

• (ii)

Let $p\in\mathbb{Z}$ be a prime greater than $n$ not dividing $\mu$. Then $\psi_{2,p}$ is a W-homomorphism.

Proof.

(i) Let $\mathcal{O}$ be the ring of integers of $\mathbb{F}$. Select an irreducible factor $\bar{f}_{j}(t)$ of $\bar{f}(t)$, and let $f_{j}(t)$ be a pre-image of $\bar{f}_{j}(t)$ in $\mathbb{Z}[t]$. The ideal $\rho$ of $\frac{1}{\mu}\mathcal{O}$ generated by $p$ and $f_{j}(\alpha)$ is maximal, and $p\not\in\rho^{2}$ (see [20, Proposition 3.8.1, Theorem 3.8.2]). Since the kernel of $\psi_{2,p}$ on $\mathrm{GL}(n,R)$ is contained in the kernel of $\psi_{\rho}$ on $\mathrm{GL}(n,\frac{1}{\mu}\mathcal{O})$, Theorem 2.1 (ii) implies that $\psi_{2,p}$ is a W-homomorphism.

(ii) This part is immediate from Theorem 2.1 (i). ∎

For example, let $\mathbb{F}$ be the $c$th cyclotomic field; if $p$ is an odd prime not dividing $\mathrm{lcm}(\mu,c)$, then $\psi_{2,p}$ is a W-homomorphism.

We denote the W-homomorphism $\psi_{2,p}$ for $p$ as in Lemma 2.2 by $\Psi_{2}=\Psi_{2,p}$.

2.2.3. Function fields

Let $\mathbb{F}=\mathbb{P}(x_{1},\ldots,x_{m})$, so $R\subseteq\frac{1}{\mu}\mathbb{P}[x_{1},\ldots,x_{m}]$ for some $\mathbb{P}$-polynomial $\mu=\mu(x_{1},\ldots,x_{m})$. Suppose that $\alpha=(\alpha_{1},\ldots,\alpha_{m})$ is a non-root of $\mu$, where the $\alpha_{i}$ are in the algebraic closure $\overline{\mathbb{P}}$ of $\mathbb{P}$. Note that if $\mathbb{P}$ is infinite then $\alpha$ can always be chosen in $\mathbb{P}^{m}$. Define $\psi_{3,\alpha}$ to be the substitution homomorphism that replaces $x_{i}$ by $\alpha_{i}$, $1\leq i\leq m$.

Let $\mathrm{char}\,R=0$. Set $\Psi_{3}=\Psi_{3,\alpha,p}=\Psi_{i,p}\circ\psi_{3,\alpha}$, where $p>n$, $i=1$ if $\mathbb{P}=\mathbb{Q}$, and $i=2$ if $\mathbb{P}\neq\mathbb{Q}$ is a number field.

If $\mathrm{char}\,R=p>n$ then set $\Psi_{3}=\Psi_{3,\alpha}=\psi_{3,\alpha}$.

In all cases $\Psi_{3}$ is a W-homomorphism by Theorem 2.1 (i).

2.2.4. Algebraic function fields

Let $\mathbb{F}=\mathbb{L}(\beta)$ where $\mathbb{L}=\mathbb{P}(x_{1},\ldots,x_{m})$, $|\mathbb{F}/\mathbb{L}|=e$ and $\beta$ has minimal polynomial $f(t)=a_{0}+\cdots+a_{e-1}t^{e-1}+t^{e}$. Then $R\subseteq\frac{1}{\mu}\mathbb{L}_{0}[\beta]$ for some $\mu\in\mathbb{L}_{0}=$ $\mathbb{P}[x_{1},\ldots,x_{m}]$. We may assume that $f(t)\in\mathbb{L}_{0}[t]$.

Define $\psi_{4,\alpha}$ on $\mathrm{GL}(n,R)$ as follows. Let $\alpha\in\overline{\mathbb{P}}^{m}$, $\mu(\alpha)\neq 0$; and let $\tilde{\beta}$ be a root of $\tilde{f}(t)=$ $\tilde{a}_{0}+\cdots+\tilde{a}_{e-1}t^{e-1}+t^{e}$ where $\tilde{a}_{i}:=\psi_{3,\alpha}(a_{i})$. Each element of $R$ may be uniquely expressed as $\sum^{e-1}_{i=0}c_{i}\beta^{i}$ for some $c_{i}\in\frac{1}{\mu}\mathbb{L}_{0}$. Then

[TABLE]

where $\tilde{c}_{i}=\psi_{3,\alpha}(c_{i})$.

Suppose that $\mathrm{char}\,R=0$, so we can choose $\alpha\in\mathbb{P}^{m}$. Set $\Psi_{4}=\Psi_{4,\alpha,p}=\Psi_{i,p}\circ\psi_{4,\alpha}$ where $p>n$, $i=1$ if $\mathbb{P}=\mathbb{Q}$ and $\tilde{\beta}\in\mathbb{Q}$, and $i=2$ otherwise.

If $\mathrm{char}\,R=p>n$ then set $\Psi_{4}=\psi_{4,\alpha}$.

By Theorem 2.1 (i), $\Psi_{4}$ is a W-homomorphism.

Remark 2.3*.*

An SW-homomorphism on $\mathrm{GL}(n,R)$ is a congruence homomorphism with finite image such that every torsion element of its congruence subgroup is unipotent (see [28, 4.8, p. 56] and [16, Section 4]). This property of the congruence subgroup is crucial to the algorithms of [14] for finiteness testing and structural analysis of finite matrix groups over infinite fields. The W-homomorphisms $\Psi_{i}$ are SW-homomorphisms; moreover, this remains true for $\Psi_{3}$ and $\Psi_{4}$ without requiring that $p>n$.

3. Testing virtual solvability

3.1. Preliminaries

If $\psi_{\rho}$ is a W-homomorphism on $\mathrm{GL}(n,R)$, then $G$ is solvable-by-finite if and only if $G_{\rho}^{\prime}$ is unipotent. In this subsection we develop procedures to test whether a finitely generated subgroup of $\mathrm{GL}(n,R)$ is unipotent-by-abelian. Denote the $\mathbb{F}$-enveloping algebra of $M\subseteq\mathrm{Mat}(n,\mathbb{F})$ by $\langle\hskip 0.56905ptM\hskip 0.7113pt\rangle_{\mathbb{F}}$, and the $\mathbb{F}$-linear span of $M$ by $\mathrm{span}_{\mathbb{F}}(M)$.

Lemma 3.1**.**

Let $H\leq\mathrm{GL}(n,\mathbb{F})$ be unipotent-by-abelian. Then $gh-hg\in\mathrm{Rad}\langle\hskip 0.56905ptH\hskip 0.7113pt\rangle_{\mathbb{F}}$ for all $g,h\in H$.

Proof.

(Cf. [17, p. 256] and [1, Lemma 5].) Since $H^{\prime}$ is unipotent, $h_{1}=[g,h]-1_{n}$ is nilpotent. For every $a\in\langle\hskip 0.56905ptH\hskip 0.7113pt\rangle_{\mathbb{F}}$, the matrix $ah_{1}$ is nilpotent (as $H$ is triangularizable), and so $h_{1}\in\mathrm{Rad}\langle\hskip 0.56905ptH\hskip 0.7113pt\rangle_{\mathbb{F}}$. Thus $gh-hg=hgh_{1}\in\mathrm{Rad}\langle\hskip 0.56905ptH\hskip 0.7113pt\rangle_{\mathbb{F}}$. ∎

Lemma 3.2**.**

Let $H\unlhd G$ where $H$ is unipotent-by-abelian. If $x\in\mathrm{Rad}\langle\hskip 0.56905ptH\hskip 0.7113pt\rangle_{\mathbb{F}}$ then there is a non-zero $G$-module in the nullspace of $x$.

Proof.

The hypotheses on $H$ ensure that $x^{g}\in\mathrm{Rad}\langle\hskip 0.56905ptH\hskip 0.7113pt\rangle_{\mathbb{F}}$ for all $g\in G$. Thus, the nullspace of $\mathrm{Rad}\langle\hskip 0.56905ptH\hskip 0.7113pt\rangle_{\mathbb{F}}$ is a (non-zero) $G$-module in the nullspace of $x$. ∎

In [13, p. 4155] we describe a simple recursive procedure ${\tt ModuleViaNullSpace}(S,x)$ that finds, in no more than $n$ iterations, a $G$-module $U$ in the nullspace of $x\in\mathrm{Mat}(n,\mathbb{F})$ that contains every such $G$-module. Hence, if $x$ is as in Lemma 3.2 then $U$ is non-zero.

We now establish a convention. For a subset $K=\{h_{1},\ldots,h_{k}\}$ of $\mathrm{Mat}(n,\mathbb{F})$, define

[TABLE]

If $K\subseteq G$ then $\langle\hskip 0.56905ptK^{G}\hskip 0.7113pt\rangle$ is the normal closure of $\langle\hskip 0.56905ptK\hskip 0.7113pt\rangle$ in $G$, which is usually denoted $\langle\hskip 0.56905ptK\hskip 0.7113pt\rangle^{G}$.

We next state a procedure that will be needed in several places later.

${\tt BasisAlgebraClosure}(K,S)$

Input: finite subsets $K$ and $S=\{g_{1},\ldots,g_{r}\}$ of $\mathrm{GL}(n,\mathbb{F})$.

Output: A basis of the $\mathbb{F}$-enveloping algebra of $\langle\hskip 0.56905ptK^{G}\hskip 0.7113pt\rangle$, where $G=\langle\hskip 0.56905ptS\hskip 0.7113pt\rangle$.

1. (1)

$\mathcal{A}:=K\cup K^{-1}$. 2. (2)

While $\exists\,g\in S\cup S^{-1}$ and $A\in\mathcal{A}$ such that $g^{-1}Ag\notin\mathrm{span}_{\mathbb{F}}(\mathcal{A})$, do

$\mathcal{A}:=\mathcal{A}\cup\{g^{-1}Ag\}$. 3. (3)

‘Spin up’ to construct a basis $\mathcal{B}$ of the $\mathbb{F}$-enveloping algebra of $\langle\hskip 0.56905pt\mathcal{A}\hskip 0.7113pt\rangle$. 4. (4)

Return $\mathcal{B}$.

${\tt BasisAlgebraClosure}$ terminates in at most $n^{2}$ iterations. For a discussion of the well-known ‘spinning up’ method in step (3), see, e.g., [12, Section 3.1]. One feature of ${\tt BasisAlgebraClosure}$ is that the basis $\mathcal{B}$ returned consists of elements of $\langle\hskip 0.56905ptK^{G}\hskip 0.7113pt\rangle$.

Remark 3.3*.*

If $K\subseteq\mathrm{Mat}(n,\mathbb{F})$ contains non-invertible elements, then the obvious modifications should be made to ${\tt BasisAlgebraClosure}$. That is, $\mathcal{A}$ is initialized to $K$ in step (1); and in step (3) a basis of $\langle\hskip 0.56905pt\mathcal{A}\hskip 0.7113pt\rangle_{\mathbb{F}}$ is constructed (by the same spinning up as before). The output of this modified procedure, which we name ${\tt BasisAlgebraClosure}^{*}$, is a basis of $\langle\hskip 0.56905ptK^{G}\hskip 0.7113pt\rangle_{\mathbb{F}}$.

3.2. Testing virtual solvability

Let $U$ be a $H$-submodule of $V:=\mathbb{F}^{n}$, where $H\leq\mathrm{GL}(n,\mathbb{F})$. Extend a basis of $U$ to one of $V$, with respect to which $H$ has block triangular form. We denote the projection homomorphism of $H$ onto the corresponding block diagonal group in $\mathrm{GL}(n,\mathbb{F})$ by $\pi_{U}$. The kernel of $\pi_{U}$ is a unipotent normal subgroup of $H$.

${\tt NormalGenerators}$ is a procedure that accepts $S$ and a W-homomorphism $\Psi=\psi_{\rho}$ as input, and returns normal generators for $G_{\rho}$, i.e., generators for a subgroup whose normal closure in $G$ is $G_{\rho}$. This procedure first finds a presentation $\mathcal{P}$ of $\Psi(G)$ on the generating set $\Psi(g_{1}),\ldots,\Psi(g_{r})$. Such presentations can be computed using algorithms from [4, 22]. The relators in $\mathcal{P}$ are then evaluated by replacing each occurrence of $\Psi(g_{i})$ in each relator by $g_{i}$, $1\leq i\leq r$. The resulting words in the $g_{i}$ constitute the output of ${\tt NormalGenerators}$.

We also need the following recursive procedure.

${\tt ExploreBasis}(\mathcal{A},T)$

Input: finite subsets $\mathcal{A}$, $T$ of $\mathrm{GL}(m,\mathbb{F})$, where $\mathcal{A}\subseteq\langle\hskip 0.56905ptT\hskip 0.7113pt\rangle$.

Output: ${\tt true}$ or ${\tt false}$.

1. (1)

If $[A_{i},A_{j}]=1_{m}$ $\forall\,A_{i},A_{j}\in\mathcal{A}$ then return ${\tt true}$. 2. (2)

$U_{1}:={\tt ModuleViaNullSpace}(T,A_{i}A_{j}-A_{j}A_{i})$ where $[A_{i},A_{j}]\neq 1_{m}$.

If $U_{1}=\{0\}$ then return ${\tt false}$. 3. (3)

$\pi:=\pi_{U_{1}}$, $U_{2}:=V/U_{1}$. 4. (4)

For $\ell=1,2$ do

$\mathcal{A}_{\ell}:=\{\pi(A_{j})_{|U_{\ell}}\mid A_{j}\in\mathcal{A}\}$, $T_{\ell}:=\{\pi(h_{j})_{|U_{\ell}}\mid h_{j}\in T\}$;

if ${\tt ExploreBasis}(\mathcal{A}_{\ell},T_{\ell})={\tt false}$ then return ${\tt false}$. 5. (5)

Return ${\tt true}$.

Now we can assemble our algorithm to decide the Tits alternative.

${\tt IsSolvableByFinite}(S)$

Input: $S=\{g_{1},\ldots,g_{r}\}\subseteq\mathrm{GL}(n,R)$.

Output: ${\tt true}$ if $G=\langle\hskip 0.56905ptS\hskip 0.7113pt\rangle$ is solvable-by-finite and ${\tt false}$ otherwise.

1. (1)

$K:={\tt NormalGenerators}(S,\Psi)$, $\Psi$ a W-homomorphism on $\mathrm{GL}(n,R)$. 2. (2)

$\mathcal{A}:={\tt BasisAlgebraClosure}(K,S)$. 3. (3)

Return ${\tt ExploreBasis}(\mathcal{A},S)$.

Remark 3.4*.*

When $\mathbb{F}=\mathbb{Q}$, ${\tt IsSolvableByFinite}$ is similar to the algorithm of [1, p. 1280]—but see the first paragraph of [1, Section 10.1].

${\tt IsSolvableByFinite}$ terminates in no more than $n$ iterations at step (3). A report of ${\tt false}$ is correct by Lemmas 3.1 and 3.2. Note that if ${\tt true}$ is returned at the first pass through step (1) of ${\tt ExploreBasis}$, then $G$ is abelian-by-finite.

Algorithms to test solvability of matrix groups over finite fields are implemented in [3, 8]. We can augment ${\tt IsSolvableByFinite}$ by checking solvability of $\Psi(G)$ during step (1), and thus obtain a solvability testing algorithm for finitely generated subgroups of $\mathrm{GL}(n,\mathbb{F})$. Moreover, when $R=\mathbb{Z}$, these algorithms decide whether $G$ is polycyclic or polycyclic-by-finite (cf. [5, Theorem 4.2]).

We now point out some further additions to our basic method for deciding virtual solvability.

First suppose that $\mathrm{char}\,\mathbb{F}=0$. Sometimes we can quickly detect that $G$ is not solvable-by-finite, by means of the following observations. A classical theorem of Jordan states that there is a function $f:\mathbb{N}\rightarrow\mathbb{N}$ (independent of $\mathbb{F}$) such that if $G$ is a finite subgroup of $\mathrm{GL}(n,\mathbb{F})$, then $G$ has an abelian normal subgroup of index bounded by $f(n)$. It follows from [28, 10.11, p. 142] that if $G$ is solvable-by-finite, then the solvable radical of $\Psi(G)$ has index bounded by $f(n)$. To apply this criterion, we use an algorithm described in [19, Section 4.7.5] to compute the index of the solvable radical of a matrix group over a finite field, and then we compare this index with $f(n)$. Collins [9] has found the optimal function $f$ for all $n$. In particular, $f(n)=(n+1)!$ for $n\geq 71$.

Next, recall that if $\Psi=\psi_{\rho}$ is $\Psi_{3,\alpha,p}$ or $\Psi_{4,\alpha,p}$, then $p$ must be greater than $n$ by definition. However, with extra restrictions in place, it is possible to test virtual solvability in characteristic $p\leq n$ too. Suppose that $\rho$ is a proper ideal of $R$ such that either (i) $\mathrm{char}\,R=0$, $\mathrm{char}(R/\rho)>0$ and $G_{\rho}$ is generated by unipotent elements; or (ii) $\mathrm{char}\,R>0$ and $G_{\rho}$ is generated by diagonalizable elements. Then $G$ is solvable-by-finite if and only if $G_{\rho}^{\prime}$ is unipotent: this follows from the last paragraph of [29, Section 1], and [29, Theorem 1 (d)]. We can determine whether (i) or (ii) holds by checking whether each normal generator of $G_{\rho}$ is unipotent or diagonalizable.

4. Completely reducible groups

Some of our problems coincide in an important special case.

Lemma 4.1**.**

Suppose that $G\leq\mathrm{GL}(n,\mathbb{F})$ is completely reducible, where $\mathbb{F}$ is any field. Then the following are equivalent:

• (i)

$G$* is solvable-by-finite;*

• (ii)

$G$* is nilpotent-by-finite;*

• (iii)

$G$* is abelian-by-finite.*

Proof.

Trivially (iii) $\Rightarrow$ (ii) $\Rightarrow$ (i). If $G$ is solvable-by-finite, then a normal unipotent-by-abelian subgroup of $G$ must be abelian, because a completely reducible unipotent group is trivial. Thus (i) implies (iii). ∎

Motivated by Lemma 4.1, we consider how to decide whether a solvable-by-finite group $G$ is completely reducible. Let $\psi_{\rho}$ be a W-homomorphism on $\mathrm{GL}(n,R)$. If $G_{\rho}$ is completely reducible (hence abelian) and $\mathrm{char}\,R$ does not divide $|G:G_{\rho}|$, then $G$ is completely reducible by [26, Theorem 1, p. 122]. Therefore, in characteristic zero, $G$ is completely reducible if and only if the elements of ${\tt BasisAlgebraClosure}(K,S)$ commute pairwise and are all diagonalizable, where $K={\tt NormalGenerators}(S,\psi_{\rho})$. If $\mathrm{char}\,R=p>0$ divides $|G:G_{\rho}|$, then we cannot decide complete reducibility of $G$; otherwise we apply the characteristic zero criterion.

A finitely generated solvable linear group may not be finitely presentable [28, 4.22, p. 66]. However, if $G$ is both solvable-by-finite and completely reducible, then $G_{\rho}$ is a finitely generated abelian normal subgroup of finite index. So we can compute presentations of $G_{\rho}$ and $\psi_{\rho}(G)$, and combine them as explained in [1, 4], to obtain a finite presentation of $G$.

5. Testing virtual nilpotency and related algorithms

We now consider the problems of deciding whether a finitely generated linear group is nilpotent-by-finite, abelian-by-finite, or central-by-finite. Algorithms for nilpotency testing and computing with finitely generated nilpotent groups over arbitrary fields are given in [10, 11].

Henceforth $\mathrm{char}\,\mathbb{F}=0$ unless stated otherwise.

5.1. Preliminaries

Lemma 5.1**.**

Let $H\leq\mathrm{GL}(n,\mathbb{F})$ be nilpotent-by-finite (resp. abelian-by-finite), $\mathbb{F}$ any field. If $H$ is connected then $H$ is nilpotent (resp. abelian).

Proof.

(Cf. [17, Lemma 9].) Let $N\leq H$ be nilpotent (resp. abelian) of finite index. Then the Zariski closure of $N$ in $H$ is nilpotent (resp. abelian) and contains the connected component of $H$; see [28, Chapter 5]. The lemma follows. ∎

Corollary 5.2**.**

Suppose that $R$ is a Dedekind domain of characteristic zero, and $\rho$ is a maximal ideal of $R$ such that $\mathrm{char}(R/\rho)=p>2$, where $p\notin\rho^{p-1}$. Then $G\leq\mathrm{GL}(n,R)$ is nilpotent-by-finite (resp. abelian-by-finite) if and only if $G_{\rho}$ is nilpotent (resp. abelian).

Proof.

This follows from Theorem 2.1 (ii) and Lemma 5.1. ∎

Denote by $g_{d},g_{u}\in\mathrm{GL}(n,\mathbb{F})$ the diagonalizable and unipotent parts of $g\in\mathrm{GL}(n,\mathbb{F})$, i.e., $g=g_{d}g_{u}=g_{u}g_{d}$ is the Jordan decomposition of $g$. For $X\subseteq\mathrm{GL}(n,\mathbb{F})$ we put

[TABLE]

Proposition 5.3**.**

Let $H=\langle\hskip 0.56905ptK^{G}\hskip 0.7113pt\rangle$, where $K$ is a finite subset of $G$. Then $H$ is nilpotent and $H^{\prime}$ is unipotent if and only if $\langle\hskip 0.56905ptK_{d}^{G}\hskip 0.7113pt\rangle$ is abelian, $\langle\hskip 0.56905ptK_{u}^{G}\hskip 0.7113pt\rangle$ is unipotent, and $[K_{d}^{G},K_{u}^{G}]=\{1_{n}\}$.

Proof.

If $\langle\hskip 0.56905ptK_{d}^{G}\hskip 0.7113pt\rangle$ is abelian, $\langle\hskip 0.56905ptK_{u}^{G}\hskip 0.7113pt\rangle$ is unipotent, and these groups centralize each other, then the group $L$ that they generate is unipotent-by-abelian and nilpotent. Hence the same is true for $H\leq L$.

Now suppose that $H$ is unipotent-by-abelian and nilpotent. Then $f_{d}:H\rightarrow H_{d}$, $f_{u}:H\rightarrow H_{u}$ defined by

[TABLE]

are homomorphisms by [25, Proposition 3, p. 136]. Thus

[TABLE]

Now $h^{g}=h_{d}^{g}h_{u}^{g}$ and $h_{d}^{g}$, $h_{u}^{g}$ are diagonalizable, unipotent respectively. Uniqueness of the Jordan decomposition implies that $h_{d}^{g}=(h^{g})_{d}$ and $h_{u}^{g}=(h^{g})_{u}$, so

[TABLE]

Thus $\langle\hskip 0.56905ptK_{u}^{G}\hskip 0.7113pt\rangle$ is unipotent. Since $H$ is nilpotent, $[K_{d}^{G},K_{u}^{G}]=\{1_{n}\}$ (see [25, Proposition 3, p. 136] again). Finally, since $\langle\hskip 0.56905ptK_{d}^{G}\hskip 0.7113pt\rangle=H_{d}$ is unipotent-by-abelian and completely reducible, it must be abelian. ∎

5.2. Nilpotent-by-finite and abelian-by-finite groups

Our algorithms for deciding whether $G$ is nilpotent-by-finite or abelian-by-finite require that $G$ be defined over a Dedekind domain $R$. Hence they apply, for example, when $\mathbb{F}$ is $\mathbb{Q}$, a number field, or (a finite extension of) a univariate function field.

Lemma 5.4**.**

Let $K\subseteq\mathrm{GL}(n,\mathbb{F})$, and $\widetilde{K}:=\{h-1_{n}\mid h\in K\cup K^{-1}\}$. Then $H=\langle\hskip 0.56905ptK\hskip 0.7113pt\rangle$ is unipotent if and only if $\langle\hskip 0.56905pt\widetilde{K}\hskip 0.7113pt\rangle_{\mathbb{F}}$ is nilpotent.

Proof.

Observe that $\langle\hskip 0.56905pt\widetilde{K}\hskip 0.7113pt\rangle_{\mathbb{F}}=\mathrm{span}_{\mathbb{F}}(\{h-1_{n}\mid h\in H\})$. Therefore, if $H$ is unipotent then $H^{x}$ is unitriangular for some $x\in\mathrm{GL}(n,\mathbb{F})$, so $\langle\hskip 0.56905pt\widetilde{K}\hskip 0.7113pt\rangle_{\mathbb{F}}$ is nilpotent. Conversely, if $\langle\hskip 0.56905pt\widetilde{K}\hskip 0.7113pt\rangle_{\mathbb{F}}$ is nilpotent then $h-1_{n}$ is nilpotent for all $h\in H$, i.e., $H$ is unipotent. ∎

Let $K$ be a finite subset of $\mathrm{GL}(n,\mathbb{F})$. The procedure ${\tt IsAbelianClosure}$ determines whether $\langle\hskip 0.56905ptK^{G}\hskip 0.7113pt\rangle$ is abelian by testing whether the elements of ${\tt BasisAlgebraClosure}(K,S)$ commute pairwise. Another auxiliary procedure is the following (recall Remark 3.3).

${\tt IsUnipotentClosure}(K,S)$

Input: finite subsets $K=\{h_{1}\ldots,h_{k}\}$ and $S$ of $\mathrm{GL}(n,\mathbb{F})$, where the $h_{i}$ are unipotent.

Output: ${\tt true}$ if $\langle\hskip 0.56905ptK^{G}\hskip 0.7113pt\rangle$ is unipotent, ${\tt false}$ otherwise, where $G=\langle\hskip 0.56905ptS\hskip 0.7113pt\rangle$.

1. (1)

$\widetilde{K}:=\{h_{j}-1_{n}\mid 1\leq j\leq k\}$. 2. (2)

$\mathcal{B}:={\tt BasisAlgebraClosure}^{*}(\widetilde{K},S)$. 3. (3)

If $|\mathcal{B}|>n(n-1)/2$, or $B$ is not nilpotent for some $B\in\mathcal{B}$ (i.e., $B^{n}\neq 0_{n}$), then return ${\tt false}$. 4. (4)

If $\langle\hskip 0.56905ptB+1_{n}:B\in\mathcal{B}\hskip 0.7113pt\rangle$ is unipotent then return ${\tt true}$; else return ${\tt false}$.

Remark 5.5*.*

Lemma 5.4 guarantees correctness of ${\tt IsUnipotentClosure}$. See [10, Section 2.1] for a procedure to test whether a finitely generated linear group is unipotent.

Let $\Psi$ be a W-homomorphism as in Corollary 5.2. By Proposition 5.3, we have the following algorithm to test virtual nilpotency.

${\tt IsNilpotentByFinite}(S)$

Input: a finite subset $S$ of $\mathrm{GL}(n,R)$, $R$ a Dedekind domain of characteristic zero.

Output: ${\tt true}$ if $G=\langle\hskip 0.56905ptS\hskip 0.7113pt\rangle$ is nilpotent-by-finite, and ${\tt false}$ otherwise.

1. (1)

$K:=\{h_{1},\ldots,h_{k}\}={\tt NormalGenerators}(S,\Psi)$. 2. (2)

$K_{d}:=\{(h_{i})_{d}\mid 1\leq i\leq k\}$, $K_{u}:=\{(h_{i})_{u}\mid 1\leq i\leq k\}$. 3. (3)

If not ${\tt IsUnipotentClosure}(K_{u},S)$ or not ${\tt IsAbelianClosure}(K_{d},S)$

or $[K_{d}^{G},K_{u}^{G}]\neq\{1_{n}\}$ then return ${\tt false}$; else return ${\tt true}$.

Remark 5.6*.*

In step (3) we use the fact that $[K_{d}^{G},K_{u}^{G}]=\{1_{n}\}$ if and only if the elements of ${\tt BasisAlgebraClosure}(K_{d},S)$ commute with the elements of ${\tt BasisAlgebraClosure}(K_{u},S)$ (these two bases are already computed in this step).

Similarly, for Dedekind domains $R$ of characteristic zero, the algorithm ${\tt IsAbelianByFinite}(S)$ decides whether $G$ is abelian-by-finite: it returns ${\tt IsAbelianClosure}(K,S)$, where as usual $K$ is ${\tt NormalGenerators}(S,\Psi)$.

If either of ${\tt IsNilpotentByFinite}(S)$ or ${\tt IsAbelianByFinite}(S)$ returns ${\tt true}$, then we can decide complete reducibility of $G$: now $G$ is completely reducible if and only if $K_{u}=\{1_{n}\}$.

5.3. Central-by-finite groups

In this subsection, instead of a W-homomorphism we may use more generally an SW-homomorphism (see Remark 2.3).

Lemma 5.7**.**

Let $H$ be a group such that $H^{\prime}$ is finite. If $A$ is a torsion-free normal subgroup of $H$, then $A$ is central.

Proof.

Since $[A,H]\leq A\cap H^{\prime}=\{1\}$, this is clear. ∎

Corollary 5.8**.**

Let $\mathbb{F}$ be any field of characteristic zero, and let $\Psi=\psi_{\rho}$ be an SW-homomorphism on $\mathrm{GL}(n,R)$. Then $G\leq\mathrm{GL}(n,\mathbb{F})$ is central-by-finite if and only if $G_{\rho}$ is central.

Proof.

If $G$ is central-by-finite then $G^{\prime}$ is finite by a result of Schur [24, 10.1.4, p. 287]. Since $G_{\rho}$ is torsion-free, it is central by Lemma 5.7. The other direction is trivial because $|G:G_{\rho}|$ is finite. ∎

Corollary 5.8 underpins a simple procedure ${\tt IsCentralByFinite}(S)$ which returns ${\tt true}$ if $[K,S]=\{1_{n}\}$, where $G_{\rho}=\langle\hskip 0.56905ptK^{G}\hskip 0.7113pt\rangle$; else it returns ${\tt false}$. Here $\mathbb{F}$ is any field of characteristic zero. The same procedure works for the fields $\mathbb{F}$ of positive characteristic in Sections 2.2.3–2.2.4, provided that $\Psi$ is a W-homomorphism as defined there and $G_{\rho}$ is completely reducible (hence torsion-free).

We could also decide whether $G$ is central-by-finite by checking whether the ‘adjoint’ representation that arises from the conjugation action of $G$ on $\langle\hskip 0.56905ptG\hskip 0.7113pt\rangle_{\mathbb{F}}$ has finite image (using, e.g., the algorithms of [14]), as suggested in [7]. While this approach is valid for all fields $\mathbb{F}$, it may involve computing with matrices of dimension $n^{2}$.

6. Implementation and performance

We have implemented our algorithms as part of the Magma package Infinite [15]. We use the CompositionTree package [4, 22] to study congruence images and construct their presentations.

In practice, the single most expensive task is evaluating relators to obtain normal generators for the kernel of a W-homomorphism.

We describe below sample outputs covering the main domains and types of groups. The experiments were performed using Magma V2.17-2 on a 2GHz machine. The examples are randomly conjugated so that generators are not sparse, and matrix entries are typically large. All (algebraic) function fields $\mathbb{F}$ in these examples are univariate, and if they have zero characteristic are over $\mathbb{Q}$. Since random selection plays a role in some of the algorithms, times have been averaged over three runs. The complete examples are available in the Infinite package.

1. (1)

$G_{1}\leq\mathrm{GL}(7,\mathbb{F})$ where $\mathbb{F}$ is a function field of characteristic zero. It is conjugate to an infinite monomial subgroup of $\mathrm{GL}(7,\mathbb{Q})$. We decide that this $4$-generator group is abelian-by-finite in $82$s. 2. (2)

$G_{2}\leq\mathrm{GL}(40,\mathbb{F})$ where $\mathbb{F}$ is an algebraic function field of characteristic zero. It is conjugate to an infinite completely reducible nilpotent subgroup of $\mathrm{GL}(40,\mathbb{Q})$. We decide that this $4$-generator group is central-by-finite in $30$s. 3. (3)

$G_{3}\leq\mathrm{GL}(56,\mathbb{F})$ where $\mathbb{F}$ is an algebraic function field of characteristic zero. It is conjugate to the Kronecker product of an infinite reducible nilpotent subgroup of $\mathrm{GL}(8,\mathbb{Q})$ with a primitive complex reflection group from the Shephard-Todd list. We decide that this $7$-generator group is nilpotent-by-finite in $219$s. 4. (4)

$G_{4}\leq\mathrm{GL}(18,\mathbb{F})$ where $\mathbb{F}$ is a function field over $\mathrm{GF}(19)$. It is conjugate to the Kronecker product of a solvable subgroup of $\mathrm{GL}(6,19)$ with an infinite triangular subgroup of $\mathrm{GL}(3,\mathbb{F})$. We decide that this $13$-generator group is solvable in $80$s. 5. (5)

$G_{5}\leq\mathrm{GL}(32,\mathbb{F})$ where $\mathbb{F}$ is the fifth cyclotomic field. It is conjugate to the Kronecker product of an infinite solvable subgroup of $\mathrm{GL}(8,\mathbb{Q})$ from [3] with a primitive complex reflection group from the Shephard-Todd list. We decide that this $8$-generator group is solvable-by-finite in $90$s. 6. (6)

$G_{6}\leq\mathrm{GL}(12,\mathbb{F})$ where $\mathbb{F}$ is a function field of characteristic zero. It is conjugate to $\mathrm{SL}(12,\mathbb{Z})$. We decide that this $3$-generator group is not solvable-by-finite in $10$s. 7. (7)

$G_{7}\leq\mathrm{GL}(32,\mathbb{F})$ where $\mathbb{F}$ is a number field of degree $4$ over $\mathbb{Q}$. It is conjugate to the Kronecker product of \big{\langle}\tiny\left(\begin{matrix}1&1\\ 0&1\end{matrix}\right),\left(\begin{matrix}1&0\\ 2&1\end{matrix}\right)\big{\rangle} with an infinite reducible nilpotent rational matrix group. We decide that this $4$-generator group is not solvable-by-finite in $56$s.

Acknowledgment

We are very much indebted to Professor B. A. F. Wehrfritz, who kindly provided us with his new results [29] on congruence subgroups of solvable-by-finite linear groups.