Incorporating Weisfeiler-Leman into algorithms for group isomorphism
Peter A. Brooksbank, Joshua A. Grochow, Yinan Li, Youming Qiao, James, B. Wilson
TL;DR
This paper introduces a novel algorithm for general finite group isomorphism testing by integrating algebraic group techniques with combinatorial Weisfeiler-Leman methods on hypergraphs, enhancing the efficiency and flexibility of GpI algorithms.
Contribution
It combines multiple advanced GpI algorithms with Weisfeiler-Leman hypergraph techniques, creating a unified, adaptable framework for testing finite group isomorphism.
Findings
Framework effectively encodes low-genus quotients as hypergraphs.
The approach is flexible to include additional invariants.
Combines algebraic and combinatorial methods for improved GpI.
Abstract
In this paper we combine many of the standard and more recent algebraic techniques for testing isomorphism of finite groups (GpI) with combinatorial techniques that have typically been applied to Graph Isomorphism. In particular, we show how to combine several state-of-the-art GpI algorithms for specific group classes into an algorithm for general GpI, namely: composition series isomorphism (Rosenbaum-Wagner, Theoret. Comp. Sci., 2015; Luks, 2015), recursively-refineable filters (Wilson, J. Group Theory, 2013), and low-genus GpI (Brooksbank-Maglione-Wilson, J. Algebra, 2017). Recursively-refineable filters -- a generalization of subgroup series -- form the skeleton of this framework, and we refine our filter by building a hypergraph encoding low-genus quotients, to which we then apply a hypergraph variant of the k-dimensional Weisfeiler-Leman technique. Our technique is flexible enough…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
1 Introduction
The problem of deciding whether two finite groups are isomorphic () has a century-old history that straddles several fields, including topology, computational algebra, and computer science. It also has several unusual variations in complexity. For example, the dense input model—where groups are specified by their multiplication “Cayley” tables, has quasi-polynomial time complexity and it reduces to the better known Graph Isomorphism problem (); cf. [Luks93]*Section 10. Meanwhile, for a sparse input model for groups, such as by permutations, matrices, or black-box groups, reduces from ; cf. \citelist[HL][Lip]. At present sparse is in and it is not known to lie in either nor ; see [BS84]*Propostion 4.8, Corollary 4.9. In fact in the model of groups input by generators and relations, Adian and Rabin famously showed is undecideable [Adi57, Rab58].
Following L. Babai’s breakthrough proof that is in quasi-polynomial time [Bab16], dense (Cayley table) is now an essential bottleneck to improving graph isomorphism. So while the methods we explore here can be applied in both the dense and the sparse models of , we concentrate our complexity claims on the dense case. In particular, when we say polynomial-time, we mean polynomial time in the group order unless specified otherwise. Our contribution here is to expose from within the structure of groups, graph theoretic properties which relate to the difficulty of solving . We expect this to facilitate the systematic use of combinatorial isomorphism techniques within that interplay with existing algebraic strategies.
We introduce a colored (hyper-)graph based on an algebraic data structure known as a recursively-refineable filter which identifies abelian groups and vector spaces layered together to form the structure of a finite group. Filters have been useful in several isomorphism tests \citelist[Wilson:alpha][Maglione:filters][Maglione:filters2][BOW]. As the name suggests, filters can be refined, and with each refinement the cost of isomorphism testing decreases after refinement to a function in , for some . The more rounds of refinement we can carry out the lower the cost of isomorphism. Existing uses of filter refinement find characteristic structure algebraically; our principal innovation is to add a combinatorial perspective to refinement. We color (co)dimension- subspaces of the layers of the filter using local isomorphism invariants. This parameter will be referred to as the genus parameter. The layers are in turn connected to each other according to their position within the group, and this presents further opportunities for coloring. With so much nuanced local information, the graph we associate to a group is well suited to individualization and refinement techniques like the dimension- Weisfeiler–Leman procedure [WL68, Babai79, ImmermanLander, CFI]. The critical work is to refine these graphs compatibly with the refinement of the filter (Theorem A). Thus, one maintains the relationship between the group and graph isomorphism properties as we recursively refine. While our methods do not apply to structures as general as semigroups and quaisgroups, they can be adapted to other problems, such as ring isomorphism [KS06].
To explore the implications of this technique we introduce a model for random finite groups. In doing so we consider pitfalls identified in previously suggested models for random finite groups. We are especially concerned with coverage—the idea that we are able to easily sample from groups within natural classes such as non-solvable, solvable, nilpotent, and abelian—and that within each subclass the number of isomorphism types is dense on a log scale (Theorem C). Log-scale is for now the best granularity we know for the enumeration of groups, cf. [BNV:enum]. We then prove (Theorem D) that in our random model, genus-1 1-WL-refinement on average refines to a series of length , which thereby achieves the expected refinement length posed in [Wilson:alpha]*p. 876. Following the refinement, the average width of the filter is thus constant, though the cost of refinement increases. (If the maximum width were constant it would result in a polynomial-time average-case isomorphism test in our model.)
Finally within our random model there are several “base cases” where the recursive refinements become less likely, or where our analysis is inadequate. We demonstrate that in two of these cases, isomorphism can be solved either in polynomial time in the average-case sense (Theorem E) or in nearly polynomial time () in the worst-case sense (Theorem F). The former also solves a related problem of average complexity of tensor equivalence.
Our strategy harnesses critical features of a great variety of existing approaches to isomorphism (code equivalence, filter refinements, adjoint-tensor methods, bidirectional collision) and uses Weisefeiler–Leman refinement as the top-level strategy to combine the various implications. That diversity was not so much a plan but the result of hitting barriers and looking to the literature for solutions. The result, however, is a framework that is rather flexible and is well suited to accommodate future ideas, both algebraic and combinatorial, as featured here already. That strength of course comes at a cost that the mechanics and analysis are rather involved. We expect that in time better analysis and simplified models will improve our understanding.
1.1 The context of this work
Much recent progress in has been had by considering special classes of groups; the recent papers \citelist[BMW][GQ][LGR] survey and supplement these results. That has created powerful but highly varied strategies with no obvious means of synthesis. Within our refinement model of computing we have the opportunity to begin merging some of the many options that have been developed to date. To help explain our approach we consider examples of groups of invertible matrices over finite fields of prime order, as graphically communicated in Figure 1.1. In fact, these examples will later evolve into the aforementioned random model for finite groups.
First thread: connection with linear and multilinear algebras.
Algorithms and data structures for linear and multilinear structures are on the whole far more evolved than counterparts for groups. This explains why progress for groups can be made by mapping problems into the realm of linear and multilinear algebra. Such a correspondence has been known for close to a century, originating in work of Brahana [Bra35] and Baer [Bae]. Consider groups of the following form.
[TABLE]
Figure 1.1(a) illustrates a possible . In the creation of our random model we shall sample groups by selecting random matrices in . A surprising necessity is that we sample only sparse matrices. Although this might seem counter to the goals of seeding a group with lots of entropy, we will demonstrate that groups with too much random seeding become virtually identical (Theorem 5.2).
As a general remark it will be helpful throughout this work to regard all groups as having been enriched by the addition of a second binary operation known as commutation. In this way groups behave much more like rings than they do like semigroups or quasigroups. In particular, very nearly distributes over the usual binary operation in , in that . That explains the link to multilinear algebra. In the case of :
[TABLE]
Stripping away the addition leaves us to compare bilinear (and later multilinear) products such as , under base changes. We treat these as functions , where indicates the function is multilinear. Equivalently, we must study the orbits of groups acting on elements of the tensor space . Such reductions of group isomorphism to multilinear equivalence, and more general tensor equivalence problems have been the key to the recent progress on some of the largest and most difficult instances of \citelist [BMW][BW:isom][LQ][IQ] [LW12][BOW][Wilson:profile]. The strategies buried within those methods are nevertheless quite distinct. For example, several focus on -algebras and properties of rings and modules acting on tensors. Others focus on tensors as high-dimensional arrays, and perform individualization and refinement techniques on slices of this data structure. Our model of refinement allows for both strategies.
Second thread: relationship to code equivalence.
Now consider the types of groups we could place on the block diagonal of the matrix group examples in Figure 1.1. These could include groups like . We could also use subgroups such as , as well as natural families of geometrically interesting groups such as orthogonal, unitary, and symplectic groups. We may even embed the same group several times into multiple blocks on the diagonal, e.g. . Those blocks could further be permuted producing groups of block monomial matrices such as . We can capture the spirit of such a group graphically in Figure 1.1(b). Indeed, our random group model builds random semi-simple and quasi-semisimple groups in just this way.
Isomorphism in the context of groups of this kind has been approached mostly through the use of code equivalence. For example, for semisimple groups—those with no non-trivial abelian normal subgroups—there is an algorithm that runs in time polynomial in the group order [BCGQ, BCQ], as well as an algorithm that is efficient in practice [CH03]. The key algorithmic idea is dynamic programming, and its use follows the one by Luks in the context of hypergraph isomorphism and code equivalence [Luks99]. Later [GQ] considers the further implications when the groups centrally extend abelian groups similar to the general family we have described in this thread.
Third thread: composition series and filters.
In recent years there has been some progress on improving general isomorphism using subgroup chains. Rosenbaum and Wagner [RosenbaumWagner] demonstrated that one can fix a composition series for group and then, given a composition series for another group , efficiently decide if there is an isomorphism sending to . Luks gave an improvement of that test [LuksCompSeries]. In this way, the putative cost of steps to decide isomorphism by brute-force is reduced to the number of possible composition series, which is at most .
Filters can use characteristic subgroups to recursively find more characteristic subgroups, ultimately producing a large enough collection of fixed subgroups that an isomorphism test along the lines of Rosenbaum–Wagner becomes efficient. For several families of groups such refinements have been discovered \citelist[Wilson:alpha][Maglione:filters-U]. Our approach here extends the filtration process by taking the methods known and combining them into a colored hypergraph where individualization-refinement techniques can be applied. The goal is to make it even more likely to reach a situation in which the Rosenbaum–Wagner and Luks algorithms can be applied efficiently.
1.2 An outline of the Weisfeiler–Leman procedure for groups
Our approach to uses recursively-refineable filters to build and refine a colored hypergraph within, and between, abelian layers of a given group. A filter on a group assigns to a -tuple of natural numbers (including [math]) a normal subgroup of subject to natural compatibility requirements. Let denote the set of normal subgroups of , and for let .
Definition 1.1** (Filter [Wilson:alpha]).**
A filter on a group is a map , where
[TABLE]
Note that the first condition implies that the subgroups form a descending chain of subgroups, though in general it is not a proper chain. Computationally we only store the lexicographically least label for each distinct subgroup in the image of . Thus, a filter’s image is bounded by the length of the longest subgroup chain. For a group of order this is at most .
We begin with a filter known from the structure of general finite groups, and then refine by increasing the value of . That refinements exist is proved in [Wilson:alpha] and that they can be computed efficiently is shown in [Maglione:filters]. Our initial value for will be the number of distinct primes dividing . For each prime , we let denote the intersection of all Sylow -subgroups of , the maximum normal subgroup having order a power of . Let , sorted lexicographically (so that ), and define as follows:
[TABLE]
Here the product means the normal subgroup generated by the terms . For example the group of permutations on letters would have
[TABLE]
The boundary filter is defined by (if , then ), and the quotients are the layers of . Note that for each , is abelian, and in fact a -module. In the selected filter above these are in fact -vector spaces for some . The set , with homogeneous bilinear products
[TABLE]
is a graded Lie algebra whose graded components are invariant under [Wilson:alpha]*Theorem 3.1.
A bilinear map (bimap) is said to have genus if it is defined over a field such that , or (see [BMW] for details) if it is built from such maps by certain elementary products (such as direct products, but even “central” products are allowed). We will primarily be concerned with the case where and we consider bimaps whose codomain has dimension at most , but our results extend without difficulty to the more general notion of genus.
In our setting the layers of are elementary abelian, and our approach is to build a hypergraph whose vertices are the union of the points (1-spaces) in the projective geometries of the layers. For , let denote the set of -dimensional subspaces of . Define a family of hypergraphs , where is a parameter, with vertices and hyperedges defined as follows:
- The vertex set of is .
- The hyperedge set of is , where is a hypergraph with edges and 3-edges on .
Having defined the hypergraph , we shall apply the -dimensional Weisfeiler-Leman (WL) procedure to it in an appropriate way. Briefly, it is a hypergraph version of the WL procedure [WL68] on graphs [Babai79, ImmermanLander]. When , such a WL procedure on hypergraphs was recently studied by Böker [Boker].
To this end we obtain an algorithm that, given a finite group and integers , computes a suitable characteristic filter , where is the Fitting subgroup, and an associated hypergraph . Further, it colors the hyperedges of in a certain desirable way. If is a coloring of hyperedges, denote the corresponding colored hypergraph by .
Theorem A**.**
There is a deterministic algorithm that, given a finite group and integers , constructs the Fitting subgroup , a characteristic filter whose non-zero layers are elementary abelian -modules, the hypergraph , and a coloring satisfying:
- (i)
* is hereditary in the following sense: for each , the vertex-and-edge-colored hypergraph obtained by restricting to is a refinement of the colored hypergraph for based on the filter truncated at .* 2. (ii)
* is also hereditary in in the following sense: the underlying hypergraphs of and are identical, and the coloring of the latter refines the coloring of the former.* 3. (iii)
If , there is a colored hypergraph isomorphism such that
[TABLE]
The time complexity is .
The algorithm to construct the colored hypergraph is an iterative procedure that we describe in detail in Section 3. Within a fixed iteration, we apply a Weisfeiler–Leman type individualization procedure to obtain a stable coloring (a hypergraph analogue of -dimensional WL). We then use that stable coloring to search for characteristic structure in not already captured by the filter . If we succeed, we use this structure to refine and iterate.
Given the result of our WL-algorithm and applying Luks’s extension [LuksCompSeries] of the Rosenbaum–Wagner composition series comparison [RosenbaumWagner], whenever we refine we improve our isomorphism test, resulting in:
Theorem B**.**
Let and denote the filter and colored hypergraph from Theorem A. Let denote the maximum dimension of any layer of , and let be the product over all layers of , where is the smallest color class in layer . Then given a nilpotent group of order , isomorphism can be tested in time
[TABLE]
We extend this with an individualize-and-refine technique in Section 4.3, though for that we do not have as cleanly stated an upper bound.
Remark 1.3*.*
The initial filter described above can be extended to solvable groups, and in particular the solvable radical of any group, by doing something similar to the above within each layer of the Fitting series. This would let us extend all our results from using the Fitting subgroup to using the solvable radical instead, and would extend Theorem B from nilpotent to solvable groups.
1.3 An outline of the random model
Unlike sampling a random graph, where edges can freely be added or omitted, sampling groups of a fixed order requires some delicacy. For example, there are 15 isomorphism types of groups of order 16 but only 1 each of orders 15 and 17. Sampling random groups has hitherto been approached in one of the following two ways.
Quotient Sampling.
Fix a free group of all strings on an alphabet , and consider quotients by normal subgroups sampled by choosing by some aleatory process.
Subgroup Sampling.
Fix an automorphism group of a structure, such as the group of permutations of a set , or the group of invertible linear transformations of a vector space . Consider subgroups where is sampled by some aleatory process.
Evidently, both methods yield groups, but neither offers sufficient variability when restricted to finite groups. For instance Gromov studied quotient sampling as a function of the word lengths of elements in , finding most quotients are , , or infinite [Gromov]. Also, subgroup sampling in (respectively ) has been shown by Dixon, Kantor–Lubotsky [KL], and others to essentially sample , (respectively, subgroups ).
To escape these conditions we adopt a method of sampling that appears antithetical to random models: we strongly bias our random selections. We settle on a model related to subgroup sampling in since this affords us easy-to-use group operations. (Note, Novikov–Boone demonstrated that the word problem in the free group is undecidable and thus working with quotients is not in general feasible [Nov55, Boo59].)
First, we sample random upper -unitriangular matrices but we insist that they are -sparse, for some constant . Then
[TABLE]
samples a subgroup whose order is a power of the prime characteristic of our fixed field . As we shall demonstrate in Theorem 5.2, without limiting our randomness to sparse matrices the groups will almost always contain the following subgroup.
[TABLE]
In essence, this is a -group analogue of the observations we made about sampling in and . However, sampling with sparsity gives substantial variation, as illustrated simply by comparing orders in Figure 1.2. An interesting recent study by R. Gilman describes a similar situation for permutations analyzed by Kolomogorov complexity [Gilman].
Secondly, once we have selected a suitably random upper unitriangular group , an extension to this group is selected by adding to its block-diagonal. That process consists of choosing a partition of the series of common generalized eigen -spaces (the fixed point flag) of the group . In each block we select a random (almost) quasisimple group with a representation of dimension at most the size of the block. We further allow for multiplicity and for permutations of isomorphic modules. This extends first by a block-diagonal abelian group, then a product of simple groups, followed by a layer of abelian groups, and a final layer of permutations. It is well known that every finite group has such a decomposition, often referred to as the Babai–Beals filtration [BB99]. We note our own filtration descends to the Fitting subgroup instead of to the solvable radical as in the Babai–Beals treatment; revisit Figure 1.1 for an illustration.
Along with the proposal of such a model inevitably come questions as to its efficacy. We address two of the more critical issues here. First, our model samples a large number of groups:
Theorem C**.**
A random group over , as above, samples from each of the following classes of groups.
- (i)
finite abelian groups of exponent dividing and order at most . 2. (ii)
For each -bilinear map , with , the Brahana groups [Bra35] with product (also denoted by ) as
[TABLE] 3. (iii)
For each alternating -bilinear map , with , the Baer groups [Bae] with product (also denoted by ) as
[TABLE] 4. (iv)
All classical groups for rank over where . 5. (v)
All permutation groups of degree at most .
In particular this class of groups samples from pairwise non-isomorphic groups of order which is a logarithmically dense set of all isomorphism types of groups of order . Furthermore, this class of groups is closed to direct products and subdirect products.
Secondly, for groups selected from our model, even a genus-1, 1-WL refinement results in a filter with constant average width. (Note, constant max width would result in a polynomial-time isomorphism test.)
Theorem D**.**
For a random group sampled by our model, one of the following cases occurs on average when and are large enough:
- (a)
* is abelian; or* 2. (b)
* has characteristic WL-filter refinement of length .*
It was predicted in [Wilson:alpha] that most -groups had characteristic filters of length , owing in part to a result of Helleloid–Martin [Helleloid-Martin]. However, outside of examples in \citelist[Wilson:alpha][Maglione:filters] there where no large classes of groups where it could be demonstrated that such a filter could be efficiently computed. In a survey of 500,000,000 groups of order conducted by J. Maglione and the fifth author, it was discovered that of groups admitted a filter refinement by algebraic methods, with most stabilizing at terms. Furthermore, in a sample of 100,000 -groups having orders between and , most filters refined to a factor of about times the original length. Theorem D offers a theoretical explanation for those experimental results.
1.4 Testing pseudo-isometry of alternating bilinear maps
One base case for which the application of Weisfeiler–Leman is unlikely to go much further is -groups of class and exponent . (This special case has long been considered as difficult as the general group isomorphism problem.) As we have seen in Baer’s correspondence [Bae] (cf. Theorem C (iii)), when is odd testing isomorphism of such groups is equivalent to the following problem: given two alternating bilinear maps , decide whether they are pseudo-isometric, that is, whether they are the same under the natural action of .
Let denote the linear space of all alternating matrices over , namely the matrices such that for all . Note, and denotes transposition on vectors and matrices, respectively. An alternating bilinear map with and will be represented by an -tuple of such matrices. Testing pseudo-isometry of alternating bilinear maps translates to the following: given two -tuples of alternating matrices over , and , decide whether there exists , such that the linear spans of and are the same. For an odd prime , testing the pseudo-isometry of alternating bilinear maps over in time is equivalent to testing isomorphism of -groups of class and exponent in time polynomial in group order. Also note that the naïve brute-force algorithm—enumerating all possible —takes time .
In [LQ] it was shown that when and are linearly related, for all but at most fraction of , there is an algorithm in time to test isometry of with an arbitrary .111The main result in [LQ] is stated in a so-called linear algebraic Erdős–Rényi model. This model is not essentially different from sampling random alternating matrix tuples. See also Remark 6.20 for some details. The technique used to derive this result merits further comment. It was inspired by, and can be viewed as a linear algebraic analogue of, a classical combinatorial idea from graph isomorphism testing, namely the individualization and refinement technique. More specifically, it follows the use and analysis of this technique by Babai, Erdős, and Selkow, in the first efficient average-case algorithm for graph isomorphism [BES]. By incorporating the genus concept [BMW] into the individualization and refinement scheme as used in [LQ, BES] we can both extend and improve this result and at the same time greatly simplify the algorithm. Indeed, we have implemented an effective version of this new algorithm in Magma [MAGMA]. We prove:
Theorem E**.**
Suppose is larger than some constant. There is an algorithm that, for all but at most fraction of , tests the pseudo-isometry of to an arbitrary -tuple of alternating matrices , in time .
We briefly outline a simplified version of the algorithm, which is easy to describe and straightforward to implement. A more detailed description can be found in Section 6.1. The simplified version has already captured the essence of the strategy, but it comes with two small drawbacks. First, it does not work over fields of characteristic . Secondly, the average-case analysis does not achieve the level stated in Theorem E. Both issues will be remedied in the algorithm presented in Section 6.2, followed by a rigorous average-case analysis.
Assume we are given two -tuples of and from for sufficiently large and odd . Let be the subspace of spanned by . Take the first matrices of to form a tuple for some constant . Note,
every pseudo-isometry from to maps to a -tuple of matrices in .
This simple observation leads to the following algorithm. (We say two -tuples of alternating matrices and are isometric if there exists an invertible matrix such that , and the autometry group of is .) First, check if the autometry group of is too large (larger than ). If so, does not satisfy our generic condition. Thus, suppose the autometry group is not too large, and enumerate all possible -tuples in . Exhaustively check if any of them is isometric to , and, in the case of isometry, check if any isometry between and extends to an pseudo-isometry between and . The number of isometries between and is also not too large, because it is equal to the order of the autometry group of .
Note that the coset of isometries between and can be computed in time over fields of characteristic not [BW:isom, IQ]. Enumerating all possible -tuples in incurs a multiplicative cost . Given an isometry between and , we can check whether and are pseudo-isometric in . Thus, the overall time complexity is bounded above by , where is the order of the autometry group of . As we shall prove in Section 6, there is an absolute constant such that for almost all -tuple of alternating matrices , the first matrices have autometry group of order at most . Thus, the overall time complexity of the aforementioned isometry test is for almost all and arbitrary .
Performance. We implemented the above algorithm in Magma with some key adjustments (see Section 6.1 for details). The implementation is publicly available on GitHub as part of a comprehensive collection of tools—developed and maintained by the first and last authors and their collaborators—to compute with groups, algebras, and multilinear functions [Git].
Absent additional characteristic structure that can be exploited, the traditional approach to deciding pseudo-isometry between alternating bilinear maps is as follows. Let denote the linear maps induced by . Compute the natural (diagonal) action of on , and decide if and —each of codimension in —belong to the same orbit. An alternative version of brute force is to enumerate and check if one of these transformations lifts to a pseudo-isometry from to . Which of these two brute-force options represents the best choice depends on the dimensions of and .
Our implementation is typically an improvement over both options. For example, in a preliminary experiment, our implementation readily decides pseudo-isometry between randomly selected alternating bilinear maps , while both brute-force options failed to complete. Note that the worst-case for all methods should be when are not isometric, since in that case one must exhaust the entire enumerated list (or orbit) to confirm non-equivalence. However, the modifications we made tend to detect non-equivalence rather easily, since other (easily computed) invariants typically do not align in this case. We were therefore careful to also run tests with equivalent inputs, so as to ensure a fair comparison with default methods.
1.5 On groups with genus- radicals
There are examples by the fifth author of non-isomorphic -groups having all proper nontrivial subgroups of a common order isomorphic, and likewise for quotients [Wilson:profile]. No amount of local invariants will distinguish such groups, so when a WL-refinement style algorithm such as ours encounters such a group it can go no further. Even so, those examples are low genus and thus isomorphism can be decided efficiently by unrelated methods [BMW]. However, should these groups arise as for a non-nilpotent group it remains to contend with them as a base case. Combining the code equivalence technique of [BCGQ], the cohomological techniques of [GQ], and results on the automorphism groups of low-genus groups [BMW], we are able to get a nearly-polynomial running time for testing isomorphism in an important subclass of such groups.
Theorem F**.**
Let be the class of groups such that —the largest solvable normal subgroup of —is a -group of class 2, exponent , such that acts on by inner automorphisms. Given groups of order , it can be decided in time if they lie in . If so, isomorphism can be decided, and a generating set for found, in time , where is the genus of .
Structure of the paper.
After presenting some preliminaries in Section 2, we detail the construction of the colored hypergraphs and prove Theorem A in Section 3. We then explain the combination of filters and composition series isomorphism in , proving Theorem B in Section 4. The model of random groups, and the effect of the refinement procedure in this model, are the subject of Section 5, where Theorems C and D are proved. Finally, we provide the average-case algorithm for -groups of class and exponent (Theorem E) in Section 6, and the worst-case algorithm for groups with genus- radical (Theorem F) in Section 7.
2 Preliminaries
Notation.
Let for . We use to denote the Gaussian binomial coefficient with parameters , and with base . Let (resp. ) be the linear space of all (resp. ) matrices over . The general linear group of degree over is denoted by . When for some prime power , we write simply and in place of and .
Definitions of bilinear maps.
Let be vector spaces over a field . A (-)bilinear map is a function such that
[TABLE]
If is another -bilinear map, we regard as a function on the same domain and codomain as by selecting arbitrary linear isomorphisms , , and . We say are isotopic if there exists such that for all , and principally isotopic if there is an isotopism of the form . If , we often require that . We say are pseudo-isometric if there is an isotopism of the form , and that they are isometric if there is a pseudo-isometry of the form . A bilinear map is alternating, if for any , .
Computational models.
Suppose, after fixing bases, that , , and , which we regard as column spaces. A bilinear map can be represented as a tuple of matrices , where
[TABLE]
Suppose is represeted by . The concepts of isotopism and principal isotopism then have natural and straightforward interpretations in terms of these matrices. Namely, we say are isotopic, if there exist invertible matrices , and , such that
[TABLE]
where denotes the -th entry of for . We say and are principal isotopic if they are isotopic with .
Similarly, an alternating bilinear map can be represented by a tuple of alternating matrices. Recall that an matrix over is alternating if for every , . When is not of characteristic , this is equivalent to the skew-symmetry condition. Let be the linear space of all alternating matrices over (and when ). Then pseudo-isometry and isometry have analogous formulations in terms of alternating matrix tuples.
Given two tuples of alternating matrices , the set of isometries between and is denoted as
[TABLE]
the group of autometries (or self-isometries) of is denoted as . The set of pseudo-isometries between and is defined as
[TABLE]
the group of pseudo-autometries (or self-pseudo-isometries) of is denoted as . It is straightforward to see that is a (possibly empty) coset of , and is a (possibly empty) coset of .
Some algorithms for bilinear maps.
We note several of the algorithms we cite are described as Las Vegas randomized algorithm in that they depend on factoring polynomials over finite fields. That is known to be deterministic if the characteristic of the field is bounded. In our input model we are given a list of the group elements, so all primes are bounded and so we cite these as deterministic algorithms.
Theorem 2.1**.**
Let be bilinear maps of vector spaces over a finite field .
*In time one can decide if are principally isotopic **[BOW]**Theorem 3.7. 2. 2.
If and the characteristic of is not , in time one can decide if are isometric [IQ].
In each case an affirmative answer is accompanied by a principal isotopism (or isometry).
We also require the following, which follows directly from Theorem 2.1 by enumerating .
Theorem 2.2**.**
Let be bilinear maps of vector spaces over a finite field .
In time one can decide if are isotopic.[BOW] 2. 2.
If and the characteristic of is not , in time one can decide if are pseudo-isometric [IQ].
The following theorem is the automorphism version of Theorem 2.1. Note that, unlike the case of graph isomorphism, for the problems here there are no known reductions from the isomorphism version to the automorphism version.
Theorem 2.3**.**
Let be a bilinear map of vector spaces over a finite field .
In time , one can compute a generating set for the group of principal autotopisms of [BOW]. 2. 2.
If and the characteristic of is not , in time , one can compute a generating set for the group of autometries of [BW:isom].
Remark 2.4* (Shuffles).*
A bilinear map can be encoded as a -dimensional array. Transposing that array allows us to change swap the roles of , for example creating a bilinear map or , etc. (Here is the dual space of ). This swapping is functorial and therefore isotopisms are permuted accordingly; cf. [BOW]. So while we highlight the situation for principal isotopisms we could indeed specialize any one of the three spaces. We shall assume throughout that when necessary a bilinear map is shuffled.
3 The colored hypergraph algorithm
A high-level description of our algorithm to construct a colored hypergraph associated to a finite group was given in the introduction. We now provide the details; for convenient reference, an outline is given in Algorithm 1 below.
3.1 Coloring within layers: low-genus quotients and restrictions
For , is a -vector space for some prime of dimension . Recall that for any vector space , denotes the projective geometry of , which we may think of as a poset whose elements are the vector subspaces of , (partially) ordered by inclusion, and we use to denote the set of -dimensional subspaces. Let denote the set of linear maps from to , i. e., the dual vector space of . Then the map is an order-reversing bijection . By the Fundamental Theorem of Projective Geometry, there is a bijective linear transformation222We note that in some cases, it makes sense to consider a layer as being defined over a larger field , thus effectively reducing its dimension, and reducing the size of the hypergraph. In such cases, this map is only guaranteed to be semi-linear, that is, , but where is an automorphism of the field . This doesn’t present any essential difficulties, but needs to be kept track of. such that . Let be the linear form defined by . For , let .
The vertices and hyperedges of are, respectively,
[TABLE]
(Recall that .) To regard as a hyperedge, when convenient we identify the -subspace with the set of points (1-spaces) it contains. The initial coloring is as follows.
- •
Vertices. The initial color of a vertex is simply the index of the layer such that .
- •
Hyperedges corresponding to subspaces of codimension (dimension ), when . The initial color of these hyperedges is determined by together with a set of labels indexed by pairs such that as follows: if , the label of corresponding to the pair is the isotopism type of the projection ; when it is the pseudo-isometry type of this projection.
- •
Hyperedges corresponding to subspaces of dimension (elements of ), when . The initial color of these hyperedges is determined by together with a set of labels indexed by as follows: the label of corresponding to is the isotopism type of the restriction of the bimap to . (When the dimension is such that dimension and codimension subspaces are the same, this set of labels is appended to the set of labels for codimension subspaces; the two sets of labels are kept separate by their indexing.)
- •
Hyperedges when . In this case, there is only a single hyperedge corresponding to the entire layer . It is given a color that is similar to the previous two, namely, for each such that , gets a set of labels indexed by the pairs , labeled by the isotopism type of (resp., pseudo-isometry type if ), together with, for each (now including ) the isotopism (resp., pseudo-isometry) type of the bimap .
Observe, one need not pre-compute all isotopism (resp. pseudo-isometry) types. Instead, one can generate labels on the fly by pairwise comparison. Namely, given a new hyperedge to label, test for isotopism (or pseudo-isometry) between and all distinctly labelled , introducing a new label for if necessary.
By Theorem 2.2, isotopism and pseudo-isometry of bilinear maps can be decided in time , and also (by Remark 2.4) in time . (When , this can be decided very efficiently using the algorithm in [BMW].) It follows that we can label all hyperedges in time . Note that if the charactistic is , then even for maps of the form , we only use the isotopism label instead of pseudo-isometry label, because the results of [IQ] are not yet known to extend to characteristic 2. While this is less refined information, it is still useful.
3.2 Coloring between layers
The colored hypergraph described in the previous section already contains much local information from which global characteristic structure may be inferred, extracted, and used. However, we can often elucidate further characteristic structure by examining individual commutator relations between the layers. Of the various possible strategies one could try, we propose one that is both elementary and effective.
For each distinct pair , add to the the following edges. For each such that in (that is, ), we add an edge from to . For each which do not commute modulo , we add a hyperedge of size 3, connecting , , and . Upon refinement, this allows the vertex colors within each layer to affect the colors in the other layers.
3.3 The Weisfeiler–Leman procedure
Given a vertex-and-hyperedge-colored (hereafter just “colored”) hypergraph , where ( a finite set of colors), we show here how to apply the -dimensional Weisfeiler–Leman procedure -WL, originally developed in the context of graphs independently by Babai–Mathon [Babai79] and Immerman–Lander [ImmermanLander] (see [CFI] and [Bab16] for more detailed history). For the case of (color refinement) applied to hypergraphs, the same procedure was proposed and studied in the very recent preprint by Böker [Boker]. In particular, Böker shows that when we consider a graph as a (2-uniform) hypergraph, this procedure coincides with the usual color refinement procedure on graphs.
Let denote the colored hypergraph resulting from applying -WL to . The two key properties we will need in our application of this procedure are that: (1) can be computed from in time, and (2) If is another colored hypergraph, then and are isomorphic (as colored hypergraphs) iff and are isomorphic as colored hypergraphs. (In fact, the set of isomorphisms will be the same: ).
We find it simplest to describe the application of WL to hypergraphs by using instead their “incidence (bipartite) graphs.” We believe this bijection between vertex-and-edge-colored hypergraphs and vertex-colored bipartite graphs is essentially folklore; we include it here for completeness. Given a hypergraph , its incidence graph is the bipartite graph where , , . It is not hard to see that every bipartite graph arises from a unique hypergraph in this manner, so is a bijection and is well-defined.
An isomorphism between two vertex-and-edge-colored hypergraphs () is a bijection such that (1) , (2) for all , and (3) for all . We say that two vertex-colored bipartite graphs () are isomorphic if there are bijections and such that (1) and (2) for all and for all .
Proposition 3.2** (Folklore).**
Given two vertex-and-edge-colored hypergraphs , there is a natural bijection between and ; in particular, is isomorphic to iff their vertex-colored bipartite incidence graphs are isomorphic. Furthermore, both and can be computed in time.333 for hypergraphs and for bipartite graphs; for hypergraphs and for bipartite graphs.
Proof sketch.
Notation as above. Given , a vertex-and-edge coloring on a hypergraph , we get a coloring on the vertices of , which we also denote by by abuse of notation. The coloring on is the same as before, since and . The inverse is similar. The running time results from the fact that and can essentially be described by identical underlying data structures.
We show the natural bijection between and . Given an isomorphism from to , we define an isomorphism from to in the natural way: for , and for we define , that is, is the vertex in which corresponds to the hyperedge . To see that is an isomorphism we must check that it preserves incidences and colors. For incidences, we have iff (thinking of and ) iff (since is an isomorphism of hypergraphs) iff , by the definition of . To see that the colors are preserved, for , we have, by definition (and abuse of notation), that , and for we have . The inverse construction of an isomorphism from an isomorphism is essentially gotten by reading all the preceding equations in reverse. ∎
Our -WL procedure is to apply standard (graph) -WL to , then applying to get back a refined colored hypergraph.
Finally, we recall the -WL procedure as applied to a vertex-colored graph. If the graph is bipartite and we want to preserve the bipartition —as in our setting—we assume that the vertices in have distinct colors from those in . Given a vertex-colored graph , -WL refinement is the following procedure. Each -tuple of vertices is initially assigned a color according to its colored, ordered isomorphism type; that is, two such -tuples and are given the same initial color iff (1) for all , (2) iff for all , and (3) iff for all . Two -tuples and are said to be -neighbors if they are equal except that . In each step of the refinement procedure, the coloring is refined as follows: the new color of a tuple is a -tuple of multisets, where the -th multiset is the multiset of colors of all the -neighbors of . At each stage, the coloring partitions ; the procedure terminates when this partition doesn’t change upon further refinement. Once the coloring on has stabilized, we get a new coloring on by defining for to be the color of the diagonal -tuple . We denote the resulting colored graph by . From , can be trivially computed in time ; the current best-known running time is still [ImmermanLander, Section 4.9]. For more details on running time, implementation, and the properties of -WL on graphs, see, e. g., [Weisfeiler, WL68, ImmermanLander, AFKV, DGR].
3.4 Extracting characteristic structure
Each color class of vertices of provides (by lifting from to along the natural projection) characteristic subsets of , but not necessarily characteristic subgroups; it is only the latter which can be used to refine the filter . To get characteristic subgroups instead, we consider the subgroup generated by all the vertices in a given color class. We now write out this procedure more formally.
Let denote the refined coloring function of . For each , let denote the restriction of to the vertices in . For each color in the image of , let be the subgroup of generated by the elements that are colored . Finally, let be the natural projection; we lift to a characteristic subgroup of (and hence of ) as .
Finally, the set of new characteristic subgroups we consider is
[TABLE]
If , its members may be supplied to Theorem 3.4 to refine , in which case step 3 of Algorithm 1 is repeated. If not, then our colored hypergraph is now stable and Algorithm 1 terminates.
3.5 Refining filters
One filter refines another filter on the same group if the image of contains that of (the image is the collection of all subgroups in the filter). If is a characteristic filter and is a characteristic subgroup such that for some , then can be refined to a characteristic filter that includes . This was first introduced in [Wilson:alpha], and shown to be computable in polynomial time by Maglione [Maglione:filters]:
Theorem 3.4** ([Maglione:filters, Theorem 1]).**
Let be a filter on , and such that there exists with . Then a filter refining and including can be computed in polynomial time. Furthermore, if and are characteristic, then so is the refined filter.
We proceed sequentially through the characteristic subgroups of , refining as we go.
3.6 Proof of Theorem A
For part (i), let . Observe, if Step 3 (c) was omitted from Algorithm 1, then colors would only be assigned to hyperedges on points in fixed layers. In that case, moreover, the color of a hyperedge in layer is determined completely by pairs with ; the coloring function does not depend at all on layers . That is to say, if Step 3 (c) is omitted, then restricted to would be identical to the colored hypergraph based on truncated at . Step 3 (c) colors edges between layers using information from layers ‘lower’ in the filter; this means the restricted hypergraph is a refinement of the hypergraph on the truncated filter.
For part (ii), let and be two finite groups. Suppose we first construct . Next, we construct introducing new color for only when it is new to both colored hypergraphs. Evidently, if , then and are isomorphic with identical color sets.
Finally, we analyze the running time. Computing the Fitting subgroup and the initial characteristic filter can be done in time, even by naive algorithms (which can be improved significantly when is given by generating permutations, generating matrices, or black-box generators). Building the hypergraph can be done in time linear in the number of hyperedges, which is the number of codimension- subspaces of each layer , which is , and thus in total is at most . The hyperedges can then be colored in time using the isotopism and isometry algorithms (Theorem 2.2). As with -WL for graphs, -WL for hypergraphs can be computed in , which in our case is . Extracting the characteristic subgroups from can easily be done in time, and refining the filter can then be done in time as well [Maglione:filters] (reproduced as Theorem 3.4 above). The only remaining question is how many times the main refinement loop can run. Because we only refine when a characteristic subgroup is found which lies strictly in between some and , and the indices and are both at least 2, refinement can happen at most times. Thus the total running time is . ∎
3.7 Incorporating additional invariants
Our algorithm is not particular to the initial characteristic filter we choose. In any given group class, further characteristic subgroups (or subsets, or collections of subgroups) may be available which could be used to refine the filter, either at the beginning, or in each iteration of the main loop of Algorithm 1. We give two examples here without much discussion, just to illustrate the concept, without detracting from the main foci of the paper.
First, it may be the case that some of the bimaps are defined over a field larger than , i. e., for some . If this is true for sufficiently many of the bimaps, we may be able to treat some layers entirely over , thus reducing their dimension by , and reducing the number of vertices in the corresponding factor of the hypergraph by a factor of in the exponent (from to ).
Second, as acts on by conjugation, and the layers of are -invariant, for each we can compute a linear representation of on the elementary abelian layer . Using standard module machinery—for example, the version of the Meataxe algorithm described in [HR]—in time polynomial in each -module may be decomposed first into indecomposable summands, and then into isotypic components. The collection of isotypic components is a characteristic subset of subgroups—namely, they can be permuted amongst themselves by the action of , but that’s it. We can either group these into -orbits of isotypic components to get characteristic subgroups to refine the filters, or keep the characteristic subset of subgroups and incorporate it into Rosenbaum’s composition series isomorphism technique, discussed in Section 4.
3.8 The procedure through an example
We examine the procedure with a toy example as follows. Consider the following alternating matrix tuple in , which was also considered in [BOW].
[TABLE]
We construct a bipartite graph , where , and , so that for and , if and only if . In particular, note that .
For each , we choose a non-zero vector on as its representative. So
[TABLE]
For each , since is a -dimensional subspace of , we choose one defining linear equation , , as its representative. So is also
[TABLE]
In this notation, connects to , if and only if .
For , we define in . We use to give the vertex color. Using red for rank and blue for rank , we have
[TABLE]
The first step of refinement uses the colors on the side to color the vertices on the side. For example, on the side is adjacent to {\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}(1,0,0)}, {\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}(0,1,0)}, {\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}(1,1,0)}, {\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}(1,2,0)}. So obtains the color as “3 blues and 1 red”, or 3B1R for short. We therefore let blue for 4B, green be 3B1R, and red for 2B2R. We then have
[TABLE]
Note that these colors, which comes from genus-1 information, already gives the genus-2 isomorphism types.
The second refinement uses the colors of the side to recolor the vertices on the side. For example, on the side is adjacent to {\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}(1,1,1)}, {\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}(1,2,0)}, {\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}(1,0,2)}, {\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}(0,1,2)} on the side. So obtains the color as “2 blues and 2 reds”, or 2B2R for short. We therefore let red for 3R1G, blue for 1R2G1B, and green for 2R2B. We then have
[TABLE]
It can be checked that we reach at a stable coloring after this step.
Note that these colors suggest the green points form a characteristic set. This characteristic set would generate the whole group, so it does not yield a non-trivial characteristic subgroup. However, this characteristic set is already interesting, because it does suggest that the Weisfeiler–Leman procedure, or even the naive refinement, gives non-trivial information regarding group elements under the action of automorphisms. We discuss how to take advantage of such characteristic subsets in isomorphism testing in the next section.
4 Isomorphism testing using the filter and hypergraph
Our colored hypergraph and filter constructions can be used to refine the composition-series isomorphism method of Rosenbaum and Wagner [RosenbaumWagner], thereby speeding up the resulting isomorphism test. Here, we present an isomorphism algorithm which runs in time if the filter output by Algorithm 1 with also has “width” at most (defined below). Though we claim no asymptotic improvements in the worst case, we expect our test to perform well for many specific group classes, as well as for groups chosen randomly (including groups selected from the random model we discuss in detail in the following section). In practice, one should also apply Rosenbaum’s bidirectional collision technique [RosenbaumBidirectional] to get a square-root speed-up, but this causes no new technical difficulties.
In fact, the running time we get is . We note that the largest that both and the width can be is ; if we allow to be near-maximal (take ), and this results in a filter whose width is just slightly less than maximal, say, , then the entire algorithm runs in time , asymptotically beating the trivial algorithm by a factor in the exponent. Because this is such a generous bound on and a weak desired outcome for the width, we expect this runtime to hold for many classes of groups.
We begin with a simple version, building up to Theorem B in steps.
4.1 Simple version: choose composition series compatible with the filter
We begin by recalling the composition-series isomorphism technique of Rosenbaum and Wagner [RosenbaumWagner], and show the simplest way to incorporate our characteristic filter into that technique. (Recall that, although we are not using the colored hypergraph here directly, it contributed to the construction of the filter.) Composition Series Isomorphism is the following problem: given two groups , and a composition series of each and , decide whether there an isomorphism such that for all . Rosenbaum and Wagner [RosenbaumWagner] show how to reduce -group isomorphism to Composition Series Isomorphism, and then how to reduce the resulting Composition Series Isomorphism Problem to Graph Isomorphism on graphs of degree at most ; Rosenbaum more generally showed how to reduce to Composition Series Isomorphism in time. Luks [LuksCompSeries] showed how to solve Composition Series Isomorphism in time. Recall the socle series of a group is defined as follows: the socle is the subgroup generated by all minimal normal subgroups. is always a direct product of simple groups. We then recursively define to be the preimage of in , that is, if is the natural projection, then . The reduction is to pick a composition series for that is compatible with its socle series, and then to try all possible composition series for compatible with its socle series. One of the keys to their running time is to show that the number of composition series compatible with the socle series is bounded by .
Within , we refine the socle series with our characteristic filter. Without loss of generality, we may assume that the restriction of our characteristic filter to the Fitting subgroup refines the socle series of . If it doesn’t originally, we may further refine it using the socle series, then iterate the main loop of Algorithm 1 until it stabilizes again. Our algorithm here is to reduce to Composition Series Isomorphism, but to only consider composition series that are compatible both with our filter and with the socle series. If the filter has many small layers, this will cut down the number of composition series that need to be considered, thus reducing—for such groups—the dominant factor in the running time of [RosenbaumWagner, RosenbaumBidirectional, RosenbaumSolvable].
To illustrate the potential savings, we define the width of a filter with elementary abelian layers to be the maximum dimension of any layer:
[TABLE]
Then we have:
Theorem 4.1**.**
Let be a solvable group of order , and be a characteristic filter on computable in time . Then isomorphism of with any group can be tested, and an isomorphism found, in time .
In particular, using the characteristic filter output by Algorithm 1 with parameters and , isomorphism of solvable groups can be solved in time
[TABLE]
Proof.
The outline of the algorithm follows Rosenbaum–Wagner [RosenbaumWagner], also using Luks’s polynomial-time algorithm for Composition Series Isomorphism [LuksCompSeries]; the key difference here is that we only consider composition series which refine our characteristic filter , rather than more general composition series as in Rosenbaum and Wagner. The runtime of their algorithm is a product of the running time to enumerate the desired composition series, and the time to solve Composition Series Isomorphism. Our improvement is in the first step. So we only need calculate the number of composition series of compatible with . In our case, we must first compute the filter , which takes time .
Let be a second solvable group. Enumerating the composition series of compatible with can be achieved as follows. Go through in lexicographic order, starting with the lexicographically largest such that . Within each layer we choose all possible composition series. By [RosenbaumWagner, Lemma 3.1], this can be done in time . Taking the product over all layers, we get a bound of . For each such composition series, we then use Luks’s -time algorithm for Composition Series Isomorphism, yielding the stated result.
For the “in particular,” we compute using Algorithm 1, which takes time. ∎
4.2 Intermediate version: choose composition series compatible with the filter and hypergraph
The vertex coloring of the hypergraph may inform us of characteristic subsets that are not subgroups. Although the filter has been refined as much as possible (in particular, any one of the color classes of the hypergraph in a given layer must generate the whole layer), we can nonetheless take advantage of these characteristic subsets in the preceding algorithm, by further restricting the composition series that we need to consider.
Towards this end, for each layer let denote the smallest color class in , and define the color ratio of a layer as . Finally, define the color ratio of a solvable group as
[TABLE]
We now restate (a slightly refined version) of Theorem B:
Theorem B (Refined).
Let be a solvable group of order . Let and be the filter and colored hypergraph for output by Algorithm 1 with parameters . In each layer , let denote the smallest color class. Then isomorphism of with any group can be tested, and an isomorphism found, in time
[TABLE]
Proof of Theorem B.
The outline of the algorithm is the same as in Theorem 4.1; the key difference is how we enumerate composition series within each layer (and how many we enumerate). To see how to take advantage of the size of the smallest color class , we must recall the details of Rosenbaum & Wagner’s Lemma 3.1 [RosenbaumWagner], on enumerating composition series. In the algorithm of Theorem 4.1 above we have already taken care of the ordering of the layers, so the only difference here will be on how we enumerate the part of the composition series within each layer . That is, we may assume that we have already built a composition series of , which we now want to extend to a composition series of . Since the subgroup generated by would be a characteristic subgroup of , and has already been refined according to the coloring on , it must be the case that generates all of . Thus we may select only those composition series where the generator of each step of the composition series comes from . Since any generating set (and hence any composition series) for has size , the number of choices of composition series where all the generators in the series are chosen from is bounded by
[TABLE]
This analysis already gives the second bound in the statement of the theorem. To get the more refined bound, within each layer , if , then we employ the above strategy, and otherwise we use the strategy from Rosenbaum–Wagner [RosenbaumWagner]. ∎
4.3 Advanced version: refine the filter and hypergraph as you go (individualize and refine)
Finally, we give a version of the individualize-and-refine paradigm from Graph Isomorphism as applied to composition series that are compatible with our filter and colored hypergraph. The algorithm is similar to that from the previous section, except now, each time we pick a subgroup in our composition series, we give a new color to the corresponding vertex in our hypergraph, and then we run more iterations of the main loop of Algorithm 1 until the filter and hypergraph again stabilize, before we pick the next subgroup in our composition series. This can potentially have the effect of reducing the width of the layers and/or the size of the smallest color class in each layer as we go.
In somewhat more detail: compute the filter and colored hypergraph as before. We build up a composition series in and simultaneously keep a list of partial composition series in that we want to test for isomorphism in the end. Suppose we are at the point where we already have built a composition series up to in , and we have a list of composition series up to in . Then we extend the partial composition series of by picking an element of (the smallest color class in ). We then color the corresponding vertex in a new color, and refine both and until stabilization (as in the main loop of Algorithm 1). Within , we try each element of in turn, refining the filter and hypergraph for . If for any the refinement does not agree with the refinement we got in , we throw it away, otherwise we extend our composition series for by the subgroup generated by and , and add this new partial composition series to our list . This comes at a multiplicative cost of . We then continue this process within the (potentially new, smaller ) until we get a composition series that now includes all of . The total multiplicative cost within the layer is thus at most , so this at most squares the total running time from Theorem B.
Thus, asymptotically, we get a similar worst-case upper bound. We could state a more refined upper bound along the lines of Theorem B, but the definitions involved are somewhat delicate and recursive (because they depend on how the width and the color-ratio change as the algorithm progresses). Nonetheless, in practice, we expect this individualize-and-refine technique to perform much better, as the layers and color class should decrease in size as the algorithm progresses.
5 A random model of groups
Inspired by a suggestion of A. Mann [Mann1999]*Question 8 (answered in [KTW]), we describe here a model for random finite groups. We first give a simplified model that samples only random finite nilpotent groups. Later we extend this to sample solvable, semisimple, and general finite groups.
5.1 A model for random finite nilpotent groups
As a first approximation we choose random upper unitriangular -matrices , over the integers modulo a fixed positive integer . The are drawn according to a fixed distribution . Later we shall discuss the effect of on the group theory, but first we survey the possible outcomes.
An immediate observation is that is a subgroup of the full group of upper unitriangular matrices. Therefore, is nilpotent of order at most . In particular, if denotes the Sylow -subgroup of , then . The choice of generators also has a fingerprint within the structure of our groups . In particular by Burnside’s Basis Theorem, for each ,
[TABLE]
Thus, there is a certain amount of structure which is fixed by the choice of parameters . Nevertheless, the coverage asserted in Theorem C shows the diversity of these groups.
5.2 General model
To sample a more general class of groups, we add terms to the block-diagonal. Sampling random invertible square matrices will almost always generate the entire general linear group. As noted in Section 1, a more nuanced approach is called for.
Our strategy is as follows:
- (a)
Add solvable groups by selecting any matrix that is diagonalizable over the algebraic closure. We call this a random toral subgroup. 2. (b)
From the classification of finite simple groups we can select at random, according to a fixed distribution, a non-abelian finite simple group and let —that is, choose , a (possibly trivial) central extension of an almost simple group. Then we form the group algebra . We then sample from the minimal left ideals of . This defines a linear representation where is a -module. It is a straightforward exercise to see that endomorphisms of finite modules are representable as chequered matrices. We copy the image of a generating set for into chequered matrices, and then place this on the block diagonal. We repeat until we exceed a bound on . 3. (c)
Add permutation to the block diagonal to any two terms with isomorphic representations. 4. (d)
As a final step we now sample block upper unitriangular matrices.
It is important to proceed in this order to avoid redundant choices. The number of variability of the simple modules represented on the block diagonal is again controlled by the distribution and that can have substantial impact on the resulting group.
Proposition 5.1**.**
The class of groups sampled includes: A permutation group , central extensions of almost simple groups, where is sampled as above and is a -module.
5.3 Coverage: Proof of Theorem C
For (i) consider matrices of the form . If then all such commute and are independent. So fix a divisor chain and coefficients (in some index order) having additive order , it follows that these generate an abelian group with the specified invariants.
For (ii-iii), let where , represented as a matrix. Let be representation of as in (i), and likewise with . Then
[TABLE]
For the count note that it suffices to count the number of distinct bilinear maps . As Higman demonstrates [Hig60], there are such maps.
Finally, for a given list of groups sampled in smaller dimensions, form block diagonal representations. This affords the direct product of the list. For subdirect products take a subgroup of the block-diagonal group. This completes the proof of Theorem C.
5.4 The importance of the right distribution
If we sample dense matrices when we shall call the result the dense random subgroup model for the general linear group . While this is an easy model to reason about it is also fairly rigid, as the following result illustrates.
Theorem 5.2**.**
If are chosen uniformly at random from the group of upper uni-triangular matrices and then
[TABLE]
In fact we shall prove the following stronger claim: with high probability, such groups contain the group of commutators of .
In this model, groups can range widely in isomorphism types, one does not see much variability in coarse isomorphism invariants such as group order, numbers of subgroups or quotients, conjugacy classes, and so forth.
Our proof of Theorem 5.2 relies on some details of Sims’ proof on the asymptotic upper bound on the number of isomorphism types of -groups [Sim65]. It begins as follows. For a group let be the th term in the lower central series. Every -group has a subgroup such that and where the least number of group elements to generate , is minimal with that property [BNV:enum]*Proposition 3.8. We call the the Sims’ rank of .
Definition 5.3**.**
A Sims subgroup of a nilpotent group is a subgroup minimal with respect to . The Sims rank of is the minimum number of generators needed to generate a Sims subgroup.
Fix , , . Then there is a bimap given by commutation:
[TABLE]
Fix a subgroup of , and put . Observe that is a Sims subgroup if, and only if, . Also observe that after taking natural bases for and , the bimap can be represented as follows. Let be defined, in a parametrized form, by where
[TABLE]
That is, could be understood as a -tensor of size , whose th frontal slice is given according to .
Proof of Theorem 5.2.
Our approach is to show that a subgroup generated by enough elements is a Sims subgroup. To do this it suffices to show that for most sufficiently large dimensions, the bilinear map of (5.5) has the property that most satisfy . For notation we let with basis and with basis .
If is the row span of the full rank -matrix then
[TABLE]
This defines a natural -tensor of size by
[TABLE]
Notice if, and only if, . That is, if we flatten the tensor into a -matrix , as follows,
[TABLE]
then we are asking that is of full rank. Now we argue that for this is the expected behavior.
By our model, each entry in is drawn independently at random. However the entries of (and therefore ) are dependent. Nevertheless, we can observe that the values of are almost independent of . Certianly is independent of if . Also, if , if then nothing can be said about . Even if it may be impossible to predict , the exception is when . So there is chance of dependence between with the exception of pairs of [math]. Each such dependency will be compensated for by adding a row such that . Thus will be independent of . Since the selection of a nonzero entry is a event the addition of one row is highly likely to break dependence. Thus, at a cost of sampling rows we obtain with high probability a matrix whose associated matrix is full rank. ∎
5.5 Sparsity
For added variation a different distribution is required, one which favors sparse matrices. Fix positive integers and . Let be the number of non-zero values in the upper unitriangular . Fix a distribution on and a distribution on . Define a -random triangular matrix as an sampled according to a distribution with probability and for each , is sampled according to . Notice uniquely determines an upper unitriangular matrix:
[TABLE]
The distribution describes how large the support of is expected to be, and describes what non-zero values in will be used as entries.
Finally define a -random unitriangular group as the group generated by independently sampling upper unitriangular -matrices over according to their -distribution. The precise outcomes of this distribution appear intricate. Through some empirical testing (e.g. Figure 1.2) we have produced the following question:
If for , does approach a discrete Gaussian distribution on ?
Our model makes several constraining choices in order that it avoids the analysis that would otherwise create rather similar groups. The cost of this is that we can so far only offer heuristic explanations for the behavior. Even so, we explain what we understand and encourage a thorough exploration in the future.
The first question is what to expect the length of the block diagonal to be in . Suppose we assume that the block diagional is chosen uniformly at a partition of . From Vershik’s theorem [Vershik], the shape of the tableaux of random partition of with at least terms tends to . That implies that there are relatively few large blocks as those are in the tail of the random distribution. Thus there would be many blocks of small size. This however requires one justify that sampling at random samples partitions of uniformly at random. That need not be the case. So we ask
Is the typical sparsely sample group convex (tending toward the middle) or concave (tending away from the middle)?
The answer to this speaks to the expected nilpotence class of the groups . The length of this block diagonal is a bound on the nilpotence class. For example, if there are just two blocks, then
[TABLE]
implies that is abelian. In general, if denotes the subspace flag determining the block structure of , then the nilpotence class of is at most .
To see a reason that sparse matrices should sample a wider class of groups than dense matrices we consider a sufficient condition to avoid being a Sims subgroup.
Lemma 5.7**.**
Fix an alternating bimap with . Let be a basis of and define . For , if there exists an such that , then .
Proof.
If then for with , . ∎
Now here is the situation. The maps are alternating bilinear forms, possibly degenerate. The subspaces with are what are known as totally isotropic. The number of maximal totally isotropic subspaces of is where . Therefore the smaller the radical the much large the number of totally isotropic subspaces there are and therefore the less likely that a subspace generates . So as we move towards bimaps for unipotent hulls for flags of fixed length at least 3, then the commutator involved will have quotients to alternating forms with large numbers of totally isotropic subspaces. Thus more subspaces will fail to generate . As result, fewer subgroups will be Sims subgroups. This however is only a crude guide to the number of Sims subgroups and we encourage an actual analysis with better insights.
5.6 WL-refinement in our random model
So now let us consider the effects of refinement in our random model. Our proof is in two parts. Either our unipotent groups have long block diagonal series or it has bounded class. In the former case we reduce the refinement analysis to a result of Maglione [Maglione:filters]. In the later case we appeal to classical results on nonsingular products. In either case we discover refinements. We aim to prove Theorem D.
Refinements for many blocks.
First let us consider groups with many blocks.
Theorem 5.8**.**
The refinement length of a random subgroup is on average at where is the length of is generalized eigen -space flag.
Primarily we want to appeal to the following. Note that in this case we do not apply the Weisfeiler–Leman procedure developed in this paper; instead, it will be used in the next setting.
Theorem 5.9** (Maglione [Maglione:filters-U]).**
The group has an (adjoint) characteristic filter refinement of length .
However we do not have the group . Instead, we have a subgroup sampled at random either with dense or sparse matrices. First we dispense with the dense case.
Corollary 5.10**.**
A subgroup generated by dense matrices with has on average a characteristic filter refinement of length .
Proof.
By Theorem 5.2, is almost certainly a Sims subgroups of and therefore . As a scholium to Maglione’s theorem we observe that the adjoint filter refinement of can be defined as refinement through terms for in the filter. As a result these same terms appear in the filter of and so refines to a length of . Since the result follows. ∎
Next we need to consider the sparse case as this is where our model presides. What we do is demonstrate a form of Morita condensation theory that transports our sparse problem into a dense problem [Wilson:Skolem-Noether]. What we observe is that each right-hand edge of block on the block diagonal of is defined by the presence of an element with a non-zero value , otherwise the block would be wider. We select one such row for each block , and one such column . Thus out of the -block matrix , we create an -matrix by copying the entire in . For example
[TABLE]
This may seem a bit unnatural but in fact it is applying a functorial property not on the level of groups but on the level of the enveloping algebra of the matrices and more importantly on the level of bilinear maps. While this function has no relationship in the context of groups, it is by considering the associated ring context that we see that we have simply performed a condensation of modules, that is we have changed to an equivalent category. So for each of the blocks we let be the matrix with zero in every position except . Set . Then is matrix with at most nonzero entries. Removing the all zero rows and columns produces an -matrix. In the example above,
[TABLE]
In particular this induces a functorial Morita condensation of each bilinear map , see [Wilson:Skolem-Noether]. We therefore denote this group to remind us of the natural process to create this smaller matrix group.
Having applied this transform, notice is now a dense subgroup of . Therefore we arrive at the following.
Proof of Theorem 5.8.
Suppose generate by random matrices . If the are dense then by Corollary 5.10 there is a computable filter refinement of length where is the number of blocks of . If on the other hand the are sparse, then has a refinement of length . As the map is functorial in the bilinear maps used to select refinement, it follows that also has a refinement of length . ∎
Refinements for few blocks.
The last case to concern us is when has a bounded number of blocks on the diagonal, but that the number of blocks is at least 3. (Otherwise the group is abelian which is the first case of Theorem D.) Because the number of blocks is bounded at least one block has dimension proportional to as .
Let us consider coloring with . This means that with a selected layer we consider labels on -dimensional subspaces by labeling the restriction . One observes this structure is nothing more than a linear transformation and is thus defined up to change of basis solely by the rank of the transformation. Therefore to each element of we record the rank of the associated matrix. We do likewise with . Finally we label the edges between and by whether or not the pair of points commutes.
In order to model this behavior in colors we make the following observation. Treating as homogeneous point in variables, the evaluation produces a matrix with entries in . The rank of this matrix changes as we evaluate but certainly there are two natural states: either has rank at most or it does not. If has rank at most then all -minors must vanish, and this produces a polynomial number degree -polynomials that must all vanish on . That is to say, the condition of the rank of is a variety (or more generally a scheme). It is in fact a determinantal variety and the subject of considerable study in the algebraic community as well as the computer science community \citelist[Harris92][Minrank]. It is important to observe that many results in the field are only known over algebraically closed fields. However it is known that these varieties are reduced and irreducible [Harris92]. Therefore to count points we can use Lang–Weil theorem [LW54], but that requires that we allow for a large field. So this portion of our estimate assume and .
Let us assume for now that has points, i.e. that for some , does not have full rank, and for other points it does. Thus our vertex set has (at least) 2 colors, say white if has full rank and black otherwise. We do the same for . Recall that we are including a hyperedge only if .
Now consider the situation. The number of black points is in general a solution to a system of random nonlinear homogeneous polynomials of degree . That this is nonlinear means we can expect that the number of black points is not a subspace. Now the points in not connected to black points are the points . In particular we have a nonlinear set parameterizing a subspace arrangement within . If we write the generator matrix of each subspace it will be the dual of a linear combination of the matrices used to define , which we sampled at random. Therefore we have a random subspace arrangement. In general this incidence relation is not equitable, so proper refinements will be discovered in the WL-refinement process.
With that we have proved the following.
Theorem 5.11**.**
If has a bounded number of blocks and are large, then there exists a proper refinement of the standard filter.
To remove the assumption that are large here, the following interesting question needs to be addressed.
Let be random matrices over . What is the typical number of non-full-rank matrices in the linear span of ’s?
5.7 Proof of Theorem D
Let us suppose is sampled according to our model. Let be the intersection of with and begin with the initial filter of our introduction. Then if is abelian we are in case (i). Otherwise has at least blocks so we can use either Theorem 5.8 for the case of large blocks, or Theorem 5.11 in the case of small blocks. In either case we obtain a proper refinement. Note that after refinement of the bounded number of blocks several times we cross over to the large number of blocks and so the result follows. ∎
6 Testing pseudo-isometry of alternating bilinear maps
6.1 The simplified main algorithm
In this subsection we formally describe the simplified main algorithm presented in Section 1.4, that is Algorithm 2. We also discuss some important adjustments used in the implementation. We need the following observation, which follows easily by computing the closure of the given generating set.
Observation 6.1**.**
Let , and let be the group generated by ’s. Let . Then there exists an algorithm that either reports that , or lists all elements in , in time .
Let us first examine the running time of Algorithm 2.
Proposition 6.2**.**
Algorithm 2 runs in time .
Proof.
If Algorithm 2 outputs , then its running time is determined by Theorem 2.3 (2) and Observation 6.1, which together require .
If , we analyze the two For-loops at Step 4 and Step 4.c, respectively. The first loop adds a multiplicative factor of , since enumerating a single element in costs . The second loop adds a multiplicative factor of , due to the fact that , as is a coset of . Other steps can be carried out in time . Therefore the overall running time is upper bounded by . ∎
We then prove the correctness of Algorithm 2, in the case that it does not report .
Proposition 6.3**.**
If Algorithm 2 does not report , then it lists the set of pseudo-isometries (possibly empty) between and . In particular, .
Proof.
By Step 4.c, every added to is a pseudo-isometry. We are left to show that contains all the pseudo-isometries. For this, take any pseudo-isometry . Since the linear spans of and are the same, we know is equal to some . So when enumerating this in Step 4, will pass all the tests in the following, and then be added to . This concludes the proof. ∎
It remains to specify the choices of and in Algorithm 2 in the average-case analysis. This is stated in the following, whose proof will be deferred to Section 6.3.
Proposition 6.4**.**
Let . For all but at most fraction of , we have .
Combining Propositions 6.2, 6.3 and 6.4, we have the following theorem.
Theorem 6.5**.**
Let , and let be a finite field of odd size. For all but at most fraction of , Algorithm 2 tests the isometry of with an arbitrary in time .
Implementation details. We now explain some issues in the implementation of Algorithm 2.
To make this algorithm suitable for practical purposes, recall that the algorithm’s running time is dominated by the two For-loops which give multiplicative factors of and , respectively. For the average-case analysis we used , but having this standing on the exponent is too expensive. In practice, actually using already imposes a severe restriction on , the order of . So we use in the implementation which gives a reasonable performance.
But having in the For-loop is still too demanding. Indeed, in practice the tolerable enumeration is around , namely and on the exponent. So with , the range of is still severely limited. (Interestingly, the algorithm seems to have a better dependence on .) It is most desirable if we could let , namely simply .
To achieve that we use the following heuristic. Note that if are low-rank matrices, then we will only need to match them with the low-rank matrices from . Our experiement shows that, for a random over when is a small constant, the number of low-rank (i.e. non-full-rank) matrices in is expected to be small (i.e. much smaller than ) and non-zero (i.e. no less than ) at the same time. So we can use to choose low-rank matrices from . Then use to compute the set of low-rank matrices from , denoted as . We can then replace enumerating with , which in general is much smaller.
6.2 The main algorithm
To state our algorithm, we need the concept of adjoint algebra. For two tuples of alternating matrices , the adjoint algebra of is defined as
[TABLE]
and the adjoint space from to is
[TABLE]
Clearly, if , then . Furthermore, if and are isometric, then .
We now introduce the algorithm (see Algorithm 3) that supports Theorem E. We point out that Algorithm 3 differs from the algorithm presented in Section 1.4 in two places.
The first and major difference is to replace the uses of and with and , thereby avoiding using Theorem 2.1 (2) and Theorem 2.3 (2). Since and are easy to compute over any field, this resolves the characteristic- field issue. Furthermore, and are also easier to analyze. But and could be larger than and , so they are less useful from the practical viewpoint. 2. 2.
The second place is step 2 in Algorithm 3: instead of just using the first matrices as in the algorithm presented in Section 1.4, Algorithm 3 slices the matrices of into segments of -tuples of matrices, and tries each segment until it finds one segment with a small adjoint algebra. This step helps in improving the average-case analysis, and can be applied to the algorithm presented in Section 1.4 as well.
Let us first examine the running time of Algorithm 3.
Proposition 6.6**.**
Algorithm 3 runs in time .
Proof.
If Algorithm 3 outputs “ does not satisfy the generic condition,” then it just executes the For-loop in Step 3, which together runs in time .
Otherwise, there are two For-loops at Step 4 and Step 4.c, which add multiplicative factors and , respectively. Other steps can be carried out in time . Therefore the whole algorithm runs in time . ∎
We then prove the correctness of Algorithm 3 in the case that that it does not report “ does not satisfy the generic condition.”
Proposition 6.7**.**
Suppose that Algorithm 3 does not report “ does not satisfy the generic condition.” Then the algorithm lists the set of pseudo-isometries (possibly empty). In particular, .
Proof.
By Step 5.c, every added to is a pseudo-isometry. So we are left to show that contains all the pseudo-isometries. For this, take an arbitrary pseudo-isometry . Then sends to some , i.e., . In particular, . So when enumerating this , will pass all the tests in the following, and then be added to . This concludes the proof. ∎
Therefore, to prove Theorem E, the key is to analyze when a random satisfies the generic condition as in Algorithm 3.
Proposition 6.8**.**
Let , and let . For all but at most fraction of , there exists some , such that, letting , we have .
Clearly, Theorem E follows from Propositions 6.6, 6.7, and 6.8.
6.3 The average-case analysis
We now formulate the key proposition that supports the proof of Proposition 6.8.
Proposition 6.9**.**
Let . For all but at most fraction of , we have .
Given Proposition 6.9, we easily obtain the following.
Proof of Proposition 6.4.
This is because, if , then . So . ∎
Proof of Proposition 6.8.
We slice into segments, where each segment consists of random alternating matrices. Each segment is some , with . Since each is chosen independently and uniformly at random, the probability of every , , with , is upper bounded by . ∎
The rest of this subsection is devoted to the proof of Proposition 6.9. For this we need the following from [LQ]. Given a tuple , define the image of under as .
Definition 6.10**.**
We say is stable, if for any nonzero, proper , we have .
Proposition 6.11** ([LQ, Proposition 10 in arXiv version]).**
If is stable, then .
A key technical result in [LQ] is that, a random is stable with probability [LQ, Proposition 20 in arXiv version]. However, we cannot directly apply that result to prove Proposition 6.9, because here we have alternating matrices instead of general matrices. So we have to run the arguments for the proof of [LQ, Proposition 20 in arXiv version] again, and carefully adjust some of the details there to accommodate the structure of alternating matrices.
To start with, we need the following easy linear algebraic result, which suggests the connection between random alternating matrices and random general matrices.
Lemma 6.12**.**
Let and . Given two random alternating matrix , we can construct a matrix , whose columns are linear combinations of the columns of and , such that is a random matrix from .
Proof.
Let and be given as
[TABLE]
where each and are independent random variables from . Define
[TABLE]
be the matrix obtained by adding the th column of to the th column of for , and add the first column of to the th column of . Let be the matrix consisting of the first columns of . We need to show that is uniformly sampled from as and are uniformaly sampled from .
To see this, first note that for any two random variable and , which are chosen independently and uniformly at random from , are also new random variables which are chosen uniformly at random from , and is independent with either or . Thus each is again a random variable which is chosen uniformly at random from for .
We then exploit the linear relations among the ’s. In fact, we only need to focus on the anti-diagonal directions, as
[TABLE]
for any . Thus, we can view (note the missing ) to be mutually independent for each , then every entry in can be viewed as chosen independently and uniformly at random. This can be verified in a straightforward way, and we can conclude the proof. ∎
Remark 6.13*.*
Following the similar argument, if we would like to get an random matrix over , we can in turn do the following: take two random alternating matrices and and construct as in Lemma 6.3. We then take another two random alternating matrices and . We add up the first column of and , of which each coordinates can be viewed as chosen independently and uniformly at random. We replace the last column of by the new random vector, which gives an random matrix.
We are now ready to prove Proposition 6.9.
Proof of Proposition 6.9.
Given Proposition 6.11, we need upper bound the probability of a random , such that is not stable, by .
By the union bound, we know that
[TABLE]
We first simplify the right-hand-side. For a non-zero, proper , let . Clearly,
[TABLE]
We show that for any two dimension- subspaces and , . To see this, let be any invertible matrix that sends to . Note that further induces a linear map from to itself by sending to . Since is invertible, this map is a bijection. Moreover, for any , we claim that . This is because
[TABLE]
where the second equality holds since left and right multiplying invertible matrices does not change the rank of a matrix. To summarize, if , then
[TABLE]
The right-hand-side of 6.14 can be then simplified as
[TABLE]
where is the -dimensional subspace of spanned by the first standard basis .
The next goal is to upper bound for .
Let be the matrices consists of the first columns of for . (Note that the superscript here does not denote exponentiation.) Let . Then is just the rank of . Note that for , the first row of can be viewed as a random alternating matrix from , and the last rows of can be viewed as a random matrix. Moreover, these two matrices can be viewed as being chosen independently.
By Lemma 6.12 together with Remark 6.13, there exist a series of column operations represented by an invertible matrix , such that the following holds. Let be the matrix consists of the first columns of . Then can be viewed as chosen independently and uniformly at random from , as is chosen uniformly at random from . Note that when , the first row of is [math] for all . This degenerate case suggest us to consider as randomly choosing from . Note that
[TABLE]
and
[TABLE]
for .
We consider how to construct an matrix such that its rank is not larger than . One way to do so is to pick one column fix its coordinates; then let the rest columns be scalar of the picked ones. This procedure gives the bound
[TABLE]
So we have
[TABLE]
Using the same idea, we deal with . All possible such that can be constructed by first choosing columns in and fixing their entries, and then choosing the other columns from their linear span. This gives the bound
[TABLE]
where the last inequality uses . For , we upper bound by . This gives that
[TABLE]
For , we upper bound by . This gives that
[TABLE]
For , we note that is the probability that is not of rank when [LQ, Fact 4 in arXiv version]. This gives the bound
[TABLE]
Combining equations from 6.14 to 6.19, we have
[TABLE]
which concludes the proof. ∎
Remark 6.20* *(Upgrading to the linear algebraic Erdős-Rényi
model).
In [LQ], the linear algebraic Erdős-Rényi model, , was introduced as the uniform distribution over all -dimensional subspaces of . Randomly sampling -tuples of alternating matrices was termed as the naive model in [LQ]. It was also shown in [LQ] that the analysis in the naive model can be upgraded, with a mild loss in the parameters, to an analysis in . Such an upgrade can also be done similarly for the analysis here, though with a little bit more work than in [LQ]. We omit the details.
7 On testing isomorphism of groups with genus 2 radicals
In this section we show how to combine the methods of [GQ] for groups with abelian radicals and the methods of [BMW] to study subclasses of groups whose solvable radicals are -groups of class 2. Recall that -groups of class 2 are considered as difficult as the general case for group isomorphism, so we did not expect to beat the bound for this entire class. However, as a corollary of the results in this section, we give an -time isomorphism test for a class of groups whose radicals have genus 2. We shall work throughout with the following class of groups:
Let be the class of groups whose solvable radical, , is a -group of exponent and class 2 upon which acts as inner automorphisms of .
In [GQ] the classical strategy of using actions and cohomology was formally analyzed, showing that “splits” into two problems: Action Compatibility (), and Cohomology Class Isomorphism (); we state their definitions in the relevant sections below. When has a normal subgroup we may consider as an extension of by ; both and have as their witnesses certainly elements of , and two groups are isomorphic if, and only if, there is a witness that works simultaneously for and (see [GQ] for a leisurely exposition). Furthermore, and each reduce to .
The two key cases to handle first are the extreme situations with regards to this natural splitting: semi-direct products, where the isomorphism problems reduce to just ; and “central” products (or rather, where acts trivially on the radical ), where the problem reduces to (nonabelian) . The class that we consider here is of the second type of extreme situation. We expect the first yield to techniques in [GQ, Section 3], perhaps using methods to solve isometry [IQ], but we are not yet able to see a clear path to this case.
7.1 Preliminaries on genus 2 groups
We briefly recall definitions and results on the automorphism group of groups of genus 2; see [BMW] for details. For any group , let and ; then we define the commutator map of as . Two groups are isoclinic if there are isomorphisms and such that . When are nilpotent of class 2, their commutator maps are in fact -bilinear (note that in this case is abelian), and the groups are isoclinic iff and are pseudo-isometric, by definition (recall §2). Given a bilinear map ( abelian groups), its centroid is
[TABLE]
the centroid is the largest ring of scalars over which is bilinear. A nilpotent group of class 2 is isoclinic to a direct product of directly indecomposable groups; the genus of is the maximum rank of as a -module. Although the concept of genus is fully general, we focus on -groups of exponent and class 2; in this case isoclinism and isomorphism coincide, and centrally indecomposable -groups of class 2 and exponent have their centroids a finite field of characteristic . For a biadditive map , let denote its group of pseudo-isometries; if is bilinear over a field , let . Given a finite field of characteristic , its Galois group denoted , consists of those field automorphisms of that act trivially on the prime subfield ; is cyclic of order , generated by the Frobenius automorphism .
Proposition 7.1** (See, e. g., [BMW, Prop. 2.4]).**
Let be a -group of class 2 and exponent satisfying . Then . If is -bilinear, then , with quotient .
Note that elements of are faithfully represented by matrices \left(\begin{array}[]{cc}\alpha_{V}&d\alpha\\ 0&\alpha_{Z}\end{array}\right), where , , and is linear.
Recall that a map of -vector spaces is -semilinear if it is additive () and it is “twisted” linear, that is, , where . From the preceding, it follows immediately that:
Observation 7.2**.**
Let be a -group of class 2 and exponent such that is -bilinear. For any , the induced automorphisms on and are both -semilinear.
Observation 7.3**.**
If is a -group of class 2 and exponent such that , then , where is characteristic subgroup of and satisfies , and is an elementary abelian -group. Moreover, and and the isomorphism can be constructed in polynomial time in the number of generators, even when the groups are given as a black box.
Standard proof sketch.
since is of class 2. Since is of exponent , is elementary abelian, and thus is a vector space . Let be a generating set of . Let . Then . Let be a -linear complement to in . ∎
Theorem 7.4** ([BMW, IQ]).**
Let be a -group of class 2, exponent , and genus . Given , one can test whether extends to an automorphism in poly-logarithmic time when , and in polynomial time otherwise.
Proof.
When , the result is immediate from [BMW, Thm. 3.22], and their comments about its constructive nature (see [BMW, §6.2]). In general, this is an isometry problem, which is solvable in polynomial time [IQ]. ∎
Theorem 7.5**.**
Isomorphism of -groups of class 2, exponent , and genus can be decided in poly-logarithmic time [BMW, Thm. 1.1], and of genus can be decided in polynomial time [IQ, Thm. 3].
7.2 Testing isomorphism in the class
Our goal in this final section is to prove Theorem F, which for convenience we now recall:
Theorem F.
Let be the class of groups defined at the start of Section 7. Given groups of order , it can be decided in time if they lie in . If so, isomorphism can be decided, and a generating set for found, in time , where is the genus of .
We will need the following two results from Grochow–Qiao [GQ], which first require a few concepts we haven’t yet discussed. Recall that a pair of subgroups is a central decomposition of if and . Given two groups and an isomorphism between two subgroups , the quotient of by is the central product of and along , denoted , and is called the amalgamating map. In this case, is a central decomposition of ; conversely, if is a central decomposition of a group , then there exist and an isomorphism such that .
Lemma 7.6** ([GQ, Lem. 3.10]).**
Let , and suppose acts on as inner automorphisms of . Then there is a subgroup , constructible in time , such that , , and is a central decomposition of . We denote this subgroup by .
Proposition 7.7** (Special case of [GQ, Prop. 3.13]).**
Let () be a group such that is a -group of class 2, exponent , and genus 2, and such that acts on by inner automorphisms of . Suppose that (as in Lem. 7.6), which we denote by , and let be the corresponding amalgamating maps. Then iff there exist such that .
Proposition 7.8** (See [GQ, §6.1.2, p. 1186]).**
Let be a group with , and let an elementary abelian group. Given , one can compute in time a single and a basis of a linear subspace such that iff .
Proof of Thm. F.
Let be groups satisfying the hypotheses. In time, find and denote this by . By Lem. 7.6, construct and the amalgamating maps . Using Thm. 7.5 [BMW, IQ], decide whether ; if not, then and we can stop, and if so, then let be such an isomorphism.
Note (Observation 7.3) that it may be the case that for some abelian groups ; if this is the case, we can find and such that in polynomial time. Replace by and replace by ; this will not hurt us later because is characteristic in , and therefore also in . Intuitively, the only place that interacts with is as a direct product, and the only way interacts with is as a subgroup of its center, where still appears.
Next, since is a group with , by [GQ] we can decide whether in time ; if not, then and we can stop, and if so, let be such an isomorphism. Let and . These are both isomorphisms , so from now on we let and , and we have for .
Now, by Proposition 7.7, iff there exists such that
[TABLE]
By Observation 7.2, is -semilinear, and since has genus , . Enumerate ; for each , check whether extends to an automorphism of (Theorem 7.4 [BMW, IQ]). Let . Enumerate . For each that extends to an automorphism of , and each , we seek such that induces an automorphism of and satisfies (7.9). By Proposition 7.8, the set of such such that is an automorphism of is an affine linear space , where is a linear subspace of , and we can compute and a basis for in polynomial time. Once is fixed, (7.9) is linear in . Intersecting the linear space which solves (7.9) with the affine space is standard linear algebra, and can thus be computed in polynomial time.
To summarize, for each and each , we can compute a single element and generating set for those such that extends to an automorphism , , and satisfy (7.9). Taking the union over all choices in and gives us the coset of isomorphisms .
Analysis of running time. When , we have where , so their number is not too large, and is easily enumerated in time. By [BCGQ], can be listed in time . Since we are enumerating over both of these, we take their product , which ends up dominating the runtime. By [GQ], isomorphism of and can be tested in time. The rest is polynomial time or poly-logarithmic time by previous results, or linear algebra (poly-logarithmic time in ). ∎
Remark 7.10*.*
There is some hope when in Theorem F—due to the poly-logarithmic isomorphism test of [BMW]—to improve this poly-logarithmic time. However, a prerequisite is first solving isomorphism of groups with no abelian normal subgroups in poly-logarithmic time, rather than just polynomial [BCQ].
Acknowledgments
The authors would like to acknowledge V. Arvind and M. Grohe for useful comments on hypergraph -WL, Avinoam Mann for discussions on random generation of -groups, and László Babai and Xiaorui Sun for discussions on average-case algorithms for testing isomorphism of -groups of class and exponent . P. A. B. was partially supported by NSF grant DMS-1620362. J. A. G. was partially supported by NSF grant DMS-1750319. Y. L. was partially supported by ERC Consolidator Grant 615307-QPROGRESS. Y. Q. was partially supported by the Australian Research Council DECRA DE150100720. J. B. W. was partially supported by NSF grant DMS-1620454. P. A. B. and J. B. W. also acknowledge the Hausdorff Institute for Mathematics, and the University of Auckland where some of this research was conducted. P. A. B., J. A. G., J. B. W., and Y. Q. also acknowledge the Santa Fe Institute, where some of this research was conducted.
References