Learning the symmetric group: large from small

TL;DR

This paper demonstrates that transformer models trained on smaller symmetric groups can generalize to larger groups, enabling scalable learning of complex mathematical structures with high accuracy.

Contribution

It introduces a scalable method for training models on simplified tasks that generalize to more complex symmetric group problems, using identity augmentation and partitioned windows.

Findings

Transformer trained on S10 generalizes to S25 with near 100% accuracy.

Model trained on S10 with adjacent transpositions generalizes to S16.

Identity augmentation effectively manages variable word lengths.

Abstract

Machine learning explorations can make significant inroads into solving difficult problems in pure mathematics. One advantage of this approach is that mathematical datasets do not suffer from noise, but a challenge is the amount of data required to train these models and that this data can be computationally expensive to generate. Key challenges further comprise difficulty in a posteriori interpretation of statistical models and the implementation of deep and abstract mathematical problems. We propose a method for scalable tasks, by which models trained on simpler versions of a task can then generalize to the full task. Specifically, we demonstrate that a transformer neural-network trained on predicting permutations from words formed by general transpositions in the symmetric group $S_{10}$ can generalize to the symmetric group $S_{25}$ with near 100\% accuracy. We also show that $S_{10}$ generalizes to $S_{16}$ with similar performance if we only use adjacent transpositions. We employ identity augmentation as a key tool to manage variable word lengths, and partitioned windows for training on adjacent transpositions. Finally we compare variations of the method used and discuss potential challenges with extending the method to other tasks.