Learning the symmetric group: large from small
Max Petschack, Alexandr Garbali, Jan de Gier

TL;DR
This paper demonstrates that transformer models trained on smaller symmetric groups can generalize to larger groups, enabling scalable learning of complex mathematical structures with high accuracy.
Contribution
It introduces a scalable method for training models on simplified tasks that generalize to more complex symmetric group problems, using identity augmentation and partitioned windows.
Findings
Transformer trained on S10 generalizes to S25 with near 100% accuracy.
Model trained on S10 with adjacent transpositions generalizes to S16.
Identity augmentation effectively manages variable word lengths.
Abstract
Machine learning explorations can make significant inroads into solving difficult problems in pure mathematics. One advantage of this approach is that mathematical datasets do not suffer from noise, but a challenge is the amount of data required to train these models and that this data can be computationally expensive to generate. Key challenges further comprise difficulty in a posteriori interpretation of statistical models and the implementation of deep and abstract mathematical problems. We propose a method for scalable tasks, by which models trained on simpler versions of a task can then generalize to the full task. Specifically, we demonstrate that a transformer neural-network trained on predicting permutations from words formed by general transpositions in the symmetric group can generalize to the symmetric group with near 100\% accuracy. We also show that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
