CayleyPy RL: Pathfinding and Reinforcement Learning on Cayley Graphs

TL;DR

This paper introduces CayleyPy RL, a reinforcement learning approach for pathfinding on large Cayley graphs, demonstrating its effectiveness over classical methods and supporting mathematical conjectures through machine learning.

Contribution

It combines reinforcement learning with diffusion distances, benchmarks various neural architectures, and applies the method to mathematical problems like the Cayley graph of the symmetric group.

Findings

Reinforcement learning methods outperform GAP in specific examples.

Supports the OEIS-A186783 conjecture on graph diameter.

Provides bounds and conjectures on Cayley graph properties.

Abstract

This paper is the second in a series of studies on developing efficient artificial intelligence-based approaches to pathfinding on extremely large graphs (e.g. $10^{70}$ nodes) with a focus on Cayley graphs and mathematical applications. The open-source CayleyPy project is a central component of our research. The present paper proposes a novel combination of a reinforcement learning approach with a more direct diffusion distance approach from the first paper. Our analysis includes benchmarking various choices for the key building blocks of the approach: architectures of the neural network, generators for the random walks and beam search pathfinding. We compared these methods against the classical computer algebra system GAP, demonstrating that they "overcome the GAP" for the considered examples. As a particular mathematical application we examine the Cayley graph of the symmetric group with cyclic shift and transposition generators. We provide strong support for the OEIS-A186783 conjecture that the diameter is equal to n(n-1)/2 by machine learning and mathematical methods. We identify the conjectured longest element and generate its decomposition of the desired length. We prove a diameter lower bound of n(n-1)/2-n/2 and an upper bound of n(n-1)/2+ 3n by presenting the algorithm with given complexity. We also present several conjectures motivated by numerical experiments, including observations on the central limit phenomenon (with growth approximated by a Gumbel distribution), the uniform distribution for the spectrum of the graph, and a numerical study of sorting networks. To stimulate crowdsourcing activity, we create challenges on the Kaggle platform and invite contributions to improve and benchmark approaches on Cayley graph pathfinding and other tasks.