Finding path and cycle counting formulae in graphs with Deep Reinforcement Learning

TL;DR

This paper introduces Grammar Reinforcement Learning (GRL), a novel method combining MCTS and transformer architectures to discover efficient matrix-based formulas for counting paths and cycles in graphs, outperforming existing methods.

Contribution

The paper develops a new framework for generating gramformers within CFGs, applying GRL to optimize formulas for graph substructure counting, and discovering novel formulas that enhance computational efficiency.

Findings

GRL discovers new formulas for path and cycle counting.

Formulas improve computational efficiency by factors of 2 to 6.

The approach outperforms state-of-the-art methods.

Abstract

This paper presents Grammar Reinforcement Learning (GRL), a reinforcement learning algorithm that uses Monte Carlo Tree Search (MCTS) and a transformer architecture that models a Pushdown Automaton (PDA) within a context-free grammar (CFG) framework. Taking as use case the problem of efficiently counting paths and cycles in graphs, a key challenge in network analysis, computer science, biology, and social sciences, GRL discovers new matrix-based formulas for path/cycle counting that improve computational efficiency by factors of two to six w.r.t state-of-the-art approaches. Our contributions include: (i) a framework for generating gramformers that operate within a CFG, (ii) the development of GRL for optimizing formulas within grammatical structures, and (iii) the discovery of novel formulas for graph substructure counting, leading to significant computational improvements.