Discovering Algorithms with Computational Language Processing

TL;DR

This paper introduces a novel framework that automates the discovery of algorithms by representing them as token sequences, using reinforcement learning-guided Monte Carlo tree search to generate and improve algorithms for complex problems.

Contribution

The framework uniquely combines token-based algorithm representation with RL-guided MCTS to discover and optimize algorithms tailored to specific problem instances.

Findings

Outperforms existing methods on NP-hard combinatorial problems

Rediscovers and improves quantum algorithms like Grover's and QAOA

Generates algorithms tailored to individual problem instances

Abstract

Algorithms are the engine for reproducible problem-solving. We present a framework automating algorithm discovery by conceptualizing them as sequences of operations, represented as tokens. These computational tokens are chained using a grammar, enabling the formation of increasingly sophisticated procedures. Our ensemble Monte Carlo tree search (MCTS) guided by reinforcement learning (RL) explores token chaining and drives the creation of new tokens. This methodology rediscovers, improves, and generates new algorithms that substantially outperform existing methods for strongly NP-hard combinatorial optimization problems and foundational quantum computing approaches such as Grover's and Quantum Approximate Optimization Algorithm. Operating at the computational rather than code-generation level, our framework produces algorithms that can be tailored specifically to problem instances, not merely classes.