PUZZLES: A Benchmark for Neural Algorithmic Reasoning

TL;DR

PUZZLES is a new benchmark based on logic puzzles designed to evaluate and advance reinforcement learning algorithms in the domain of algorithmic and logical reasoning.

Contribution

The paper introduces PUZZLES, a comprehensive and diverse puzzle benchmark for testing RL agents' reasoning and generalization abilities, along with baseline evaluations.

Findings

RL algorithms show varied performance on PUZZLES

The benchmark reveals strengths and limitations of current RL methods in reasoning tasks

PUZZLES enables systematic evaluation of generalization in RL agents

Abstract

Algorithmic reasoning is a fundamental cognitive ability that plays a pivotal role in problem-solving and decision-making processes. Reinforcement Learning (RL) has demonstrated remarkable proficiency in tasks such as motor control, handling perceptual input, and managing stochastic environments. These advancements have been enabled in part by the availability of benchmarks. In this work we introduce PUZZLES, a benchmark based on Simon Tatham's Portable Puzzle Collection, aimed at fostering progress in algorithmic and logical reasoning in RL. PUZZLES contains 40 diverse logic puzzles of adjustable sizes and varying levels of complexity; many puzzles also feature a diverse set of additional configuration parameters. The 40 puzzles provide detailed information on the strengths and generalization capabilities of RL agents. Furthermore, we evaluate various RL algorithms on PUZZLES, providing baseline comparisons and demonstrating the potential for future research. All the software, including the environment, is available at https://github.com/ETH-DISCO/rlp.