Symbol-Equivariant Recurrent Reasoning Models

TL;DR

The paper introduces Symbol-Equivariant Recurrent Reasoning Models (SE-RRMs) that explicitly encode symbol symmetries, leading to improved reasoning performance and generalization in structured problems like Sudoku and ARC-AGI.

Contribution

It proposes SE-RRMs that enforce permutation equivariance at the architectural level, enhancing robustness and scalability over prior RRMs.

Findings

SE-RRMs outperform prior RRMs on 9x9 Sudoku.

SE-RRMs generalize to different Sudoku sizes without retraining.

They achieve competitive results on ARC-AGI tasks with fewer parameters.

Abstract

Reasoning problems such as Sudoku and ARC-AGI remain challenging for neural networks. The structured problem solving architecture family of Recurrent Reasoning Models (RRMs), including Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM), offer a compact alternative to large language models, but currently handle symbol symmetries only implicitly via costly data augmentation. We introduce Symbol-Equivariant Recurrent Reasoning Models (SE-RRMs), which enforce permutation equivariance at the architectural level through symbol-equivariant layers, guaranteeing identical solutions under symbol or color permutations. SE-RRMs outperform prior RRMs on 9x9 Sudoku and generalize from just training on 9x9 to smaller 4x4 and larger 16x16 and 25x25 instances, to which existing RRMs cannot extrapolate. On ARC-AGI-1 and ARC-AGI-2, SE-RRMs achieve competitive performance with substantially less data augmentation and only 2 million parameters, demonstrating that explicitly encoding symmetry improves the robustness and scalability of neural reasoning. Code is available at https://github.com/ml-jku/SE-RRM.