Recurrent Aggregators in Neural Algorithmic Reasoning

TL;DR

This paper introduces Recurrent Aggregators in neural algorithmic reasoning, replacing traditional permutation-equivariant functions with recurrent neural networks, leading to state-of-the-art results on classical algorithmic tasks.

Contribution

It proposes a novel recurrent aggregation approach for neural algorithmic reasoning, challenging the standard GNN design and demonstrating superior performance on key benchmarks.

Findings

Achieves state-of-the-art results on Heapsort and Quickselect tasks.

Outperforms existing neural reasoners on established benchmarks.

Handles tasks with natural node ordering effectively.

Abstract

Neural algorithmic reasoning (NAR) is an emerging field that seeks to design neural networks that mimic classical algorithmic computations. Today, graph neural networks (GNNs) are widely used in neural algorithmic reasoners due to their message passing framework and permutation equivariance. In this extended abstract, we challenge this design choice, and replace the equivariant aggregation function with a recurrent neural network. While seemingly counter-intuitive, this approach has appropriate grounding when nodes have a natural ordering -- and this is the case frequently in established reasoning benchmarks like CLRS-30. Indeed, our recurrent NAR (RNAR) model performs very strongly on such tasks, while handling many others gracefully. A notable achievement of RNAR is its decisive state-of-the-art result on the Heapsort and Quickselect tasks, both deemed as a significant challenge for contemporary neural algorithmic reasoners -- especially the latter, where RNAR achieves a mean micro-F1 score of 87%.