Deep Equilibrium Algorithmic Reasoning

TL;DR

This paper introduces a novel approach to neural algorithmic reasoning by directly solving equilibrium equations, eliminating the need for iterative steps, and demonstrating improved performance on classical algorithm tasks.

Contribution

It proposes an equilibrium-based method for neural algorithmic reasoning that bypasses recurrent architectures and step-count supervision, enhancing efficiency and effectiveness.

Findings

Improved GNN performance on algorithm execution tasks.

Ability to train models without ground-truth step counts.

Validation on CLRS-30 benchmark algorithms.

Abstract

Neural Algorithmic Reasoning (NAR) research has demonstrated that graph neural networks (GNNs) could learn to execute classical algorithms. However, most previous approaches have always used a recurrent architecture, where each iteration of the GNN matches an iteration of the algorithm. In this paper we study neurally solving algorithms from a different perspective: since the algorithm's solution is often an equilibrium, it is possible to find the solution directly by solving an equilibrium equation. Our approach requires no information on the ground-truth number of steps of the algorithm, both during train and test time. Furthermore, the proposed method improves the performance of GNNs on executing algorithms and is a step towards speeding up existing NAR models. Our empirical evidence, leveraging algorithms from the CLRS-30 benchmark, validates that one can train a network to solve algorithmic problems by directly finding the equilibrium. We discuss the practical implementation of such models and propose regularisations to improve the performance of these equilibrium reasoners.