Recursive Inference Machines for Neural Reasoning

TL;DR

This paper introduces Recursive Inference Machines (RIMs), a neural reasoning framework inspired by classical inference engines, which enhances neural models' reasoning capabilities and achieves superior performance on various benchmarks.

Contribution

The paper presents RIMs as a novel neural reasoning framework that unifies and extends existing models like TRMs, incorporating recursive inference for improved reasoning performance.

Findings

RIMs outperform TRMs on reasoning benchmarks.

Reweighting in RIMs improves accuracy on complex tasks.

RIMs enhance reasoning in tabular data classification.

Abstract

Neural reasoners such as Tiny Recursive Models (TRMs) solve complex problems by combining neural backbones with specialized inference schemes. Such inference schemes have been a central component of stochastic reasoning systems, where inference rules are applied to a stochastic model to derive answers to complex queries. In this work, we bridge these two paradigms by introducing Recursive Inference Machines (RIMs), a neural reasoning framework that explicitly incorporates recursive inference mechanisms inspired by classical inference engines. We show that TRMs can be expressed as an instance of RIMs, allowing us to extend them through a reweighting component, yielding better performance on challenging reasoning benchmarks, including ARC-AGI-1, ARC-AGI-2, and Sudoku Extreme. Furthermore, we show that RIMs can be used to improve reasoning on other tasks, such as the classification of tabular data, outperforming TabPFNs.