Exact Learning of Arithmetic with Differentiable Agents

TL;DR

This paper introduces a differentiable framework using Finite-State Transducers for exact algorithmic learning, demonstrating strong length generalization in arithmetic tasks with gradient-based methods.

Contribution

It presents a novel differentiable model family, DFSTs, enabling exact learning of algorithms with length generalization using structured supervision.

Findings

Models trained on small datasets generalize to much longer inputs.

DFSTs achieve error-free performance on arithmetic tasks.

End-to-end differentiable training of algorithmic skills is feasible.

Abstract

We explore the possibility of exact algorithmic learning with gradient-based methods and introduce a differentiable framework capable of strong length generalization on arithmetic tasks. Our approach centers on Differentiable Finite-State Transducers (DFSTs), a Turing-complete model family that avoids the pitfalls of prior architectures by enabling constant-precision, constant-time generation, and end-to-end log-parallel differentiable training. Leveraging policy-trajectory observations from expert agents, we train DFSTs to perform binary and decimal addition and multiplication. Remarkably, models trained on tiny datasets generalize without error to inputs thousands of times longer than the training examples. These results show that training differentiable agents on structured intermediate supervision could pave the way towards exact gradient-based learning of algorithmic skills. Code available at \href{https://github.com/dngfra/differentiable-exact-algorithmic-learner.git}{https://github.com/dngfra/differentiable-exact-algorithmic-learner.git}.