Algorithmic Language Models with Neurally Compiled Libraries

TL;DR

This paper proposes augmenting large language models with differentiable libraries of fundamental operations and algorithms, enabling more robust reasoning and planning capabilities by directly compiling algorithms into the model architecture.

Contribution

It introduces a method for integrating differentiable algorithms into LLMs, enhancing their ability to perform algorithmic tasks without learning from scratch.

Findings

Feasibility demonstrated with small transformers on simple tasks

Augmentation improves reasoning capabilities

Direct compilation of algorithms into models

Abstract

Important tasks such as reasoning and planning are fundamentally algorithmic, meaning that solving them robustly requires acquiring true reasoning or planning algorithms, rather than shortcuts. Large Language Models lack true algorithmic ability primarily because of the limitations of neural network optimization algorithms, their optimization data and optimization objective, but also due to architectural inexpressivity. To solve this, our paper proposes augmenting LLMs with a library of fundamental operations and sophisticated differentiable programs, so that common algorithms do not need to be learned from scratch. We add memory, registers, basic operations, and adaptive recurrence to a transformer architecture built on LLaMA3. Then, we define a method for directly compiling algorithms into a differentiable starting library, which is used natively and propagates gradients for optimization. In this preliminary study, we explore the feasability of augmenting LLaMA3 with a differentiable computer, for instance by fine-tuning small transformers on simple algorithmic tasks with variable computational depth.