Modeling Next-Token Prediction as Left-Nested Intuitionistic Implication

TL;DR

This paper proposes a novel neural architecture called the Arrow Language Model, which interprets next-token prediction through intuitionistic logic, encoding sequences as nested implications and validating properties with theorem provers.

Contribution

It introduces a logic-inspired neural model based on intuitionistic implication, offering a new perspective on sequence modeling and connecting proof theory with neural architectures.

Findings

Neural architecture aligns with multiplicative RNNs.

Validates properties using Prolog-based theorem provers.

Positions model as an alternative to Transformers and state-space models.

Abstract

We introduce the \emph{Arrow Language Model}, a neural architecture derived from an intuitionistic-logic interpretation of next-token prediction. Instead of representing tokens as additive embeddings mixed by attention, we encode a prefix as a \emph{left-nested implication chain} whose structure preserves order through non-commutative composition. Next-token prediction corresponds to \emph{modus ponens}, and sequence processing becomes constructive proof extension under the Curry--Howard correspondence. Our Prolog-based specialized theorem provers validate fundamental properties of the neural models, among which relations between commutative vs. non-commutative sequencing and single-token vs. multi-token prediction choices. We show that a neural architecture equivalent to multiplicative RNNs arises naturally from a proof-theoretic interpretation of next-token prediction as nested intuitionistic implication, we present a practical low-rank neural realization and position the model relative to Transformers and state-space models. Keywords: logic-based derivation of neural architectures, intuitionistic implicational logic, token-as-operator neural models, state-space models, alternatives to transformer-based foundational models.