On the Limits of Hierarchically Embedded Logic in Classical Neural Networks

TL;DR

This paper establishes theoretical limits on the logical reasoning capabilities of deep neural networks, showing they cannot represent higher-order logic beyond a certain depth, which explains some observed model behaviors.

Contribution

It introduces a formal model linking neural network depth to logical reasoning limits, providing a theoretical foundation for understanding their expressiveness and limitations.

Findings

Neural network depth bounds logical reasoning capacity.

Higher-order predicates cannot be faithfully represented in shallow networks.

The framework explains phenomena like hallucination and repetition in language models.

Abstract

We propose a formal model of reasoning limitations in large neural net models for language, grounded in the depth of their neural architecture. By treating neural networks as linear operators over logic predicate space we show that each layer can encode at most one additional level of logical reasoning. We prove that a neural network of depth a particular depth cannot faithfully represent predicates in a one higher order logic, such as simple counting over complex predicates, implying a strict upper bound on logical expressiveness. This structure induces a nontrivial null space during tokenization and embedding, excluding higher-order predicates from representability. Our framework offers a natural explanation for phenomena such as hallucination, repetition, and limited planning, while also providing a foundation for understanding how approximations to higher-order logic may emerge. These results motivate architectural extensions and interpretability strategies in future development of language models.