Deliberative Reasoning Network: An Uncertainty-Driven Paradigm for Belief-Tracked Inference with Pretrained Language Models

TL;DR

The paper introduces the Deliberative Reasoning Network (DRN), a new uncertainty-driven paradigm for logical reasoning in language models that improves interpretability, robustness, and transferability by explicitly tracking belief states and minimizing epistemic uncertainty.

Contribution

It presents DRN, a novel framework that shifts from probability maximization to uncertainty minimization, enhancing reasoning accuracy and interpretability in language models.

Findings

Up to 15.2% improvement on adversarial reasoning benchmark.

Boosts accuracy from 20% to 80% when integrated with Mistral-7B.

Improves TruthfulQA performance by 23.6% in zero-shot setting.

Abstract

Large language models often fail at logical reasoning when semantic heuristics conflict with decisive evidence - a phenomenon we term cognitive traps. To address this fundamental limitation, we introduce the Deliberative Reasoning Network (DRN), a novel paradigm that reframes logical reasoning from probability maximization to uncertainty minimization. Instead of asking "Which answer is most likely?", DRN asks "Which hypothesis has the most internally consistent evidence?". DRN achieves intrinsic interpretability by explicitly tracking belief states and quantifying epistemic uncertainty for competing hypotheses through an iterative evidence synthesis process. We validate our approach through two complementary architectures - a bespoke discriminative model that embodies the core uncertainty minimization principle, and a lightweight verification module that enhances existing generative LLMs. Evaluated on LCR-1000, our new adversarial reasoning benchmark designed to expose cognitive traps, the bespoke DRN achieves up to 15.2% improvement over standard baselines. When integrated as a parameter-efficient verifier with Mistral-7B, our hybrid system boosts accuracy from 20% to 80% on the most challenging problems. Critically, DRN demonstrates strong zero-shot generalization, improving TruthfulQA performance by 23.6% without additional training, indicating that uncertainty-driven deliberation learns transferable reasoning principles. We position DRN as a foundational, verifiable System 2 reasoning component for building more trustworthy AI systems.