Dual-Density Inference for Efficient Language Model Reasoning

TL;DR

This paper introduces Denser, a framework that separates information density for reasoning and answering in language models, significantly reducing token use while maintaining or improving accuracy in complex reasoning tasks.

Contribution

It proposes a novel dual-density inference framework that optimizes information density separately for reasoning and answering phases, enhancing efficiency.

Findings

Reduces token consumption by up to 62% compared to Chain-of-Thought methods.

Maintains or improves accuracy across multiple reasoning benchmarks.

Significantly benefits complex multi-step reasoning tasks.

Abstract

Large Language Models (LLMs) have shown impressive capabilities in complex reasoning tasks. However, current approaches employ uniform language density for both intermediate reasoning and final answers, leading to computational inefficiency. Our observation found that reasoning process serves a computational function for the model itself, while answering serves a communicative function for human understanding. This distinction enables the use of compressed, symbol-rich language for intermediate computations while maintaining human-readable final explanations. To address this inefficiency, we present Denser: \underline{D}ual-d\underline{ens}ity inf\underline{er}ence, a novel framework that optimizes information density separately for reasoning and answering phases. Our framework implements this through three components: a query processing module that analyzes input problems, a high-density compressed reasoning mechanism for efficient intermediate computations, and an answer generation component that translates compressed reasoning into human-readable solutions. Experimental evaluation across multiple reasoning question answering benchmarks demonstrates that Denser reduces token consumption by up to 62\% compared to standard Chain-of-Thought methods while preserving or improving accuracy. These efficiency gains are particularly significant for complex multi-step reasoning problems where traditional methods generate extensive explanations.