APCD: Adaptive Path-Contrastive Decoding for Reliable Large Language Model Generation

TL;DR

APCD introduces an adaptive multi-path decoding framework for large language models that enhances reliability and factual accuracy by dynamically managing exploration and interaction among multiple decoding paths.

Contribution

The paper presents a novel adaptive multi-path decoding method with entropy-based branching and divergence-aware path contrast to improve LLM generation quality.

Findings

Improved factual accuracy on eight benchmarks.

Maintains decoding efficiency with adaptive exploration.

Effectively manages inter-path interactions for reliable outputs.

Abstract

Large language models (LLMs) often suffer from hallucinations due to error accumulation in autoregressive decoding, where suboptimal early token choices misguide subsequent generation. Although multi-path decoding can improve robustness by exploring alternative trajectories, existing methods lack principled strategies for determining when to branch and how to regulate inter-path interactions. We propose Adaptive Path-Contrastive Decoding (APCD), a multi-path decoding framework that improves output reliability through adaptive exploration and controlled path interaction. APCD consists of two components: (1) Entropy-Driven Path Expansion, which delays branching until predictive uncertainty - measured by Shannon entropy over top candidate tokens - indicates multiple plausible continuations; and (2) Divergence-Aware Path Contrast, which encourages diverse reasoning trajectories while dynamically attenuating inter-path influence as prediction distributions diverge. Experiments on eight benchmarks demonstrate improved factual accuracy while maintaining decoding efficiency. Our code is available at https://github.com/zty-king/APCD.