Truth or Deceit? A Bayesian Decoding Game Enhances Consistency and Reliability

TL;DR

This paper introduces a Bayesian game-theoretic decoding method for large language models that improves output consistency and reliability without extra training, outperforming larger models in truthfulness and coherence.

Contribution

It presents a novel multistage Bayesian decoding game that enhances LLM output consistency and reliability through dynamic consensus and ambiguity calibration, without human feedback.

Findings

Smaller models outperform larger ones in truthfulness (e.g., LLaMA13B vs PaLM540B).

The method improves consistency and reliability in LLM outputs.

The approach integrates various strategies and models using game-theoretic tools.

Abstract

Large Language Models (LLMs) often produce outputs that -- though plausible -- can lack consistency and reliability, particularly in ambiguous or complex scenarios. Challenges arise from ensuring that outputs align with both factual correctness and human intent. This is problematic in existing approaches that trade improved consistency for lower accuracy. To mitigate these challenges, we propose a novel game-theoretic approach to enhance consistency and reliability during the decoding stage of LLM output generation. Our method models the decoding process as a multistage Bayesian decoding game. This ensures consistency through Correctness Alignment and enhances reliability via Ambiguity Calibration. The model dynamically converges to a consensus on the most reliable outputs and distinguishes {Valid, Specious} outputs without human feedback or additional training. Our game design allows smaller models to outperform much larger models through game mechanisms (e.g., 78.1 LLaMA13B vs 76.6 PaLM540B), as well as integrating various LL strategies and models, demonstrating the potential of game-theoretic tools to improve the truthfulness and reliability of LLMs.