SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models

TL;DR

SLED is a decoding method that improves the factual accuracy of large language models by leveraging internal logits evolution, without external knowledge or additional fine-tuning.

Contribution

The paper introduces SLED, a novel decoding framework that uses latent knowledge within LLMs to enhance factuality through self-refinement, without external resources.

Findings

SLED consistently improves factual accuracy across multiple models and benchmarks.

SLED maintains fluency and introduces negligible latency overhead.

It can be combined with other decoding methods for further gains.

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities, but their outputs can sometimes be unreliable or factually incorrect. To address this, we introduce Self Logits Evolution Decoding (SLED), a novel decoding framework that enhances the truthfulness of LLMs without relying on external knowledge bases or requiring further fine-tuning. From an optimization perspective, our SLED framework leverages the latent knowledge embedded within the LLM by contrasting the output logits from the final layer with those from early layers. It then utilizes an approximate gradient approach to enable latent knowledge to guide the self-refinement of outputs, thereby effectively improving factual accuracy. Extensive experiments have been conducted on established benchmarks across a diverse range of model families (Gemma, Qwen, Mixtral, gpt-oss) and scales (from 1B to 45B), including more advanced architectural configurations such as the mixture of experts (MoE). Our evaluation spans a wide variety of tasks and the results demonstrate that SLED consistently improves factual accuracy compared to existing decoding methods while maintaining natural language fluency and negligible latency overhead. Furthermore, it can be flexibly combined with other decoding methods to further enhance their performance.