Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation

TL;DR

This paper introduces Active Layer-Contrastive Decoding (ActLCD), a new decoding method for large language models that reduces hallucinations by actively selecting contrasting layers during text generation, guided by reinforcement learning.

Contribution

The paper presents a novel decoding strategy, ActLCD, which uses reinforcement learning to decide when to apply contrasting layers, improving factual accuracy over existing methods.

Findings

ActLCD outperforms state-of-the-art decoding methods on five benchmarks.

It effectively reduces hallucinations in large language model outputs.

The approach enhances factual consistency in diverse generation scenarios.

Abstract

Recent decoding methods improve the factuality of large language models (LLMs) by refining how the next token is selected during generation. These methods typically operate at the token level, leveraging internal representations to suppress superficial patterns. Nevertheless, LLMs remain prone to hallucinations, especially over longer contexts. In this paper, we propose Active Layer-Contrastive Decoding (ActLCD), a novel decoding strategy that actively decides when to apply contrasting layers during generation. By casting decoding as a sequential decision-making problem, ActLCD employs a reinforcement learning policy guided by a reward-aware classifier to optimize factuality beyond the token level. Our experiments demonstrate that ActLCD surpasses state-of-the-art methods across five benchmarks, showcasing its effectiveness in mitigating hallucinations in diverse generation scenarios.