DecoPrompt : Decoding Prompts Reduces Hallucinations when Large Language Models Meet False Premises

TL;DR

DecoPrompt is a novel prompting technique that reduces hallucinations in large language models caused by false premises by decoding prompts without generating hallucinated outputs, improving factual accuracy.

Contribution

Introduces DecoPrompt, a decoding-based prompting method that mitigates hallucinations from false-premise prompts in LLMs, with demonstrated cross-model effectiveness.

Findings

Reduces hallucinations across multiple LLMs

Effective on different datasets and models

Exhibits cross-model transferability

Abstract

While large language models (LLMs) have demonstrated increasing power, they have also called upon studies on their hallucinated outputs that deviate from factually correct statements. In this paper, we focus on one important scenario of false premises, where LLMs are distracted by misaligned claims although the model possesses the required factual knowledge to answer original questions accurately. Inspired by the observation that entropy of the false-premise prompt is closely related to its likelihood to elicit hallucination generation, we propose a new prompting algorithm, named DecoPrompt, to mitigate hallucination. DecoPrompt leverages LLMs to "decode" the false-premise prompts without really eliciting hallucination output from LLMs. We perform experiments on two datasets, demonstrating that DecoPrompt can reduce hallucinations effectively on outputs from different LLMs. Moreover, DecoPrompt exhibits cross-model transferability, which facilitates its applications to scenarios such as LLMs of large sizes or unavailable model logits.