Iter-AHMCL: Alleviate Hallucination for Large Language Model via Iterative Model-level Contrastive Learning

TL;DR

This paper proposes Iter-AHMCL, an iterative contrastive learning method that reduces hallucinations in large language models by contrasting models trained on hallucinated and non-hallucinated data, improving factual accuracy.

Contribution

It introduces a novel iterative contrastive learning approach at the model level to effectively mitigate hallucinations in pre-trained LLMs without sacrificing their capabilities.

Findings

Achieves an average of 10.1 points improvement on TruthfulQA benchmark.

Effectively reduces hallucinations while maintaining LLM performance.

Validated on four different pre-trained foundation LLMs.

Abstract

The development of Large Language Models (LLMs) has significantly advanced various AI applications in commercial and scientific research fields, such as scientific literature summarization, writing assistance, and knowledge graph construction. However, a significant challenge is the high risk of hallucination during LLM inference, which can lead to security concerns like factual inaccuracies, inconsistent information, and fabricated content. To tackle this issue, it is essential to develop effective methods for reducing hallucination while maintaining the original capabilities of the LLM. This paper introduces a novel approach called Iterative Model-level Contrastive Learning (Iter-AHMCL) to address hallucination. This method modifies the representation layers of pre-trained LLMs by using contrastive `positive' and `negative' models, trained on data with and without hallucinations. By leveraging the differences between these two models, we create a more straightforward pathway to eliminate hallucinations, and the iterative nature of contrastive learning further enhances performance. Experimental validation on four pre-trained foundation LLMs (LLaMA2, Alpaca, LLaMA3, and Qwen) finetuning with a specially designed dataset shows that our approach achieves an average improvement of 10.1 points on the TruthfulQA benchmark. Comprehensive experiments demonstrate the effectiveness of Iter-AHMCL in reducing hallucination while maintaining the general capabilities of LLMs.