MIH-TCCT: Mitigating Inconsistent Hallucinations in LLMs via Event-Driven Text-Code Cyclic Training

TL;DR

This paper introduces MIH-TCCT, a cyclic training framework that uses event-driven text and code to reduce hallucinations in large language models across various tasks without task-specific adaptation.

Contribution

The proposed method leverages event-based text and code cyclic training to transfer logical consistency, effectively reducing hallucinations in LLMs while maintaining performance.

Findings

Significantly reduces hallucinations across multiple LLMs and tasks

Maintains overall performance while improving consistency

Demonstrates generality without task-specific adaptation

Abstract

Recent methodologies utilizing synthetic datasets have aimed to address inconsistent hallucinations in large language models (LLMs); however,these approaches are primarily tailored to specific tasks, limiting their generalizability. Inspired by the strong performance of code-trained models in logic-intensive domains, we propose a novel framework that leverages event-based text to generate corresponding code and employs cyclic training to transfer the logical consistency of code to natural language effectively. Our method significantly reduces inconsistent hallucinations across three leading LLMs and two categories of natural language tasks while maintaining overall performance. This framework effectively alleviates hallucinations without necessitating adaptation to downstream tasks, demonstrating generality and providing new perspectives to tackle the challenge of inconsistent hallucinations.