AutoRAG-LoRA: Hallucination-Triggered Knowledge Retuning via Lightweight Adapters

TL;DR

AutoRAG-LoRA is a modular framework that reduces hallucinations in large language models by integrating retrieval, lightweight adapters, and confidence-based feedback correction, improving factual accuracy without sacrificing efficiency.

Contribution

It introduces a novel retrieval-augmented approach with lightweight adapters and hallucination detection to enhance factual grounding in LLMs.

Findings

Significantly reduces factual hallucinations in LLM outputs.

Maintains model efficiency and modularity.

Effective hallucination detection and correction loop.

Abstract

Large Language Models (LLMs) have demonstrated remarkable fluency across a range of natural language tasks, yet remain vulnerable to hallucinations - factual inaccuracies that undermine trust in real world deployment. We present AutoRAG-LoRA, a modular framework for Retrieval-Augmented Generation (RAG) that tackles hallucination in large language models through lightweight LoRA-based adapters and KL-regularized training. Our pipeline integrates automated prompt rewriting, hybrid retrieval, and low-rank adapter tuning to ground responses in retrieved evidence. A hallucination detection module, using both classifier-based and self-evaluation techniques, assigns confidence scores to generated outputs, triggering an optional feedback correction loop. This loop enforces factual alignment via contrastive KL loss and adapter fine tuning. We demonstrate that AutoRAG-LoRA significantly reduces the factual drift while preserving the efficiency and modularity of the model.