RAL2M: Retrieval Augmented Learning-To-Match Against Hallucination in Compliance-Guaranteed Service Systems

TL;DR

This paper presents RAL2M, a retrieval-augmented framework that prevents hallucinations in LLM-based service systems by using LLMs as match judges within a retrieval system, improving reliability and compliance.

Contribution

Introducing RAL2M, a novel retrieval-augmented learning-to-match framework that replaces generation with matching to eliminate hallucinations in LLM-driven systems.

Findings

Significantly outperforms strong baselines on large-scale benchmarks.

Effectively leverages the 'wisdom of the crowd' through ensemble strategies.

Reduces hallucination by repositioning LLMs as judges rather than generators.

Abstract

Hallucination is a major concern in LLM-driven service systems, necessitating explicit knowledge grounding for compliance-guaranteed responses. In this paper, we introduce Retrieval-Augmented Learning-to-Match (RAL2M), a novel framework that eliminates generation hallucination by repositioning LLMs as query-response matching judges within a retrieval-based system, providing a robust alternative to purely generative approaches. To further mitigate judgment hallucination, we propose a query-adaptive latent ensemble strategy that explicitly models heterogeneous model competence and interdependencies among LLMs, deriving a calibrated consensus decision. Extensive experiments on large-scale benchmarks demonstrate that the proposed method effectively leverages the "wisdom of the crowd" and significantly outperforms strong baselines. Finally, we discuss best practices and promising directions for further exploiting latent representations in future work.