TL;DR
This paper describes the ATLANTIS team's approach to detecting hallucinated text spans in question answering, using various methods including prompting, fine-tuning, and synthetic data, achieving top results in multilingual settings.
Contribution
The paper introduces novel methods combining few-shot prompting, token classification, and synthetic data fine-tuning to detect hallucinations in LLM-generated answers.
Findings
Top ranking in Spanish hallucination detection
Competitive results in English and German
Highlighting the effectiveness of context integration
Abstract
This paper presents the contributions of the ATLANTIS team to SemEval-2025 Task 3, focusing on detecting hallucinated text spans in question answering systems. Large Language Models (LLMs) have significantly advanced Natural Language Generation (NLG) but remain susceptible to hallucinations, generating incorrect or misleading content. To address this, we explored methods both with and without external context, utilizing few-shot prompting with a LLM, token-level classification or LLM fine-tuned on synthetic data. Notably, our approaches achieved top rankings in Spanish and competitive placements in English and German. This work highlights the importance of integrating relevant context to mitigate hallucinations and demonstrate the potential of fine-tuned models and prompt engineering.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
