Hallucination Detection for Grounded Instruction Generation

TL;DR

This paper presents a model that detects hallucinated references in instructions for navigation in simulated environments, improving accuracy over existing methods by fine-tuning a pre-trained image-text model with contrastive learning.

Contribution

It introduces a novel approach for hallucination detection in grounded instructions using a pre-trained model and contrastive fine-tuning, outperforming several baselines.

Findings

The proposed model outperforms baseline methods in hallucination detection.

Contrastive fine-tuning improves detection accuracy.

Pre-trained image-text models are effective for this task.

Abstract

We investigate the problem of generating instructions to guide humans to navigate in simulated residential environments. A major issue with current models is hallucination: they generate references to actions or objects that are inconsistent with what a human follower would perform or encounter along the described path. We develop a model that detects these hallucinated references by adopting a model pre-trained on a large corpus of image-text pairs, and fine-tuning it with a contrastive loss that separates correct instructions from instructions containing synthesized hallucinations. Our final model outperforms several baselines, including using word probability estimated by the instruction-generation model, and supervised models based on LSTM and Transformer.