Evaluation and Analysis of Hallucination in Large Vision-Language Models

TL;DR

This paper introduces HaELM, a cost-effective, reproducible framework using large language models to evaluate hallucination in large vision-language models, providing insights and mitigation strategies for this critical issue.

Contribution

The paper presents HaELM, a novel LLM-based framework for hallucination evaluation in LVLMs, with high performance and practical advantages, and offers analysis and suggestions to reduce hallucination.

Findings

HaELM achieves ~95% performance of ChatGPT in hallucination evaluation.

Evaluation reveals significant hallucination issues in current LVLMs.

Analysis identifies key factors contributing to hallucination.

Abstract

Large Vision-Language Models (LVLMs) have recently achieved remarkable success. However, LVLMs are still plagued by the hallucination problem, which limits the practicality in many scenarios. Hallucination refers to the information of LVLMs' responses that does not exist in the visual input, which poses potential risks of substantial consequences. There has been limited work studying hallucination evaluation in LVLMs. In this paper, we propose Hallucination Evaluation based on Large Language Models (HaELM), an LLM-based hallucination evaluation framework. HaELM achieves an approximate 95% performance comparable to ChatGPT and has additional advantages including low cost, reproducibility, privacy preservation and local deployment. Leveraging the HaELM, we evaluate the hallucination in current LVLMs. Furthermore, we analyze the factors contributing to hallucination in LVLMs and offer helpful suggestions to mitigate the hallucination problem. Our training data and human annotation hallucination data will be made public soon.