Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models

TL;DR

This paper introduces a unified framework and benchmark for evaluating both object and relation hallucinations in large vision-language models, revealing that relation hallucination is a significant and overlooked issue.

Contribution

The paper proposes a triplet-based evaluation framework and a new benchmark, Tri-HE, to measure object and relation hallucinations simultaneously in LVLMs, and offers a simple mitigation method.

Findings

Relation hallucination is more severe than object hallucination in LVLMs.

The proposed framework generalizes across various vision-language tasks.

A training-free approach effectively reduces hallucinations.

Abstract

Despite the outstanding performance in vision-language reasoning, Large Vision-Language Models (LVLMs) might generate hallucinated contents that do not exist in the given image. Most existing LVLM hallucination benchmarks are constrained to evaluate the object-related hallucinations. However, the potential hallucination on the relations between two objects, i.e., relation hallucination, still lacks investigation. To remedy that, we design a unified framework to measure the object and relation hallucination in LVLMs simultaneously. The core idea of our framework is to evaluate hallucinations via (object, relation, object) triplets extracted from LVLMs' responses, making it easily generalizable to different vision-language tasks. Based on our framework, we further introduce Tri-HE, a novel Triplet-level Hallucination Evaluation benchmark which can be used to study both object and relation hallucination at the same time. With comprehensive evaluations on Tri-HE, we observe that the relation hallucination issue is even more serious than object hallucination among existing LVLMs, highlighting a previously neglected problem towards reliable LVLMs. Moreover, based on our findings, we design a simple training-free approach that effectively mitigates hallucinations for LVLMs. Our dataset and code for the reproduction of our experiments are available publicly at https://github.com/wujunjie1998/Tri-HE.