Analyzing and Mitigating Object Hallucination: A Training Bias Perspective

TL;DR

This paper investigates the training data's role in object hallucination in large vision-language models, introduces a benchmark POPEv2, and proposes Obliviate, a lightweight fine-tuning method to mitigate hallucinations effectively.

Contribution

It reveals training bias as a key factor in hallucination, introduces POPEv2 benchmark, and develops Obliviate, a parameter-efficient unlearning method targeting the language modeling head.

Findings

Training bias causes hallucinations in LVLMs.

Obliviate reduces hallucinations by fine-tuning only the LM head.

Method scales well across model sizes and data volumes.

Abstract

As scaling up training data has significantly improved the general multimodal capabilities of Large Vision-Language Models (LVLMs), they still suffer from the hallucination issue, generating text that is inconsistent with the visual input. This phenomenon motivates us to systematically investigate the role of training data in hallucination. We introduce a new benchmark, POPEv2, which consists of counterfactual images collected from the training data of LVLMs with certain objects masked. Through comprehensive evaluation on POPEv2, we find that current LVLMs suffer from training bias: they fail to fully leverage their training data and hallucinate more frequently on images seen during training. Specifically, they perform poorly on counterfactual images, often incorrectly answering ``Yes'' to questions about masked objects. To understand this issue, we conduct probing experiments on the models' internal components, revealing that this training bias is primarily located in the language modeling (LM) head. Based on these findings, we propose Obliviate, an efficient and lightweight unlearning method designed to mitigate object hallucination via training bias unlearning. Obliviate identifies the discrepancy between ground-truth labels and model outputs on the training data as a proxy for bias and adopts a parameter- and data-efficient fine-tuning strategy that only updates the LM head. Extensive experiments demonstrate the effectiveness of our approach. While only reusing the training data and updating approximately 2\% of the parameters, Obliviate significantly reduces hallucination across both discriminative and generative tasks. Furthermore, it demonstrates strong scalability with respect to both model size (2B to 72B) and training data volume, and exhibits promising generalization to hallucination types beyond object-level hallucination. Our code and data will be publicly released.