Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination

TL;DR

Pensieve is a training-free method that reduces visual hallucinations in multi-modal large language models by comparing test images with similar reference images and adaptively balancing confidence scores, improving accuracy and detail in responses.

Contribution

This paper introduces Pensieve, a novel approach that mitigates visual hallucination in MLLMs without additional training by leveraging analogous images and confidence score subtraction.

Findings

Pensieve outperforms existing decoding strategies in hallucination mitigation.

It improves the specificity of image descriptions generated by MLLMs.

The method effectively identifies visual details in various benchmarks.

Abstract

Multi-modal Large Language Models (MLLMs) demonstrate remarkable success across various vision-language tasks. However, they suffer from visual hallucination, where the generated responses diverge from the provided image. Are MLLMs oblivious to the accurate visual cues when they hallucinate? Our investigation reveals that the visual branch may equally advocate both accurate and erroneous content. To address this issue, we propose Pensieve, a training-free method that leverages the analogous visual hallucinations, which are induced by images sharing common semantic and appearance characteristics, to mitigate hallucination. Specifically, Pensieve enables MLLMs to retrospect relevant images as references and compare their visual content with the test image via confidence score subtraction. Moreover, our paradigm balances the effects of addressing errors from both the visual and textual branches by adaptively scaling the subtracted scores. Experiments on Whoops, LLaVA Bench, POPE, and MME demonstrate the efficacy of Pensieve in mitigating visual hallucination, surpassing other advanced decoding strategies. Pensieve also aids MLLMs in identifying visual details and enhance the specificity of generated image descriptions.