FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs

TL;DR

FREAK is a new benchmark for detailed, fine-grained evaluation of hallucinations in multimodal large language models, revealing significant hallucination issues and providing insights into model reasoning and perception.

Contribution

This paper introduces FREAK, a comprehensive and high-quality multimodal benchmark specifically designed to assess fine-grained hallucinations in MLLMs, addressing limitations of previous benchmarks.

Findings

State-of-the-art models exhibit severe hallucination issues in detailed visual perception.

A curated subset enables indirect evaluation of models' perception of detailed information.

Analysis of Chain-of-Thought prompting reveals patterns in hallucination and reasoning.

Abstract

Multimodal Large Language Models (MLLMs) suffer from hallucinations. Existing hallucination evaluation benchmarks are often limited by over-simplified tasks leading to saturated metrics, or insufficient diversity that fails to adequately assess the hallucination extent in state-of-the-art multimodal models. To address this gap, we propose FREAK, a comprehensive multimodal benchmark designed for fine-grained hallucination assessment in MLLMs. Through high-quality photorealistic images featuring fine-grained counter-commonsense edits, FREAK innovatively evaluates hallucination phenomena in detailed visual perception of MLLMs. Extensive experiments on FREAK show severe hallucination issues in SOTA models regarding detailed visual perception. To enable deeper investigation, we curate a controlled subset to indirectly evaluate the model's ability to perceive target detailed information. Through systematic evaluation of prevailing Chain-of-Thought (CoT) prompting techniques within this task, we reveal critical insights regarding hallucination patterns and model reasoning processes.