Distorted or Fabricated? A Survey on Hallucination in Video LLMs

TL;DR

This survey analyzes hallucinations in Video Large Language Models, categorizing them into dynamic distortion and content fabrication, and reviews evaluation, mitigation strategies, and future research directions.

Contribution

It introduces a systematic taxonomy of hallucinations in Vid-LLMs and consolidates recent advances in evaluation and mitigation techniques.

Findings

Hallucinations are categorized into dynamic distortion and content fabrication.

Root causes include limited temporal representation and visual grounding.

Promising future directions involve motion-aware encoders and counterfactual learning.

Abstract

Despite significant progress in video-language modeling, hallucinations remain a persistent challenge in Video Large Language Models (Vid-LLMs), referring to outputs that appear plausible yet contradict the content of the input video. This survey presents a comprehensive analysis of hallucinations in Vid-LLMs and introduces a systematic taxonomy that categorizes them into two core types: dynamic distortion and content fabrication, each comprising two subtypes with representative cases. Building on this taxonomy, we review recent advances in the evaluation and mitigation of hallucinations, covering key benchmarks, metrics, and intervention strategies. We further analyze the root causes of dynamic distortion and content fabrication, which often result from limited capacity for temporal representation and insufficient visual grounding. These insights inform several promising directions for future work, including the development of motion-aware visual encoders and the integration of counterfactual learning techniques. This survey consolidates scattered progress to foster a systematic understanding of hallucinations in Vid-LLMs, laying the groundwork for building robust and reliable video-language systems. An up-to-date curated list of related works is maintained at https://github.com/hukcc/Awesome-Video-Hallucination .