EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understanding
Ashish Seth, Utkarsh Tyagi, Ramaneswaran Selvakumar, Nishit Anand, Sonal Kumar, Sreyan Ghosh, Ramani Duraiswami, Chirag Agarwal, Dinesh Manocha

TL;DR
EgoIllusion is a new benchmark designed to evaluate hallucination tendencies in multimodal large language models when understanding egocentric videos, highlighting current challenges and guiding future improvements.
Contribution
This paper introduces EgoIllusion, the first benchmark specifically targeting hallucinations in egocentric video understanding by MLLMs, with comprehensive evaluation data.
Findings
Current MLLMs achieve only 59% accuracy on hallucination detection.
Significant hallucination challenges remain in egocentric video understanding.
Benchmark will be open-sourced for community use.
Abstract
Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in complex multimodal tasks. While MLLMs excel at visual perception and reasoning in third-person and egocentric videos, they are prone to hallucinations, generating coherent yet inaccurate responses. We present EgoIllusion, a first benchmark to evaluate MLLM hallucinations in egocentric videos. EgoIllusion comprises 1,400 videos paired with 8,000 human-annotated open and closed-ended questions designed to trigger hallucinations in both visual and auditory cues in egocentric videos. Evaluations across ten MLLMs reveal significant challenges, including powerful models like GPT-4o and Gemini, achieving only 59% accuracy. EgoIllusion lays the foundation in developing robust benchmarks to evaluate the effectiveness of MLLMs and spurs the development of better egocentric MLLMs with reduced hallucination rates.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsMental Health and Psychiatry · Schizophrenia research and treatment · Mental Health Research Topics
