Understanding and Evaluating Hallucinations in 3D Visual Language Models

TL;DR

This paper systematically studies hallucinations in 3D-LLMs, revealing their causes and proposing new metrics to evaluate and understand these inaccuracies in scene understanding models.

Contribution

It is the first comprehensive analysis of hallucinations in 3D-LLMs, identifying key causes and introducing novel evaluation metrics for these models.

Findings

All tested 3D-LLMs are significantly affected by hallucinations.

Main causes include dataset object frequency imbalance, object correlations, and limited attribute diversity.

Proposed new metrics effectively evaluate hallucination severity and model alignment.

Abstract

Recently, 3D-LLMs, which combine point-cloud encoders with large models, have been proposed to tackle complex tasks in embodied intelligence and scene understanding. In addition to showing promising results on 3D tasks, we found that they are significantly affected by hallucinations. For instance, they may generate objects that do not exist in the scene or produce incorrect relationships between objects. To investigate this issue, this work presents the first systematic study of hallucinations in 3D-LLMs. We begin by quickly evaluating hallucinations in several representative 3D-LLMs and reveal that they are all significantly affected by hallucinations. We then define hallucinations in 3D scenes and, through a detailed analysis of datasets, uncover the underlying causes of these hallucinations. We find three main causes: (1) Uneven frequency distribution of objects in the dataset. (2) Strong correlations between objects. (3) Limited diversity in object attributes. Additionally, we propose new evaluation metrics for hallucinations, including Random Point Cloud Pair and Opposite Question Evaluations, to assess whether the model generates responses based on visual information and aligns it with the text's meaning.