Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification

TL;DR

This paper introduces Evidential Uncertainty Quantification (EUQ), a novel method for detecting misbehaviors in large vision-language models by analyzing internal conflict and ignorance, improving reliability in critical applications.

Contribution

The paper proposes EUQ, a fine-grained uncertainty quantification approach that captures both conflict and ignorance, enhancing detection of model misbehaviors over existing methods.

Findings

EUQ outperforms strong baselines across multiple misbehavior categories.

Hallucinations are linked to high internal conflict.

Out-of-distribution failures are associated with high ignorance.

Abstract

%Large vision-language models (LVLMs) have shown substantial advances in multimodal understanding and generation. However, when presented with incompetent or adversarial inputs, they frequently produce unreliable or even harmful content, such as fact hallucinations or dangerous instructions. This misalignment with human expectations, referred to as \emph{misbehaviors} of LVLMs, raises serious concerns for deployment in critical applications. These misbehaviors are found to stem from epistemic uncertainty, specifically either conflicting internal knowledge or the absence of supporting information. However, existing uncertainty quantification methods, which typically capture only overall epistemic uncertainty, have shown limited effectiveness in identifying such issues. To address this gap, we propose Evidential Uncertainty Quantification (EUQ), a fine-grained method that captures both information conflict and ignorance for effective detection of LVLM misbehaviors. In particular, we interpret features from the model output head as either supporting (positive) or opposing (negative) evidence. Leveraging Evidence Theory, we model and aggregate this evidence to quantify internal conflict and knowledge gaps within a single forward pass. %We extensively evaluate our method across four categories of misbehavior, including hallucinations, jailbreaks, adversarial vulnerabilities, and out-of-distribution (OOD) failures, using state-of-the-art LVLMs, and find that EUQ consistently outperforms strong baselines, showing that hallucinations correspond to high internal conflict and OOD failures to high ignorance. Furthermore, layer-wise evidential uncertainty dynamics analysis helps interpret the evolution of internal representations from a new perspective. The source code is available at https://github.com/HT86159/EUQ.