Are vision language models robust to uncertain inputs?

TL;DR

This paper evaluates the robustness of vision language models to uncertain inputs, revealing that larger models are more robust but still prone to hallucinations, and proposes a caption diversity method to better estimate uncertainty.

Contribution

It provides an empirical assessment of VLM robustness, demonstrates the effectiveness of prompting for abstention, and introduces a novel caption diversity mechanism for uncertainty estimation.

Findings

Larger VLMs show improved robustness but still hallucinate on ambiguous inputs.

Prompting models to abstain significantly improves reliability on natural images.

Caption diversity can predict model success in abstaining without labeled data.

Abstract

Robustness against uncertain and ambiguous inputs is a critical challenge for deep learning models. While recent advancements in large scale vision language models (VLMs, e.g. GPT4o) might suggest that increasing model and training dataset size would mitigate this issue, our empirical evaluation shows a more complicated picture. Testing models using two classic uncertainty quantification tasks, anomaly detection and classification under inherently ambiguous conditions, we find that newer and larger VLMs indeed exhibit improved robustness compared to earlier models, but still suffer from a tendency to strictly follow instructions, often causing them to hallucinate confident responses even when faced with unclear or anomalous inputs. Remarkably, for natural images such as ImageNet, this limitation can be overcome without pipeline modifications: simply prompting models to abstain from uncertain predictions enables significant reliability gains, achieving near-perfect robustness in several settings. However, for domain-specific tasks such as galaxy morphology classification, a lack of specialized knowledge prevents reliable uncertainty estimation. Finally, we propose a novel mechanism based on caption diversity to reveal a model's internal uncertainty, enabling practitioners to predict when models will successfully abstain without relying on labeled data.