Instinct vs. Reflection: Unifying Token and Verbalized Confidence in Multimodal Large Models

TL;DR

This paper introduces a novel confidence estimation framework for multimodal large language models, addressing the mismatch between token-level support and verbal self-assessment to improve reliability.

Contribution

It proposes a monotone confidence fusion method and an order-preserving alignment to enhance confidence calibration and failure prediction in MLLMs.

Findings

Improved confidence calibration across diverse MLLMs.

Enhanced failure prediction accuracy.

Consistent reliability gains demonstrated in experiments.

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in various perception and reasoning tasks. Despite this success, ensuring their reliability in practical deployment necessitates robust confidence estimation. Prior works have predominantly focused on text-only LLMs, often relying on computationally expensive self-consistency sampling. In this paper, we extend this to multimodal settings and conduct a comprehensive evaluation of MLLMs' response confidence estimation. Our analysis reveals a significant instinct-reflection misalignment: the model's implicit token-level support frequently diverges from its verbal self-assessment confidence. To address this misalignment, we propose a monotone confidence fusion framework to merge dual-channel signals and cross-channel consistency to estimate correctness. Subsequently, an order-preserving mean alignment step is applied to correct global bias, which improves calibration while preserving the risk-coverage trade-off for selective prediction. Experiments on diverse open-source and closed-source MLLMs show that our method consistently yields more reliable confidence estimates and improves both calibration and failure prediction. Code will be available at https://github.com/Yunkaidang/Instinct-vs.-Reflection.