VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning

TL;DR

This paper introduces VL-Calibration, a reinforcement learning framework that improves confidence calibration and reasoning accuracy in large vision-language models by decoupling visual and reasoning confidence.

Contribution

It proposes a novel decoupled confidence calibration method for LVLMs, addressing hallucinations and improving both calibration and reasoning performance.

Findings

VL-Calibration improves calibration across thirteen benchmarks.

It enhances visual reasoning accuracy in LVLMs.

The method generalizes across different model scales and architectures.

Abstract

Large Vision Language Models (LVLMs) achieve strong multimodal reasoning but frequently exhibit hallucinations and incorrect responses with high certainty, which hinders their usage in high-stakes domains. Existing verbalized confidence calibration methods, largely developed for text-only LLMs, typically optimize a single holistic confidence score using binary answer-level correctness. This design is mismatched to LVLMs: an incorrect prediction may arise from perceptual failures or from reasoning errors given correct perception, and a single confidence conflates these sources while visual uncertainty is often dominated by language priors. To address these issues, we propose VL-Calibration, a reinforcement learning framework that explicitly decouples confidence into visual and reasoning confidence. To supervise visual confidence without ground-truth perception labels, we introduce an intrinsic visual certainty estimation that combines (i) visual grounding measured by KL-divergence under image perturbations and (ii) internal certainty measured by token entropy. We further propose token-level advantage reweighting to focus optimization on tokens based on visual certainty, suppressing ungrounded hallucinations while preserving valid perception. Experiments on thirteen benchmarks show that VL-Calibration effectively improves calibration while boosting visual reasoning accuracy, and it generalizes to out-of-distribution benchmarks across model scales and architectures.