CAMD: Coverage-Aware Multimodal Decoding for Efficient Reasoning of Multimodal Large Language Models

TL;DR

This paper introduces CAMD, an adaptive decoding method for multimodal large language models that dynamically allocates computation based on sample difficulty, improving efficiency and reasoning accuracy.

Contribution

The paper presents a theoretical framework linking sampling coverage and difficulty, and proposes CAMD, a novel adaptive decoding mechanism for better resource allocation in multimodal reasoning.

Findings

CAMD improves reasoning accuracy on benchmarks.

CAMD reduces computational waste on easy samples.

Multimodal reasoning has a heavy-tailed difficulty distribution.

Abstract

Recent advances in Multimodal Large Language Models (MLLMs) have shown impressive reasoning capabilities across vision-language tasks, yet still face the challenge of compute-difficulty mismatch. Through empirical analyses, we identify that existing decoding methods may waste compute on easy cases while underserving hard ones, affecting both model effectiveness and efficiency. To address this issue, we first develop a theoretical framework that links sampling coverage, instance difficulty, and residual risk. Our analysis reveals that multimodal reasoning exhibits a heavy-tailed difficulty distribution; a small subset of hard or ambiguous samples dominates the residual failure probability. Based on this insight, we propose Coverage-Aware Multimodal Decoding (CAMD), an adaptive inference mechanism that dynamically allocates computation according to estimated uncertainty. CAMD integrates evidence-weighted scoring, posterior coverage estimation, and sequential Bayesian updating to balance efficiency and reliability under a limited token budget. Experiments on various benchmark datasets and baselines demonstrate the effectiveness and advantages of our approach.