VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging

TL;DR

This paper introduces VCU-Bridge, a hierarchical framework for visual connotation understanding that mimics human reasoning, along with a benchmark and data generation pipeline, to improve multimodal model performance across reasoning levels.

Contribution

It proposes a novel hierarchical reasoning framework and benchmark for visual connotation understanding, and develops a data generation pipeline to enhance model capabilities.

Findings

Performance declines at higher reasoning levels.

Strengthening low-level perception improves high-level reasoning.

Method improves results on HVCU-Bench and general benchmarks.

Abstract

While Multimodal Large Language Models (MLLMs) excel on benchmarks, their processing paradigm differs from the human ability to integrate visual information. Unlike humans who naturally bridge details and high-level concepts, models tend to treat these elements in isolation. Prevailing evaluation protocols often decouple low-level perception from high-level reasoning, overlooking their semantic and causal dependencies, which yields non-diagnostic results and obscures performance bottlenecks. We present VCU-Bridge, a framework that operationalizes a human-like hierarchy of visual connotation understanding: multi-level reasoning that advances from foundational perception through semantic bridging to abstract connotation, with an explicit evidence-to-inference trace from concrete cues to abstract conclusions. Building on this framework, we construct HVCU-Bench, a benchmark for hierarchical visual connotation understanding with explicit, level-wise diagnostics. Comprehensive experiments demonstrate a consistent decline in performance as reasoning progresses to higher levels. We further develop a data generation pipeline for instruction tuning guided by Monte Carlo Tree Search (MCTS) and show that strengthening low-level capabilities yields measurable gains at higher levels. Interestingly, it not only improves on HVCU-Bench but also brings benefits on general benchmarks (average +2.53%), especially with substantial gains on MMStar (+7.26%), demonstrating the significance of the hierarchical thinking pattern and its effectiveness in enhancing MLLM capabilities. The project page is at https://vcu-bridge.github.io .