Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance

TL;DR

This paper introduces DiSCo and Table-GLS, a framework that efficiently improves large vision-language models' ability to reason over complex table images by disentangling structure and content without heavy supervision or external tools.

Contribution

The work proposes a novel disentangled alignment framework and a structure-guided reasoning method to enhance table reasoning in LVLMs with minimal annotation and no external tools.

Findings

Significant performance improvements on diverse table reasoning benchmarks.

Effective generalization to unseen table structures.

Reduced reliance on supervised data and external tools.

Abstract

Reasoning over table images remains challenging for Large Vision-Language Models (LVLMs) due to complex layouts and tightly coupled structure-content information. Existing solutions often depend on expensive supervised training, reinforcement learning, or external tools, limiting efficiency and scalability. This work addresses a key question: how to adapt LVLMs to table reasoning with minimal annotation and no external tools? Specifically, we first introduce DiSCo, a Disentangled Structure-Content alignment framework that explicitly separates structural abstraction from semantic grounding during multimodal alignment, efficiently adapting LVLMs to tables structures. Building on DiSCo, we further present Table-GLS, a Global-to-Local Structure-guided reasoning framework that performs table reasoning via structured exploration and evidence-grounded inference. Extensive experiments across diverse benchmarks demonstrate that our framework efficiently enhances LVLM's table understanding and reasoning capabilities, particularly generalizing to unseen table structures.