Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning

TL;DR

This paper proposes a new framework for multimodal in-context learning that enhances reasoning and rule extraction in vision-language models through inductive-deductive restructuring, visual token filtering, and attention balancing.

Contribution

It introduces a structured inductive-deductive approach with visual token compression, attention rebalancing, and chain-of-thought prompting, improving multimodal ICL performance.

Findings

Significant performance improvements across eight benchmarks.

Enhanced reasoning and rule extraction capabilities.

Better handling of visual redundancy and attention skew.

Abstract

In-context learning (ICL) allows large models to adapt to tasks using a few examples, yet its extension to vision-language models (VLMs) remains fragile. Our analysis reveals that the fundamental limitation lies in an inductive gap, models often produce correct answers from flawed reasoning, while struggling to extract consistent rules across demonstrations. This gap is further exacerbated by two visual-level obstacles: an overwhelming proportion of redundant visual tokens that obscure textual cues, and a skewed attention distribution that favors the initial image at the expense of subsequent context. To address these issues, we introduce a framework that restructures multimodal ICL as a principled inductive-deductive process. The framework incorporates a similarity-based visual token compression module to filter out redundant patches, a dynamic attention rebalancing mechanism to distribute focus equitably across all images, and a chain-of-thought paradigm that explicitly guides the model to analyze individual examples, derive a generalizable rule, and then apply it to the query. An auxiliary learning pipeline combines supervised fine-tuning with reinforcement learning using verifiable rewards to reinforce faithful citation and noise filtering. Evaluations across eight benchmarks covering visual perception, logical reasoning, STEM problems, and sarcasm detection demonstrate consistent and significant improvements over standard ICL baselines for multiple open-source VLMs, highlighting the potential of equipping models with genuine inductive capabilities in multimodal settings.