ProxyThinker: Test-Time Guidance through Small Visual Reasoners

TL;DR

ProxyThinker is a novel inference-time method that enhances large vision-language models' reasoning abilities by leveraging small visual reasoners, significantly improving performance and inference speed without additional training.

Contribution

It introduces ProxyThinker, a training-free technique that transfers reasoning capabilities from small models to large models during inference, enabling efficient and improved visual reasoning.

Findings

Boosts performance on visual reasoning benchmarks

Achieves up to 38x faster inference

Enables untuned models to match RFT counterparts

Abstract

Recent advancements in reinforcement learning with verifiable rewards have pushed the boundaries of the visual reasoning capabilities in large vision-language models (LVLMs). However, training LVLMs with reinforcement fine-tuning (RFT) is computationally expensive, posing a significant challenge to scaling model size. In this work, we propose ProxyThinker, an inference-time technique that enables large models to inherit the visual reasoning capabilities from small, slow-thinking visual reasoners without any training. By subtracting the output distributions of base models from those of RFT reasoners, ProxyThinker modifies the decoding dynamics and successfully elicits the slow-thinking reasoning demonstrated by the emerged sophisticated behaviors such as self-verification and self-correction. ProxyThinker consistently boosts performance on challenging visual benchmarks on spatial, mathematical, and multi-disciplinary reasoning, enabling untuned base models to compete with the performance of their full-scale RFT counterparts. Furthermore, our implementation efficiently coordinates multiple language models with parallelism techniques and achieves up to 38 $\times$ faster inference compared to previous decoding-time methods, paving the way for the practical deployment of ProxyThinker. Code is available at https://github.com/MrZilinXiao/ProxyThinker.