Counteracting Matthew Effect in Self-Improvement of LVLMs through Head-Tail Re-balancing

TL;DR

This paper addresses the Matthew effect in self-improving LVLMs by proposing re-balancing strategies that enhance performance on complex reasoning tasks, overcoming biases towards simpler queries.

Contribution

It introduces four strategies for head-tail re-balancing during self-improvement, significantly improving LVLMs' reasoning abilities on visual tasks.

Findings

Improved reasoning performance by 3.86 points on average.

Effectively mitigated head-tail imbalance in self-improvement.

Enhanced complex reasoning capabilities of LVLMs.

Abstract

Self-improvement has emerged as a mainstream paradigm for advancing the reasoning capabilities of large vision-language models (LVLMs), where models explore and learn from successful trajectories iteratively. However, we identify a critical issue during this process: the model excels at generating high-quality trajectories for simple queries (i.e., head data) but struggles with more complex ones (i.e., tail data). This leads to an imbalanced optimization that drives the model to prioritize simple reasoning skills, while hindering its ability to tackle more complex reasoning tasks. Over iterations, this imbalance becomes increasingly pronounced--a dynamic we term the "Matthew effect"--which ultimately hinders further model improvement and leads to performance bottlenecks. To counteract this challenge, we introduce four efficient strategies from two perspectives: distribution-reshaping and trajectory-resampling, to achieve head-tail re-balancing during the exploration-and-learning self-improvement process. Extensive experiments on Qwen2-VL-7B-Instruct and InternVL2.5-4B models across visual reasoning tasks demonstrate that our methods consistently improve visual reasoning capabilities, outperforming vanilla self-improvement by 3.86 points on average.