Balanced Thinking: Improving Chain of Thought Training in Vision Language Models

TL;DR

This paper introduces SCALe, a dynamic loss weighting method that improves vision-language model reasoning by focusing training on answer segments, leading to more accurate and concise outputs with less training time.

Contribution

SCALe is a novel training approach that adaptively balances reasoning and answer supervision, outperforming standard fine-tuning and matching more complex pipelines.

Findings

SCALe improves accuracy over vanilla SFT.

SCALe reduces training time significantly.

Combining SCALe with reinforcement learning yields the best results.

Abstract

Multimodal reasoning in vision-language models (VLMs) typically relies on a two-stage process: supervised fine-tuning (SFT) and reinforcement learning (RL). In standard SFT, all tokens contribute equally to the loss, even though reasoning data are inherently token-imbalanced. Long <think> traces overshadow short but task-critical <answer> segments, leading to verbose reasoning and inaccurate answers. We propose SCALe (Scheduled Curriculum Adaptive Loss), which explicitly separates supervision over reasoning and answer segments using dynamic, length-independent weighting. Unlike vanilla SFT, which overweights the <think> segment, SCALe-SFT gradually shifts the focus from <think> to <answer> throughout training via a cosine scheduling policy, encouraging concise and well-grounded reasoning. We evaluate SCALe across diverse benchmarks and architectures. Results show that SCALe consistently improves accuracy over vanilla SFT and matches the performance of the full two-phase SFT + GRPO pipeline while requiring only about one-seventh of the training time, making it a lightweight yet effective alternative. When combined with GRPO, SCALe achieves the best overall performance, highlighting its value both as a standalone method and as a strong foundation for reinforcement refinement.