Efficient Data Selection for Multimodal Models via Incremental Optimization Utility

TL;DR

This paper introduces OST, an efficient data selection framework for multimodal models that improves training cost and performance by estimating sample utility through incremental optimization, outperforming existing heuristics.

Contribution

OST reformulates data selection as an incremental utility ranking problem, reducing costs and enhancing model performance over heuristic and full fine-tuning methods.

Findings

OST reduces training costs by 43% and total time by 17%.

OST surpasses LLM-as-a-Judge baseline by 1.8 points on benchmarks.

Using top-20 samples, OST outperforms heuristic methods and full fine-tuning.

Abstract

The scaling of Large Multimodal Models (LMMs) is constrained by the quality-quantity trade-off inherent in synthetic data. Previous approaches, such as LLM-as-a-Judge, have proven their effectiveness in addressing this but suffer from prohibitive computational costs and lack of interpretability. To bridge this gap, we propose One-Step-Train (OST), a framework that reformulates data selection as an incremental optimization utility ranking problem. Instead of relying on semantic heuristics, OST estimates the marginal utility of each sample via a simulated single-step update on a lightweight proxy. Experiments on the Qwen series across multimodal mathematical reasoning benchmarks demonstrate that OST achieves Pareto-optimal efficiency. By selecting the top-50 subset, OST reduces training costs by 43% (and total time consumption by 17) while surpassing the strong LLM-as-a-Judge baseline by 1.8 points. Furthermore, under a fixed compute budget, our method using only the top-20 subset achieves a 5.6 point gain over LLM-as-a-Judge, improves upon heuristic scoring baselines like DEITA, and outperforms the Full-SFT baseline by 8.8 points. Notably, while Full-SFT suffers from performance degradation due to noise, our optimization-grounded approach effectively identifies toxic samples, successfully reversing the negative transfer frequently observed in complex reasoning tasks.