OASIS: Online Sample Selection for Continual Visual Instruction Tuning

TL;DR

OASIS is an adaptive online sample selection method for continual instruction tuning that efficiently selects informative data, reducing training overhead while maintaining high performance in real-time scenarios.

Contribution

It introduces a reference model-free, adaptive sampling approach that considers all previously seen data and reduces redundancy, improving continual learning efficiency.

Findings

Achieves comparable performance with only 25% of data used.

Outperforms existing state-of-the-art sampling methods.

Effective across various large foundation models.

Abstract

In continual instruction tuning (CIT) scenarios, where new instruction tuning data continuously arrive in an online streaming manner, training delays from large-scale data significantly hinder real-time adaptation. Data selection can mitigate this overhead, but existing strategies often rely on pretrained reference models, which are impractical in CIT setups since future data are unknown. Recent reference model-free online sample selection methods address this, but typically select a fixed number of samples per batch (e.g., top-k), making them vulnerable to distribution shifts where informativeness varies across batches. To address these limitations, we propose OASIS, an adaptive online sample selection approach for CIT that (1) selects informative samples by estimating each sample's informativeness relative to all previously seen data, beyond batch-level constraints, and (2) minimizes informative redundancy of selected samples through iterative selection score updates. Experiments on various large foundation models show that OASIS, using only 25 percent of the data, achieves comparable performance to full-data training and outperforms the state-of-the-art sampling methods.