Speculative Coreset Selection for Task-Specific Fine-tuning

TL;DR

STAFF is a novel speculative coreset selection method that uses a small model to efficiently identify important data samples for fine-tuning large language models, significantly reducing computational overhead and improving performance.

Contribution

The paper introduces STAFF, a new coreset selection approach that leverages a small model for efficient data scoring and verification, enhancing data efficiency and reducing fine-tuning costs.

Findings

STAFF improves SOTA methods by up to 54.3% in performance.

STAFF reduces selection overhead by up to 70.5%.

Low pruning rate coresets can outperform full dataset fine-tuning.

Abstract

Task-specific fine-tuning is essential for the deployment of large language models (LLMs), but it requires significant computational resources and time. Existing solutions have proposed coreset selection methods to improve data efficiency and reduce model training overhead, but they still have limitations: 1) Overlooking valuable samples at high pruning rates, which degrades the coreset's performance. 2) Requiring high time overhead during coreset selection to fine-tune and evaluate the target LLM. In this paper, we introduce STAFF, a speculative coreset selection method. STAFF leverages a small model from the same family as the target LLM to efficiently estimate data scores and then verifies the scores on the target LLM to accurately identify and allocate more selection budget to important regions while maintaining coverage of easy regions. We evaluate STAFF on three LLMs and three downstream tasks and show that STAFF improves the performance of SOTA methods by up to 54.3% and reduces selection overhead by up to 70.5% at different pruning rates. Furthermore, we observe that the coreset selected by STAFF at low pruning rates (i.e., 20%) can even obtain better fine-tuning performance than the full dataset.