Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning

TL;DR

Data Whisperer introduces a training-free, attention-based method that uses few-shot in-context learning to efficiently select optimal data subsets for fine-tuning large language models, reducing costs and improving performance.

Contribution

It presents a novel, training-free data selection approach leveraging in-context learning, outperforming existing methods in efficiency and effectiveness.

Findings

Achieves superior performance with only 10% of data compared to full datasets.

Outperforms existing data selection methods with a 3.1-point improvement.

Provides a 7.4× speedup in data selection process.

Abstract

Fine-tuning large language models (LLMs) on task-specific data is essential for their effective deployment. As dataset sizes grow, efficiently selecting optimal subsets for training becomes crucial to balancing performance and computational costs. Traditional data selection methods often require fine-tuning a scoring model on the target dataset, which is time-consuming and resource-intensive, or rely on heuristics that fail to fully leverage the model's predictive capabilities. To address these challenges, we propose Data Whisperer, an efficient, training-free, attention-based method that leverages few-shot in-context learning with the model to be fine-tuned. Comprehensive evaluations were conducted on both raw and synthetic datasets across diverse tasks and models. Notably, Data Whisperer achieves superior performance compared to the full GSM8K dataset on the Llama-3-8B-Instruct model, using just 10% of the data, and outperforms existing methods with a 3.1-point improvement and a 7.4$\times$ speedup. The code is available at https://github.com/gszfwsb/Data-Whisperer.