Learning Multi-Indicator Weights for Data Selection: A Joint Task-Model Adaptation Framework with Efficient Proxies

TL;DR

This paper introduces a framework that learns to adapt data selection weights for large language models by jointly considering downstream task requirements and model capabilities, using efficient proxies for performance evaluation.

Contribution

It proposes a novel joint task-model adaptation method that uses in-context learning signals on small validation sets to optimize data weights without full fine-tuning.

Findings

Achieves comparable or better performance than full dataset tuning with only 30% of data.

Effectively adapts data weights to specific tasks and models, improving instruction tuning.

Identifies a trade-off between semantic diversity and logical complexity in reasoning tasks.

Abstract

Data selection is a key component of efficient instruction tuning for large language models, as recent work has shown that data quality often matters more than data quantity. Accordingly, prior studies have introduced various multi-dimensional heuristics to evaluate and filter instruction data. However, most existing methods rely on static task-agnostic and model-agnostic weighting schemes, which overlook the varying requirements of specific downstream tasks and the differing pre-existing capabilities of models. In this paper, we propose a framework for learning multi-indicator weights that jointly adapts data selection to both the downstream task and the specific model. Our method identifies optimal weight configurations without full-scale fine-tuning by utilizing in-context learning (ICL) signals on compact tiny-validation sets. These signals serve as efficient performance proxies that ensure high-fidelity evaluation at minimal computational cost. Experiments across multiple benchmarks and model families, including Mistral, Qwen, and Llama, show that the approach achieves performance comparable to or exceeding full-dataset tuning while using only 30\% of the training samples on GSM8K. Furthermore, our analysis reveals a trade-off between semantic diversity and logical complexity in reasoning tasks, highlighting the necessity of joint task-model adaptation.