InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning

TL;DR

InstructDiff is a domain-adaptive data selection method that uses differential entropy to efficiently fine-tune large language models, significantly reducing data requirements while improving performance across reasoning and general tasks.

Contribution

The paper introduces InstructDiff, a novel entropy-based data selection framework that adapts to domain-specific tasks, enhancing fine-tuning efficiency and effectiveness.

Findings

Achieves 17% improvement on mathematical reasoning tasks.

Achieves 52% improvement on general instruction-following tasks.

Uses only 10% of the data compared to full dataset training.

Abstract

Supervised fine-tuning (SFT) is fundamental to adapting large language models, yet training on complete datasets incurs prohibitive costs with diminishing returns. Existing data selection methods suffer from severe domain specificity: techniques optimized for general instruction-following fail on reasoning tasks, and vice versa. We observe that measuring entropy differences between base models and minimally instruction-tuned calibrated models reveals a pattern -- samples with the lowest differential entropy consistently yield optimal performance across domains, yet this principle manifests domain-adaptively: reasoning tasks favor entropy increase (cognitive expansion), while general tasks favor entropy decrease (cognitive compression). We introduce InstructDiff, a unified framework that operationalizes differential entropy as a domain-adaptive selection criterion through warmup calibration, bi-directional NLL filtering, and entropy-based ranking. Extensive experiments show that InstructDiff achieves 17\% relative improvement over full data training on mathematical reasoning and 52\% for general instruction-following, outperforming prior baselines while using only 10\% of the data.