SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training

TL;DR

SPICE is a novel data selection method for large language model training that reduces gradient conflicts, leading to more efficient training with less data and comparable or better performance.

Contribution

It introduces a conflict-aware data selector that maximizes information gain while penalizing gradient misalignment, improving efficiency in instruction tuning.

Findings

SPICE selects higher-information subsets than existing criteria.

Using only 10% of data, SPICE matches or exceeds full-data tuning performance.

SPICE achieves significant training cost reductions across multiple benchmarks.

Abstract

Information-based data selection for instruction tuning is compelling: maximizing the log-determinant of the Fisher information yields a monotone submodular objective, enabling greedy algorithms to achieve a $(1-1/e)$ approximation under a cardinality budget. In practice, however, we identify alleviating gradient conflicts, misalignment between per-sample gradients, is a key factor that slows down the decay of marginal log-determinant information gains, thereby preventing significant loss of information. We formalize this via an $\varepsilon$-decomposition that quantifies the deviation from ideal submodularity as a function of conflict statistics, yielding data-dependent approximation factors that tighten as conflicts diminish. Guided by this analysis, we propose SPICE, a conflict-aware selector that maximizes information while penalizing misalignment, and that supports early stopping and proxy models for efficiency. Empirically, SPICE selects subsets with higher log-determinant information than original criteria, and these informational gains translate into performance improvements: across 8 benchmarks with LLaMA2-7B and Qwen2-7B, SPICE uses only 10% of the data, yet matches or exceeds 6 methods including full-data tuning. This achieves performance improvements with substantially lower training cost.