CLUES: Collaborative High-Quality Data Selection for LLMs via Training Dynamics

TL;DR

This paper introduces CLUES, a novel method for selecting high-quality data for large language models in collaborative settings by analyzing training dynamics, improving model performance across diverse private datasets.

Contribution

The paper proposes a training dynamics-based data quality control method for collaborative LLM training, addressing privacy constraints and heterogeneous data sources.

Findings

High-quality data selection improves LLM fine-tuning performance.

Method outperforms other data selection techniques in diverse domains.

Effective in medical, multilingual, and financial datasets.

Abstract

Recent research has highlighted the importance of data quality in scaling large language models (LLMs). However, automated data quality control faces unique challenges in collaborative settings where sharing is not allowed directly between data silos. To tackle this issue, this paper proposes a novel data quality control technique based on the notion of data influence on the training dynamics of LLMs, that high quality data are more likely to have similar training dynamics to the anchor dataset. We then leverage the influence of the training dynamics to select high-quality data from different private domains, with centralized model updates on the server side in a collaborative training fashion by either model merging or federated learning. As for the data quality indicator, we compute the per-sample gradients with respect to the private data and the anchor dataset, and use the trace of the accumulated inner products as a measurement of data quality. In addition, we develop a quality control evaluation tailored for collaborative settings with heterogeneous domain data. Experiments show that training on the high-quality data selected by our method can often outperform other data selection methods for collaborative fine-tuning of LLMs, across diverse private domain datasets, in medical, multilingual and financial settings. Our code is released at github.com/Ryan0v0/CLUES.