Federated Data-Efficient Instruction Tuning for Large Language Models

TL;DR

This paper introduces FedHDS, a federated instruction tuning method that enhances large language models' responsiveness by efficiently using a representative data subset, reducing data redundancy and training costs while maintaining high performance.

Contribution

FedHDS is a novel federated data-efficient instruction tuning approach that selects representative data subsets without sharing raw data, improving efficiency and performance over existing methods.

Findings

Improves Rouge-L on unseen tasks by 10.72% over SOTA federated methods.

Uses less than 1.5% of data samples, significantly reducing training data requirements.

Enhances training efficiency by up to tens of times.

Abstract

Instruction tuning is a crucial step in improving the responsiveness of pretrained large language models (LLMs) to human instructions. Federated learning (FL) helps to exploit the use of vast private instruction data from clients, becoming popular for LLM tuning by improving data diversity. Existing federated tuning simply consumes all local data, causing excessive computational overhead and overfitting to local data, while centralized data-efficient solutions are not suitable for FL due to privacy concerns. This work presents FedHDS, a federated data-efficient instruction tuning approach, which tunes LLMs with a representative subset of edge-side data. It reduces the data redundancy at both intra- and inter-client levels without sharing raw data. Experiments with various LLMs, datasets and partitions show that FedHDS improves Rouge-L on unseen tasks by an average of 10.72% over the SOTA full-data federated instruction tuning methods, while using less than 1.5% of the data samples, improving training efficiency by up to tens of times.