Rethinking the Instruction Quality: LIFT is What You Need

TL;DR

This paper introduces LIFT, a new paradigm for improving instruction data quality in large language models by expanding high-quality data subspaces and reducing redundancy, leading to enhanced performance.

Contribution

LIFT is a novel instruction fusion transfer method that broadens data distribution and eliminates redundancy, surpassing existing data curation and expansion techniques.

Findings

LLMs maintain strong performance with limited high-quality data

LIFT surpasses some state-of-the-art results

Significant improvement in instruction quality achieved

Abstract

Instruction tuning, a specialized technique to enhance large language model (LLM) performance via instruction datasets, relies heavily on the quality of employed data. Existing quality improvement methods alter instruction data through dataset expansion or curation. However, the expansion method risks data redundancy, potentially compromising LLM performance, while the curation approach confines the LLM's potential to the original dataset. Our aim is to surpass the original data quality without encountering these shortcomings. To achieve this, we propose LIFT (LLM Instruction Fusion Transfer), a novel and versatile paradigm designed to elevate the instruction quality to new heights. LIFT strategically broadens data distribution to encompass more high-quality subspaces and eliminates redundancy, concentrating on high-quality segments across overall data subspaces. Experimental results demonstrate that, even with a limited quantity of high-quality instruction data selected by our paradigm, LLMs not only consistently uphold robust performance across various tasks but also surpass some state-of-the-art results, highlighting the significant improvement in instruction quality achieved by our paradigm.