Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency

TL;DR

This paper explores how interactions and dependencies among diverse instruction types affect fine-tuning large language models, proposing methods to optimize instruction sets and learning schemas for improved performance.

Contribution

It introduces a systematic analysis of instruction interaction patterns and develops optimization techniques using linear programming and curriculum learning.

Findings

Enhanced LLM performance on benchmarks

Effective instruction set optimization methods

Insights into instruction interaction patterns

Abstract

With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs). Previous research mainly focuses on selecting individual high-quality instructions. However, these works overlooked the joint interactions and dependencies between different categories of instructions, leading to suboptimal selection strategies. Moreover, the nature of these interaction patterns remains largely unexplored, let alone optimize the instruction set with regard to them. To fill these gaps, in this paper, we: (1) systemically investigate interaction and dependency patterns between different categories of instructions, (2) manage to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning. Experimental results across different LLMs demonstrate improved performance over strong baselines on widely adopted benchmarks.