Fine-tuning Large Language Models with Sequential Instructions

TL;DR

This paper introduces a sequential instruction tuning method for large language models, improving their performance on complex multi-step tasks by incorporating chains of interrelated instructions and proposing a new evaluation benchmark.

Contribution

It proposes a novel sequential instruction tuning approach, automates it using existing datasets, and introduces SeqEval, a benchmark for assessing multi-instruction following capabilities.

Findings

Enhanced performance in coding, maths, and open-ended tasks

Improved ability to follow complex instruction sequences

Introduced a new benchmark for sequential instruction evaluation

Abstract

Despite the success of existing instruction-tuned models, we find that they usually struggle to respond to queries with multiple instructions. This impairs their performance in complex problems whose solution consists of multiple intermediate tasks. Thus, we contend that part of the fine-tuning data mixture should be sequential--containing a chain of interrelated tasks. We first approach sequential instruction tuning from a task-driven perspective, manually creating interpretable intermediate tasks for multilingual and visual question answering: namely "translate then predict" and "caption then answer". Next, we automate this process by turning instructions in existing datasets (e.g., Alpaca and FlanCoT) into diverse and complex sequential instructions, making our method general-purpose. Models that underwent our sequential instruction tuning show improved results in coding, maths, and open-ended generation. Moreover, we put forward a new benchmark named SeqEval to evaluate a model's ability to follow all the instructions in a sequence, which further corroborates the benefits of our fine-tuning method. We hope that our endeavours will open new research avenues on instruction tuning for complex tasks.