Enhancing Complex Instruction Following for Large Language Models with Mixture-of-Contexts Fine-tuning

TL;DR

This paper introduces MISO, a mixture-of-contexts fine-tuning method for large language models that improves their ability to follow complex, multi-constraint instructions by processing parallel subcontexts, leading to better instruction adherence and training efficiency.

Contribution

The paper proposes MISO, a novel extension to transformer-based LLMs that jointly considers subcontexts during fine-tuning to enhance complex instruction following.

Findings

MISO outperforms existing fine-tuning methods in complex instruction scenarios.

MISO improves training efficiency for large language models.

Empirical results show enhanced instruction-following accuracy.

Abstract

Large language models (LLMs) exhibit remarkable capabilities in handling natural language tasks; however, they may struggle to consistently follow complex instructions including those involve multiple constraints. Post-training LLMs using supervised fine-tuning (SFT) is a standard approach to improve their ability to follow instructions. In addressing complex instruction following, existing efforts primarily focus on data-driven methods that synthesize complex instruction-output pairs for SFT. However, insufficient attention allocated to crucial sub-contexts may reduce the effectiveness of SFT. In this work, we propose transforming sequentially structured input instruction into multiple parallel instructions containing subcontexts. To support processing this multi-input, we propose MISO (Multi-Input Single-Output), an extension to currently dominant decoder-only transformer-based LLMs. MISO introduces a mixture-of-contexts paradigm that jointly considers the overall instruction-output alignment and the influence of individual sub-contexts to enhance SFT effectiveness. We apply MISO fine-tuning to complex instructionfollowing datasets and evaluate it with standard LLM inference. Empirical results demonstrate the superiority of MISO as a fine-tuning method for LLMs, both in terms of effectiveness in complex instruction-following scenarios and its potential for training efficiency.