Mixture-of-Instructions: Aligning Large Language Models via Mixture Prompting

TL;DR

This paper introduces Mixture-of-Instructions, a novel prompt strategy that improves large language model alignment across multiple tasks by using diverse system prompts and instruction packing, demonstrated on the Qwen-7B-chat model.

Contribution

The paper proposes Mixture-of-Instructions, a new technique combining instruction packing with diverse prompts to enhance multi-task alignment of large language models.

Findings

Qwen-SFT-MoI shows improved performance in coding, mathematics, and tool use tasks.

Diverse system prompts help prevent overfitting and improve inference.

Benchmark results indicate significant alignment efficiency gains.

Abstract

With the proliferation of large language models (LLMs), the comprehensive alignment of such models across multiple tasks has emerged as a critical area of research. Existing alignment methodologies primarily address single task, such as multi-turn dialogue, coding, mathematical problem-solving, and tool usage. Although there is a large amount of high-quality data available for those tasks, most of them provide only questions and answers without including the system prompt. Though a detailed analysis of the Qwen language model, we found that the system prompt has a significant impact on both training and inference processes of LLM. We attributes this phenomenon to overfitting to the system prompt. In address this issue, we introduce a novel technique termed Mixture-of-Instructions (MoI), which employs a strategy of instruction packing combined with diverse system prompts to boost the alignment efficiency of language models. We have also compiled a diverse set of seven benchmark datasets to rigorously evaluate the alignment efficacy of the MoI-enhanced language model. Our methodology was applied to the open-source Qwen-7B-chat model, culminating in the development of Qwen-SFT-MoI. This enhanced model demonstrates significant advancements in generative capabilities across coding, mathematics, and tool use tasks.