Multi-Party Supervised Fine-tuning of Language Models for Multi-Party Dialogue Generation

TL;DR

This paper introduces MuPaS, a novel multi-party fine-tuning framework for large language models that significantly improves multi-party dialogue generation, response accuracy, and out-of-distribution performance.

Contribution

The paper proposes MuPaS, a new multi-party fine-tuning method that enables LLMs to adapt to multi-party dialogues effectively, surpassing previous pairwise fine-tuning approaches.

Findings

Achieves state-of-the-art multi-party response quality

Improves next-speaker prediction accuracy

Generates coherent responses in out-of-distribution scenarios

Abstract

Large Language Models (LLM) are usually fine-tuned to participate in dyadic or two-party dialogues, which can not adapt well to multi-party dialogues (MPD), which hinders their applications in such scenarios including multi-personal meetings, discussions and daily communication. Previous LLM-based researches mainly focus on the multi-agent framework, while their base LLMs are still pairwisely fine-tuned. In this work, we design a multi-party fine-tuning framework (MuPaS) for LLMs on the multi-party dialogue datasets, and prove such a straightforward framework can let the LLM align with the multi-party conversation style efficiently and effectively. We also design two training strategies which can convert MuPaS into the MPD simulator. Substantial experiments show that MuPaS can achieve state-of-the-art multi-party response, higher accuracy of the-next-speaker prediction, higher human and automatic evaluated utterance qualities, and can even generate reasonably with out-of-distribution scene, topic and role descriptions. The MuPaS framework bridges the LLM training with more complicated multi-party applications, such as conversation generation, virtual rehearsal or meta-universe.