PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning

TL;DR

PLATO-2 introduces a curriculum learning approach with two training stages to develop a high-quality open-domain chatbot capable of generating diverse and contextually appropriate responses, achieving state-of-the-art results.

Contribution

It presents a novel two-stage training process for open-domain chatbots, combining coarse and fine-grained models with latent variables and evaluation mechanisms.

Findings

Achieves state-of-the-art performance on Chinese and English datasets.

Demonstrates effectiveness of curriculum learning in dialogue systems.

Produces diverse and contextually relevant responses.

Abstract

To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning. There are two stages involved in the learning process. In the first stage, a coarse-grained generation model is trained to learn response generation under the simplified framework of one-to-one mapping. In the second stage, a fine-grained generative model augmented with latent variables and an evaluation model are further trained to generate diverse responses and to select the best response, respectively. PLATO-2 was trained on both Chinese and English data, whose effectiveness and superiority are verified through comprehensive evaluations, achieving new state-of-the-art results.