Low-Resource Response Generation with Template Prior

TL;DR

This paper introduces a novel response generation method that leverages both limited paired data and large unpaired data using templates as priors, significantly improving performance in low-resource scenarios.

Contribution

It proposes a response generation model combining encoder-decoder architecture with template priors estimated from unpaired data, enhanced by an adversarial training approach.

Findings

Outperforms state-of-the-art models with limited paired data

Effective use of unpaired data improves response quality

Significant gains in automatic and human evaluations

Abstract

We study open domain response generation with limited message-response pairs. The problem exists in real-world applications but is less explored by the existing work. Since the paired data now is no longer enough to train a neural generation model, we consider leveraging the large scale of unpaired data that are much easier to obtain, and propose response generation with both paired and unpaired data. The generation model is defined by an encoder-decoder architecture with templates as prior, where the templates are estimated from the unpaired data as a neural hidden semi-markov model. By this means, response generation learned from the small paired data can be aided by the semantic and syntactic knowledge in the large unpaired data. To balance the effect of the prior and the input message to response generation, we propose learning the whole generation model with an adversarial approach. Empirical studies on question response generation and sentiment response generation indicate that when only a few pairs are available, our model can significantly outperform several state-of-the-art response generation models in terms of both automatic and human evaluation.