Towards a Universal NLG for Dialogue Systems and Simulators with Future Bridging

TL;DR

This paper introduces a novel Future Bridging NLG approach that enables universal, data-driven dialogue generation with minimal adaptation, leveraging future utterance information for self-supervised training across diverse dialogue scenarios.

Contribution

The paper proposes the Future Bridging NLG concept, allowing decoupled, self-supervised training on annotation-free data for universal dialogue generation.

Findings

Prototype FBNLG shows viability in task-oriented and chit-chat dialogues.

Future bridging enables minimal adaptation for new dialogue scenarios.

Pre-trained FBNLG can be applied across various dialogue types with little fine-tuning.

Abstract

In a dialogue system pipeline, a natural language generation (NLG) unit converts the dialogue direction and content to a corresponding natural language realization. A recent trend for dialogue systems is to first pre-train on large datasets and then fine-tune in a supervised manner using datasets annotated with application-specific features. Though novel behaviours can be learned from custom annotation, the required effort severely bounds the quantity of the training set, and the application-specific nature limits the reuse. In light of the recent success of data-driven approaches, we propose the novel future bridging NLG (FBNLG) concept for dialogue systems and simulators. The critical step is for an FBNLG to accept a future user or system utterance to bridge the present context towards. Future bridging enables self supervised training over annotation-free datasets, decoupled the training of NLG from the rest of the system. An FBNLG, pre-trained with massive datasets, is expected to apply in classical or new dialogue scenarios with minimal adaptation effort. We evaluate a prototype FBNLG to show that future bridging can be a viable approach to a universal few-shot NLG for task-oriented and chit-chat dialogues.