Natural Language Generation as Planning under Uncertainty Using Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning-based model for natural language generation in dialogue systems, optimizing trade-offs like information conveyance and cognitive load by learning from noisy feedback.

Contribution

It proposes a novel RL-based planning approach for NLG that adapts to noisy feedback and outperforms existing methods in presenting information effectively.

Findings

RL policy significantly outperforms baseline approaches

Effective handling of noisy feedback in NLG

Improved trade-off management between information and cognitive load

Abstract

We present and evaluate a new model for Natural Language Generation (NLG) in Spoken Dialogue Systems, based on statistical planning, given noisy feedback from the current generation context (e.g. a user and a surface realiser). We study its use in a standard NLG problem: how to present information (in this case a set of search results) to users, given the complex trade- offs between utterance length, amount of information conveyed, and cognitive load. We set these trade-offs by analysing existing MATCH data. We then train a NLG pol- icy using Reinforcement Learning (RL), which adapts its behaviour to noisy feed- back from the current generation context. This policy is compared to several base- lines derived from previous work in this area. The learned policy significantly out- performs all the prior approaches.