Taming Continuous Posteriors for Latent Variational Dialogue Policies

TL;DR

This paper demonstrates that Gaussian variational posteriors can outperform categorical ones in latent-action reinforcement learning for dialogue systems, achieving state-of-the-art success rates with improved training simplicity and response coherence.

Contribution

The authors introduce a simplified training approach for Gaussian latent posteriors in dialogue RL, surpassing categorical methods in performance and coherence.

Findings

Achieves state-of-the-art success rate on MultiWOZ benchmark.

Gaussian posteriors outperform categorical in dialogue success.

Model maintains good response coherence.

Abstract

Utilizing amortized variational inference for latent-action reinforcement learning (RL) has been shown to be an effective approach in Task-oriented Dialogue (ToD) systems for optimizing dialogue success. Until now, categorical posteriors have been argued to be one of the main drivers of performance. In this work we revisit Gaussian variational posteriors for latent-action RL and show that they can yield even better performance than categoricals. We achieve this by simplifying the training procedure and propose ways to regularize the latent dialogue policy to retain good response coherence. Using continuous latent representations our model achieves state of the art dialogue success rate on the MultiWOZ benchmark, and also compares well to categorical latent methods in response coherence.