Automatically Exposing Problems with Neural Dialog Models

TL;DR

This paper introduces automated methods, including reinforcement learning, to systematically identify safety and contradiction issues in neural dialog models, reducing reliance on manual or superficial problem detection.

Contribution

It proposes two novel automated approaches, including reinforcement learning, to expose systematic problems in dialog models more effectively than prior manual or superficial methods.

Findings

Successfully exposed safety issues in state-of-the-art models

Revealed contradiction problems through automated triggering

Demonstrated effectiveness of proposed methods

Abstract

Neural dialog models are known to suffer from problems such as generating unsafe and inconsistent responses. Even though these problems are crucial and prevalent, they are mostly manually identified by model designers through interactions. Recently, some research instructs crowdworkers to goad the bots into triggering such problems. However, humans leverage superficial clues such as hate speech, while leaving systematic problems undercover. In this paper, we propose two methods including reinforcement learning to automatically trigger a dialog model into generating problematic responses. We show the effect of our methods in exposing safety and contradiction issues with state-of-the-art dialog models.