Using In-Context Learning to Improve Dialogue Safety

TL;DR

This paper presents a retrieval-based in-context learning approach to enhance dialogue safety by steering large conversational models towards safer responses without additional training.

Contribution

It introduces a novel retrieval method using demonstrations of safe responses to improve safety in dialogue models, outperforming some fine-tuned baselines.

Findings

Competitive safety performance without training

Retrieval-based method reduces toxicity effectively

Re-ranking further improves response safety

Abstract

While large neural-based conversational models have become increasingly proficient dialogue agents, recent work has highlighted safety issues with these systems. For example, these systems can be goaded into generating toxic content, which often perpetuates social biases or stereotypes. We investigate a retrieval-based method for reducing bias and toxicity in responses from chatbots. It uses in-context learning to steer a model towards safer generations. Concretely, to generate a response to an unsafe dialogue context, we retrieve demonstrations of safe responses to similar dialogue contexts. We find our method performs competitively with strong baselines without requiring training. For instance, using automatic evaluation, we find our best fine-tuned baseline only generates safe responses to unsafe dialogue contexts from DiaSafety 4.04% more than our approach. Finally, we also propose a re-ranking procedure which can further improve response safeness.