Classification as Decoder: Trading Flexibility for Control in Medical Dialogue

TL;DR

This paper introduces a classification-based approach for medical dialogue systems that improves response quality control by mapping conversation context to response classes, balancing flexibility with safety.

Contribution

It proposes a novel method that replaces generative models with a classification head on a pretrained language model for better response control in medical dialogues.

Findings

Discriminative model's responses are worse than doctors' 12% of the time.

Generative model's responses are worse than doctors' 18% of the time.

The approach allows easy updating of responses without retraining the entire system.

Abstract

Generative seq2seq dialogue systems are trained to predict the next word in dialogues that have already occurred. They can learn from large unlabeled conversation datasets, build a deeper understanding of conversational context, and generate a wide variety of responses. This flexibility comes at the cost of control, a concerning tradeoff in doctor/patient interactions. Inaccuracies, typos, or undesirable content in the training data will be reproduced by the model at inference time. We trade a small amount of labeling effort and some loss of response variety in exchange for quality control. More specifically, a pretrained language model encodes the conversational context, and we finetune a classification head to map an encoded conversational context to a response class, where each class is a noisily labeled group of interchangeable responses. Experts can update these exemplar responses over time as best practices change without retraining the classifier or invalidating old training data. Expert evaluation of 775 unseen doctor/patient conversations shows that only 12% of the discriminative model's responses are worse than the what the doctor ended up writing, compared to 18% for the generative model.