Interpreting Answers to Yes-No Questions in Dialogues from Multiple Domains

TL;DR

This paper addresses the challenge of interpreting indirect yes-no answers in dialogues across multiple domains, proposing a new adaptable approach that improves answer interpretation accuracy.

Contribution

It introduces new benchmarks across diverse domains and a domain-adaptive training method for better interpretation of indirect answers.

Findings

F1 score improvements of 11-34% across domains

Approach is robust and never detrimental

New benchmarks for dialogue answer interpretation

Abstract

People often answer yes-no questions without explicitly saying yes, no, or similar polar keywords. Figuring out the meaning of indirect answers is challenging, even for large language models. In this paper, we investigate this problem working with dialogues from multiple domains. We present new benchmarks in three diverse domains: movie scripts, tennis interviews, and airline customer service. We present an approach grounded on distant supervision and blended training to quickly adapt to a new dialogue domain. Experimental results show that our approach is never detrimental and yields F1 improvements as high as 11-34%.