Using contradictions improves question answering systems

TL;DR

This paper explores incorporating contradiction detection in natural language inference to enhance question answering systems, showing that combining contradiction, entailment, and confidence scores improves accuracy, especially in sensitive domains.

Contribution

It introduces the use of contradiction detection alongside entailment in QA systems and demonstrates their combined effectiveness in improving performance.

Findings

Contradiction detection can slightly improve QA accuracy.

Combining contradiction, entailment, and confidence scores yields the best results.

Implications for safer QA deployment in critical domains.

Abstract

This work examines the use of contradiction in natural language inference (NLI) for question answering (QA). Typically, NLI systems help answer questions by determining if a potential answer is \emph{entailed} (supported) by some background context. But is it useful to also determine if an answer contradicts the context? We test this in two settings, multiple choice and extractive QA, and find that systems that incorporate contradiction can do slightly better than entailment-only systems on certain datasets. However, the best performances come from using contradiction, entailment, and QA model confidence scores together. This has implications for the deployment of QA systems in domains such as medicine and science where safety is an issue.