Correction of Faulty Background Knowledge based on Condition Aware and Revise Transformer for Question Answering

TL;DR

This paper introduces CAR-Transformer, a novel model that revises faulty or incomplete conditioning information in question answering, improving answer accuracy in real-world scenarios with imperfect external data.

Contribution

The paper proposes the CAR-Transformer, which revises and encodes condition values to enhance question answering robustness against incorrect or missing external conditions.

Findings

Outperforms baseline models on real-world customer service dataset

Effectively revises and utilizes condition information with errors or incompleteness

Demonstrates robustness in selecting appropriate answers despite faulty conditions

Abstract

The study of question answering has received increasing attention in recent years. This work focuses on providing an answer that compatible with both user intent and conditioning information corresponding to the question, such as delivery status and stock information in e-commerce. However, these conditions may be wrong or incomplete in real-world applications. Although existing question answering systems have considered the external information, such as categorical attributes and triples in knowledge base, they all assume that the external information is correct and complete. To alleviate the effect of defective condition values, this paper proposes condition aware and revise Transformer (CAR-Transformer). CAR-Transformer (1) revises each condition value based on the whole conversation and original conditions values, and (2) it encodes the revised conditions and utilizes the conditions embedding to select an answer. Experimental results on a real-world customer service dataset demonstrate that the CAR-Transformer can still select an appropriate reply when conditions corresponding to the question exist wrong or missing values, and substantially outperforms baseline models on automatic and human evaluations. The proposed CAR-Transformer can be extended to other NLP tasks which need to consider conditioning information.