Question rewriting? Assessing its importance for conversational question answering

TL;DR

This paper evaluates the impact of question rewriting modules on conversational question answering systems, demonstrating that effective context representation significantly improves overall system performance in a shared task setting.

Contribution

It introduces a conversational QA system tailored for the SCAI shared task and provides a detailed analysis of how different question rewriting strategies affect system effectiveness.

Findings

Question rewriting significantly influences answer accuracy.

Effective context representation improves system performance.

The proposed system achieved top results in the shared task.

Abstract

In conversational question answering, systems must correctly interpret the interconnected interactions and generate knowledgeable answers, which may require the retrieval of relevant information from a background repository. Recent approaches to this problem leverage neural language models, although different alternatives can be considered in terms of modules for (a) representing user questions in context, (b) retrieving the relevant background information, and (c) generating the answer. This work presents a conversational question answering system designed specifically for the Search-Oriented Conversational AI (SCAI) shared task, and reports on a detailed analysis of its question rewriting module. In particular, we considered different variations of the question rewriting module to evaluate the influence on the subsequent components, and performed a careful analysis of the results obtained with the best system configuration. Our system achieved the best performance in the shared task and our analysis emphasizes the importance of the conversation context representation for the overall system performance.