ControversialQA: Exploring Controversy in Question Answering

TL;DR

This paper introduces a novel controversy dataset for question answering based on user perception, highlighting the importance and difficulty of detecting controversy independently of sentiment analysis.

Contribution

It presents the first dataset defining controversy through user votes, enabling new research in controversy detection in QA systems.

Findings

Controversy detection is essential for QA systems.

No strong correlation between controversy and sentiment.

Controversy detection remains a challenging task.

Abstract

Controversy is widespread online. Previous studies mainly define controversy based on vague assumptions of its relation to sentiment such as hate speech and offensive words. This paper introduces the first question-answering dataset that defines content controversy by user perception, i.e., votes from plenty of users. It contains nearly 10K questions, and each question has a best answer and a most controversial answer. Experimental results reveal that controversy detection in question answering is essential and challenging, and there is no strong correlation between controversy and sentiment tasks.