Discord Questions: A Computational Approach To Diversity Analysis in News Coverage

TL;DR

This paper introduces a framework using Discord Questions to analyze and visualize diversity in news coverage sources, aiding readers in understanding differences in news reporting.

Contribution

It presents a novel approach for generating questions that highlight source differences and consolidating answers to reveal coverage diversity.

Findings

Generated questions are more interesting than factoid questions.

Model improves question generation performance by 5%.

Answers achieve 81% accuracy in semantic grouping.

Abstract

There are many potential benefits to news readers accessing diverse sources. Modern news aggregators do the hard work of organizing the news, offering readers a plethora of source options, but choosing which source to read remains challenging. We propose a new framework to assist readers in identifying source differences and gaining an understanding of news coverage diversity. The framework is based on the generation of Discord Questions: questions with a diverse answer pool, explicitly illustrating source differences. To assemble a prototype of the framework, we focus on two components: (1) discord question generation, the task of generating questions answered differently by sources, for which we propose an automatic scoring method, and create a model that improves performance from current question generation (QG) methods by 5%, (2) answer consolidation, the task of grouping answers to a question that are semantically similar, for which we collect data and repurpose a method that achieves 81% balanced accuracy on our realistic test set. We illustrate the framework's feasibility through a prototype interface. Even though model performance at discord QG still lags human performance by more than 15%, generated questions are judged to be more interesting than factoid questions and can reveal differences in the level of detail, sentiment, and reasoning of sources in news coverage.