Investigating Rumor News Using Agreement-Aware Search

TL;DR

This paper introduces Maester, an agreement-aware search framework that helps investigate rumor news by categorizing related articles into agree, disagree, and discuss, providing a comprehensive view of a topic or event.

Contribution

Maester is a novel framework that combines keyword/entity matching and neural network models to improve agreement-aware search for rumor news investigation.

Findings

Maester achieves up to tenfold improvement over previous solutions.

The framework effectively categorizes articles into agreement levels.

Experimental results validate the approach on the FNC dataset.

Abstract

Recent years have witnessed a widespread increase of rumor news generated by humans and machines. Therefore, tools for investigating rumor news have become an urgent necessity. One useful function of such tools is to see ways a specific topic or event is represented by presenting different points of view from multiple sources. In this paper, we propose Maester, a novel agreement-aware search framework for investigating rumor news. Given an investigative question, Maester will retrieve related articles to that question, assign and display top articles from agree, disagree, and discuss categories to users. Splitting the results into these three categories provides the user a holistic view towards the investigative question. We build Maester based on the following two key observations: (1) relatedness can commonly be determined by keywords and entities occurring in both questions and articles, and (2) the level of agreement between the investigative question and the related news article can often be decided by a few key sentences. Accordingly, we use gradient boosting tree models with keyword/entity matching features for relatedness detection, and leverage recurrent neural network to infer the level of agreement. Our experiments on the Fake News Challenge (FNC) dataset demonstrate up to an order of magnitude improvement of Maester over the original FNC winning solution, for agreement-aware search.