SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours

TL;DR

RumourEval is a shared task that focuses on identifying the truthfulness of rumours and analyzing discourse around them using a large annotated dataset and specific challenges.

Contribution

It introduces an annotation scheme, a comprehensive dataset, and defines two challenges for rumour veracity and support detection in social media texts.

Findings

Participants achieved measurable results on the defined challenges.

The dataset covers multiple topics and discourse types.

The task advances research in automated rumour verification.

Abstract

Media is full of false claims. Even Oxford Dictionaries named "post-truth" as the word of 2016. This makes it more important than ever to build systems that can identify the veracity of a story, and the kind of discourse there is around it. RumourEval is a SemEval shared task that aims to identify and handle rumours and reactions to them, in text. We present an annotation scheme, a large dataset covering multiple topics - each having their own families of claims and replies - and use these to pose two concrete challenges as well as the results achieved by participants on these challenges.