"i have a feeling trump will win..................": Forecasting Winners and Losers from User Predictions on Twitter

TL;DR

This paper explores predicting event winners from Twitter predictions by analyzing explicit user statements, using a novel automated method that outperforms baseline approaches without prior outcome data.

Contribution

It introduces a high-precision classifier for veridicality detection and demonstrates a fully automated crowd-based forecasting approach that surpasses sentiment and volume baselines.

Findings

High-precision veridicality classifier developed

Automated crowd-based forecasting outperforms baselines

Method can assess prediction reliability and identify surprises

Abstract

Social media users often make explicit predictions about upcoming events. Such statements vary in the degree of certainty the author expresses toward the outcome:"Leonardo DiCaprio will win Best Actor" vs. "Leonardo DiCaprio may win" or "No way Leonardo wins!". Can popular beliefs on social media predict who will win? To answer this question, we build a corpus of tweets annotated for veridicality on which we train a log-linear classifier that detects positive veridicality with high precision. We then forecast uncertain outcomes using the wisdom of crowds, by aggregating users' explicit predictions. Our method for forecasting winners is fully automated, relying only on a set of contenders as input. It requires no training data of past outcomes and outperforms sentiment and tweet volume baselines on a broad range of contest prediction tasks. We further demonstrate how our approach can be used to measure the reliability of individual accounts' predictions and retrospectively identify surprise outcomes.