Distilling Information Reliability and Source Trustworthiness from Digital Traces

TL;DR

This paper introduces a temporal point process framework that leverages noisy evaluation traces from online platforms to accurately and interpretably measure information reliability and source trustworthiness.

Contribution

It proposes a novel modeling approach that uses temporal traces to derive unbiased measures of reliability and trustworthiness, with an efficient optimization method for parameter learning.

Findings

Accurately predicts evaluation events on real-world data.

Provides interpretable measures of reliability and trustworthiness.

Yields insights into real-world information dynamics.

Abstract

Online knowledge repositories typically rely on their users or dedicated editors to evaluate the reliability of their content. These evaluations can be viewed as noisy measurements of both information reliability and information source trustworthiness. Can we leverage these noisy evaluations, often biased, to distill a robust, unbiased and interpretable measure of both notions? In this paper, we argue that the temporal traces left by these noisy evaluations give cues on the reliability of the information and the trustworthiness of the sources. Then, we propose a temporal point process modeling framework that links these temporal traces to robust, unbiased and interpretable notions of information reliability and source trustworthiness. Furthermore, we develop an efficient convex optimization procedure to learn the parameters of the model from historical traces. Experiments on real-world data gathered from Wikipedia and Stack Overflow show that our modeling framework accurately predicts evaluation events, provides an interpretable measure of information reliability and source trustworthiness, and yields interesting insights about real-world events.