Measuring Verifiability in Online Information

TL;DR

This paper introduces a multidimensional measure of verifiability for online information, specifically applied to Wikipedia, assessing the quality, accessibility, and technical accuracy of references across thousands of articles.

Contribution

It proposes a novel, comprehensive framework for quantifying verifiability that considers multiple aspects of source quality and accessibility, and applies it to a large Wikipedia sample.

Findings

Reference quality is generally high in Wikipedia

Verifiability varies significantly across articles

Emphasizing digital identifiers affects verifiability scores

Abstract

The verifiability of online information is important, but difficult to assess systematically. We examine verifiability in the case of Wikipedia, one of the world's largest and most consulted online information sources. We extend prior work about quality of Wikipedia articles, knowledge production, and sources to consider the quality of Wikipedia references. We propose a multidimensional measure of verifiability that takes into account technical accuracy and practical accessibility of sources. We calculate article verifiability scores for a sample of 5,000 articles and 295,800 citations, and compare differently weighted models to illustrate effects of emphasizing particular elements of verifiability over others. We find that, while the quality of references in the overall sample is reasonably high, verifiability varies significantly by article, particularly when emphasizing the use of standard digital identifiers and taking into account the practical availability of referenced sources. We discuss the implications of these findings for measuring verifiability in online information more generally.