TED-On: A Total Error Framework for Digital Traces of Human Behavior on Online Platforms

TL;DR

This paper introduces TED-On, a comprehensive framework inspired by survey error models, to systematically categorize and understand errors in research using digital traces of human behavior on online platforms.

Contribution

It proposes a novel total error framework tailored for digital traces, enhancing error diagnosis, understanding, and communication in social science research.

Findings

Identifies specific error types unique to digital trace data

Provides a standardized vocabulary for error description

Facilitates improved research accuracy and transparency

Abstract

Peoples' activities and opinions recorded as digital traces online, especially on social media and other web-based platforms, offer increasingly informative pictures of the public. They promise to allow inferences about populations beyond the users of the platforms on which the traces are recorded, representing real potential for the Social Sciences and a complement to survey-based research. But the use of digital traces brings its own complexities and new error sources to the research enterprise. Recently, researchers have begun to discuss the errors that can occur when digital traces are used to learn about humans and social phenomena. This article synthesizes this discussion and proposes a systematic way to categorize potential errors, inspired by the Total Survey Error (TSE) Framework developed for survey methodology. We introduce a conceptual framework to diagnose, understand, and document errors that may occur in studies based on such digital traces. While there are clear parallels to the well-known error sources in the TSE framework, the new "Total Error Framework for Digital Traces of Human Behavior on Online Platforms" (TED-On) identifies several types of error that are specific to the use of digital traces. By providing a standard vocabulary to describe these errors, the proposed framework is intended to advance communication and research concerning the use of digital traces in scientific social research.