Adapting Security Warnings to Counter Online Disinformation

TL;DR

This study investigates how different designs of disinformation warnings influence user behavior, demonstrating that well-designed warnings can help users identify disinformation, but may also cause experience friction.

Contribution

The paper adapts and applies information security warning methods to disinformation warnings, providing evidence of their effectiveness and a framework for evaluation.

Findings

Interstitial warnings increase user information-seeking behavior

Warning design influences user perception of risk and information source

Behavioral effects are not moderated by user comprehension or fear of harm

Abstract

Disinformation is proliferating on the internet, and platforms are responding by attaching warnings to content. There is little evidence, however, that these warnings help users identify or avoid disinformation. In this work, we adapt methods and results from the information security warning literature in order to design and evaluate effective disinformation warnings. In an initial laboratory study, we used a simulated search task to examine contextual and interstitial disinformation warning designs. We found that users routinely ignore contextual warnings, but users notice interstitial warnings -- and respond by seeking information from alternative sources. We then conducted a follow-on crowdworker study with eight interstitial warning designs. We confirmed a significant impact on user information-seeking behavior, and we found that a warning's design could effectively inform users or convey a risk of harm. We also found, however, that neither user comprehension nor fear of harm moderated behavioral effects. Our work provides evidence that disinformation warnings can -- when designed well -- help users identify and avoid disinformation. We show a path forward for designing effective warnings, and we contribute repeatable methods for evaluating behavioral effects. We also surface a possible dilemma: disinformation warnings might be able to inform users and guide behavior, but the behavioral effects might result from user experience friction, not informed decision making.