Is Seeing Believing? Evaluating Human Sensitivity to Synthetic Video

TL;DR

This research investigates how humans perceive synthetic videos and audio distortions, revealing that artifacts in deepfakes can diminish their perceived credibility and impact cognitive processing.

Contribution

It provides empirical evidence on the effects of distortions and deepfake artifacts on human credibility assessment and highlights the need for further theoretical understanding.

Findings

Video distortions reduce perceived credibility.

Deepfake artifacts influence credibility judgments.

Audio-visual distortions affect cognitive processing.

Abstract

Advances in machine learning have enabled the creation of realistic synthetic videos known as deepfakes. As deepfakes proliferate, concerns about rapid spread of disinformation and manipulation of public perception are mounting. Despite the alarming implications, our understanding of how individuals perceive synthetic media remains limited, obstructing the development of effective mitigation strategies. This paper aims to narrow this gap by investigating human responses to visual and auditory distortions of videos and deepfake-generated visuals and narration. In two between-subjects experiments, we study whether audio-visual distortions affect cognitive processing, such as subjective credibility assessment and objective learning outcomes. A third study reveals that artifacts from deepfakes influence credibility. The three studies show that video distortions and deepfake artifacts can reduce credibility. Our research contributes to the ongoing exploration of the cognitive processes involved in the evaluation and perception of synthetic videos, and underscores the need for further theory development concerning deepfake exposure.