Collab: Fostering Critical Identification of Deepfake Videos on Social Media via Synergistic Annotation

TL;DR

Collab is a web tool that enables collaborative annotation of deepfake videos, improving user detection accuracy and fostering critical evaluation through innovative aggregation and demonstration strategies.

Contribution

The paper introduces Collab, a novel web plugin with confidence-weighted aggregation and hierarchical demonstrations to enhance deepfake detection and user criticality.

Findings

Collab significantly improved deepfake identification accuracy.

Participants showed increased critical reflection with Collab.

The aggregation method outperformed non-aggregation approaches.

Abstract

Identifying deepfake videos on social media platforms is challenged by dynamic spatio-temporal artifacts and inadequate user tools. This hinders both critical viewing by users and scalable moderation on platforms. Here, we present Collab, a web plugin enabling users to collaboratively annotate deepfake videos. Collab integrates three key components: (i) an intuitive interface for spatio-temporal labeling where users provide confidence scores and rationales, facilitating detailed input even from non-experts, (ii) a novel confidence-weighted spatio-temporal Intersection-over-Union (IoU) algorithm to aggregate diverse user annotations into accurate aggregations, and (iii) a hierarchical demonstration strategy presenting aggregated results to guide attention toward contentious regions and foster critical evaluation. A seven-day online study (N=90), where participants annotated suspicious videos when viewing an online experimental platforms, compared Collab against two conditions without aggregation or demonstration respectively. Collab significantly improved identification accuracy and enhanced reflection compared to non-demonstration condition, while outperforming non-aggregation condition for its novelty and effectiveness.