Effect of AI Performance, Risk Perception, and Trust on Human Dependence in Deepfake Detection AI system

TL;DR

This study investigates how AI performance, risk perception, and trust influence human reliance on AI systems for deepfake detection through an experimental approach.

Contribution

It provides empirical insights into how AI performance affects user trust and dependence, informing the design of more transparent AI detection tools.

Findings

Participants adjust their dependence on AI based on perceived risk.

AI performance significantly influences user trust and reliance.

Insights support development of explainable AI systems for media verification.

Abstract

Synthetic images, audio, and video can now be generated and edited by Artificial Intelligence (AI). In particular, the malicious use of synthetic data has raised concerns about potential harms to cybersecurity, personal privacy, and public trust. Although AI-based detection tools exist to help identify synthetic content, their limitations often lead to user mistrust and confusion between real and fake content. This study examines the role of AI performance in influencing human trust and decision making in synthetic data identification. Through an online human subject experiment involving 400 participants, we examined how varying AI performance impacts human trust and dependence on AI in deepfake detection. Our findings indicate how participants calibrate their dependence on AI based on their perceived risk and the prediction results provided by AI. These insights contribute to the development of transparent and explainable AI systems that better support everyday users in mitigating the harms of synthetic media.