Improving Fairness in Deepfake Detection

TL;DR

This paper introduces novel loss functions to improve fairness in deepfake detection algorithms, addressing biases related to race and gender, and demonstrates their effectiveness across multiple datasets and detectors.

Contribution

It presents the first method to enhance fairness in deepfake detection at the algorithm level using new loss functions, applicable with or without demographic data.

Findings

Improved fairness across different demographic groups

Effective across multiple datasets and detectors

Flexible approach usable with existing models

Abstract

Despite the development of effective deepfake detectors in recent years, recent studies have demonstrated that biases in the data used to train these detectors can lead to disparities in detection accuracy across different races and genders. This can result in different groups being unfairly targeted or excluded from detection, allowing undetected deepfakes to manipulate public opinion and erode trust in a deepfake detection model. While existing studies have focused on evaluating fairness of deepfake detectors, to the best of our knowledge, no method has been developed to encourage fairness in deepfake detection at the algorithm level. In this work, we make the first attempt to improve deepfake detection fairness by proposing novel loss functions that handle both the setting where demographic information (eg, annotations of race and gender) is available as well as the case where this information is absent. Fundamentally, both approaches can be used to convert many existing deepfake detectors into ones that encourages fairness. Extensive experiments on four deepfake datasets and five deepfake detectors demonstrate the effectiveness and flexibility of our approach in improving deepfake detection fairness. Our code is available at https://github.com/littlejuyan/DF_Fairness.