PartialEdit: Identifying Partial Deepfakes in the Era of Neural Speech Editing

TL;DR

PartialEdit introduces a dataset and analysis for detecting and localizing neural speech editing deepfakes, highlighting the challenges of identifying partially edited speech and the limitations of existing detection models.

Contribution

The paper presents a new dataset for partial deepfake detection, evaluates existing models' performance, and provides insights into neural speech editing artifacts for improved detection.

Findings

Existing models fail to detect neural speech editing deepfakes.

Neural audio codecs introduce detectable artifacts.

Insights into neural editing artifacts aid detection efforts.

Abstract

Neural speech editing enables seamless partial edits to speech utterances, allowing modifications to selected content while preserving the rest of the audio unchanged. This useful technique, however, also poses new risks of deepfakes. To encourage research on detecting such partially edited deepfake speech, we introduce PartialEdit, a deepfake speech dataset curated using advanced neural editing techniques. We explore both detection and localization tasks on PartialEdit. Our experiments reveal that models trained on the existing PartialSpoof dataset fail to detect partially edited speech generated by neural speech editing models. As recent speech editing models almost all involve neural audio codecs, we also provide insights into the artifacts the model learned on detecting these deepfakes. Further information about the PartialEdit dataset and audio samples can be found on the project page: https://yzyouzhang.com/PartialEdit/index.html.