Deepfake Pornography is Resilient to Regulatory and Platform Shocks

TL;DR

This study examines how regulatory and platform interventions, like legislation and website shutdowns, influence the dissemination of deepfake pornography, revealing that such measures often lead to redistribution rather than reduction of this harmful content.

Contribution

It provides empirical evidence that regulatory and platform actions may shift deepfake pornography activity across platforms instead of decreasing its overall prevalence.

Findings

SNCEI sharing increased after interventions

Activity shifted across platforms and time

Interventions did not significantly reduce overall SNCEI dissemination

Abstract

Generative artificial intelligence tools have made it easier to create realistic, synthetic non-consensual explicit imagery (popularly known as deepfake pornography; hereinafter SNCEI) of people. Once created, this SNCEI is often shared on various websites, causing significant harm to victims. This emerging form of sexual abuse was recently criminalized in the US at the federal level by S.146, the TAKE IT DOWN Act. A week after the bill's passage became effectively imminent, the MrDeepfakes website -- one of the most notorious facilitators of SNCEI creation and dissemination -- shut down. Here, we explore the impact of the bill's passage and the subsequent shutdown as a compound intervention on the dissemination of SNCEI. We select three online forums where sexually explicit content is shared, each containing dedicated subforums to organize various types of sexually explicit content. By leveraging each forum's design, we compare activity in subforums dedicated to SNCEI with that in other pornographic genres using a synthetic control, quasi-experimental approach. Across websites, we observed an increase in the sharing and requests for SNCEI, and, in some cases, in new contributors. These results indicate that the compound intervention did not suppress SNCEI activity overall but instead coincided with its redistribution across platforms, with substantial heterogeneity in timing and magnitude. Together, our findings suggest that deplatforming and regulatory signals alone may shift where and when SNCEI is produced and shared, rather than reducing its prevalence.