How Spammers and Scammers Leverage AI-Generated Images on Facebook for Audience Growth

TL;DR

This paper reveals that spammers and scammers are actively using AI-generated images on Facebook to increase their reach, often without users' awareness, emphasizing the need for better transparency and provenance standards.

Contribution

It uncovers the current use of AI-generated images by malicious actors on Facebook and highlights the risks of unlabelled AI content in social media.

Findings

AI-generated images are used for audience growth on Facebook.

Unlabeled AI images are recommended to users unaware of their origin.

There is a need for improved transparency and provenance standards.

Abstract

Much of the research and discourse on risks from artificial intelligence (AI) image generators, such as DALL-E and Midjourney, has centered around whether they could be used to inject false information into political discourse. We show that spammers and scammers - seemingly motivated by profit or clout, not ideology - are already using AI-generated images to gain significant traction on Facebook. At times, the Facebook Feed is recommending unlabeled AI-generated images to users who neither follow the Pages posting the images nor realize that the images are AI-generated, highlighting the need for improved transparency and provenance standards as AI models proliferate.