Iteratively Prompting Multimodal LLMs to Reproduce Natural and AI-Generated Images

TL;DR

This paper demonstrates a method to generate images similar to AI marketplace outputs using multimodal models and prompts, highlighting economic and security implications, and provides a large dataset of prompt-image pairs.

Contribution

Introduces an attack strategy using fine-tuned CLIP, multi-label classifiers, and GPT-4V to replicate AI-generated images at lower costs, and releases a large dataset of prompt-image pairs.

Findings

Comparable images can be produced at $0.23-$0.27 each

Automated metrics and human assessments confirm image similarity

Releases a dataset of 19 million prompt-image pairs from Midjourney

Abstract

With the digital imagery landscape rapidly evolving, image stocks and AI-generated image marketplaces have become central to visual media. Traditional stock images now exist alongside innovative platforms that trade in prompts for AI-generated visuals, driven by sophisticated APIs like DALL-E 3 and Midjourney. This paper studies the possibility of employing multi-modal models with enhanced visual understanding to mimic the outputs of these platforms, introducing an original attack strategy. Our method leverages fine-tuned CLIP models, a multi-label classifier, and the descriptive capabilities of GPT-4V to create prompts that generate images similar to those available in marketplaces and from premium stock image providers, yet at a markedly lower expense. In presenting this strategy, we aim to spotlight a new class of economic and security considerations within the realm of digital imagery. Our findings, supported by both automated metrics and human assessment, reveal that comparable visual content can be produced for a fraction of the prevailing market prices ($0.23 - $0.27 per image), emphasizing the need for awareness and strategic discussions about the integrity of digital media in an increasingly AI-integrated landscape. Our work also contributes to the field by assembling a dataset consisting of approximately 19 million prompt-image pairs generated by the popular Midjourney platform, which we plan to release publicly.