Coincidental Generation

TL;DR

This paper highlights a new privacy risk in generative AI called coincidental generation, where outputs resemble real entities beyond the training data, raising legal and ethical concerns.

Contribution

It introduces the concept of coincidental generation in generative AI, emphasizing its implications for privacy, likeness rights, and regulatory compliance.

Findings

Coincidental generation is inevitable due to low-dimensional face perception.

Synthetic faces often resemble real individuals, risking misappropriation.

This phenomenon exposes organizations to legal and regulatory risks.

Abstract

Generative A.I. models have emerged as versatile tools across diverse industries, with applications in privacy-preserving data sharing, computational art, personalization of products and services, and immersive entertainment. Here, we introduce a new privacy concern in the adoption and use of generative A.I. models: that of coincidental generation, where a generative model's output is similar enough to an existing entity, beyond those represented in the dataset used to train the model, to be mistaken for it. Consider, for example, synthetic portrait generators, which are today deployed in commercial applications such as virtual modeling agencies and synthetic stock photography. Due to the low intrinsic dimensionality of human face perception, every synthetically generated face will coincidentally resemble an actual person. Such examples of coincidental generation all but guarantee the misappropriation of likeness and expose organizations that use generative A.I. to legal and regulatory risk.