Fairness for Image Generation with Uncertain Sensitive Attributes

TL;DR

This paper explores fairness in image generation without relying on predefined group labels, introducing new definitions and methods to achieve fair outcomes despite the ambiguity of group identities.

Contribution

It introduces group fairness notions that do not depend on specific groupings and proposes a method to achieve fairness obliviously in image generation.

Findings

Demographic parity depends heavily on group definitions and is often impossible to achieve obliviously.

Conditional Proportional Representation can be achieved obliviously using Posterior Sampling.

Experiments demonstrate fair image reconstruction with state-of-the-art generative models.

Abstract

This work tackles the issue of fairness in the context of generative procedures, such as image super-resolution, which entail different definitions from the standard classification setting. Moreover, while traditional group fairness definitions are typically defined with respect to specified protected groups -- camouflaging the fact that these groupings are artificial and carry historical and political motivations -- we emphasize that there are no ground truth identities. For instance, should South and East Asians be viewed as a single group or separate groups? Should we consider one race as a whole or further split by gender? Choosing which groups are valid and who belongs in them is an impossible dilemma and being "fair" with respect to Asians may require being "unfair" with respect to South Asians. This motivates the introduction of definitions that allow algorithms to be \emph{oblivious} to the relevant groupings. We define several intuitive notions of group fairness and study their incompatibilities and trade-offs. We show that the natural extension of demographic parity is strongly dependent on the grouping, and \emph{impossible} to achieve obliviously. On the other hand, the conceptually new definition we introduce, Conditional Proportional Representation, can be achieved obliviously through Posterior Sampling. Our experiments validate our theoretical results and achieve fair image reconstruction using state-of-the-art generative models.