Evaluating Demographic Misrepresentation in Image-to-Image Portrait Editing

TL;DR

This paper investigates demographic biases in instruction-guided image-to-image portrait editing, revealing pervasive, uneven failures in identity preservation and proposing a prompt-based mitigation approach to improve demographic robustness.

Contribution

It formalizes demographic failure modes in I2I editing, introduces a benchmark for evaluation, and demonstrates a prompt-level identity constraint to reduce demographic biases without model retraining.

Findings

Identity preservation failures are widespread and demographically uneven.

Implicit social priors influence demographic-conditioned editing outcomes.

Prompt-level constraints can mitigate demographic biases effectively.

Abstract

Demographic bias in text-to-image (T2I) generation is well studied, yet demographic-conditioned failures in instruction-guided image-to-image (I2I) editing remain underexplored. We examine whether identical edit instructions yield systematically different outcomes across subject demographics in open-weight I2I editors. We formalize two failure modes: Soft Erasure, where edits are silently weakened or ignored in the output image, and Stereotype Replacement, where edits introduce unrequested, stereotype-consistent attributes. We introduce a controlled benchmark that probes demographic-conditioned behavior by generating and editing portraits conditioned on race, gender, and age using a diagnostic prompt set, and evaluate multiple editors with vision-language model (VLM) scoring and human evaluation. Our analysis shows that identity preservation failures are pervasive, demographically uneven, and shaped by implicit social priors, including occupation-driven gender inference. Finally, we demonstrate that a prompt-level identity constraint, without model updates, can substantially reduce demographic change for minority groups while leaving majority-group portraits largely unchanged, revealing asymmetric identity priors in current editors. Together, our findings establish identity preservation as a central and demographically uneven failure mode in I2I editing and motivate demographic-robust editing systems. Project page: https://seochan99.github.io/i2i-demographic-bias