Towards Fair and Robust Face Parsing for Generative AI: A Multi-Objective Approach

TL;DR

This paper introduces a multi-objective learning framework for face parsing that enhances fairness and robustness, leading to improved face synthesis quality in generative AI applications.

Contribution

It proposes a homotopy-based loss function for jointly optimizing accuracy, fairness, and robustness in face parsing models, a novel approach in this domain.

Findings

Fairness-aware segmentation enhances demographic consistency.

Robust face parsing improves image synthesis quality.

Multi-objective training outperforms single-objective models in face generation tasks.

Abstract

Face parsing is a fundamental task in computer vision, enabling applications such as identity verification, facial editing, and controllable image synthesis. However, existing face parsing models often lack fairness and robustness, leading to biased segmentation across demographic groups and errors under occlusions, noise, and domain shifts. These limitations affect downstream face synthesis, where segmentation biases can degrade generative model outputs. We propose a multi-objective learning framework that optimizes accuracy, fairness, and robustness in face parsing. Our approach introduces a homotopy-based loss function that dynamically adjusts the importance of these objectives during training. To evaluate its impact, we compare multi-objective and single-objective U-Net models in a GAN-based face synthesis pipeline (Pix2PixHD). Our results show that fairness-aware and robust segmentation improves photorealism and consistency in face generation. Additionally, we conduct preliminary experiments using ControlNet, a structured conditioning model for diffusion-based synthesis, to explore how segmentation quality influences guided image generation. Our findings demonstrate that multi-objective face parsing improves demographic consistency and robustness, leading to higher-quality GAN-based synthesis.