FairHuman: Boosting Hand and Face Quality in Human Image Generation with Minimum Potential Delay Fairness in Diffusion Models

TL;DR

FairHuman introduces a multi-objective fine-tuning method for diffusion models that enhances the quality of human images, especially faces and hands, by incorporating local region supervision and fairness-aware optimization.

Contribution

It presents a novel multi-objective fine-tuning approach with MPD-guided parameter updates to improve local detail generation in human images.

Findings

Significant improvement in face and hand detail quality.

Enhanced overall human image generation performance.

Effective fairness-aware optimization strategy.

Abstract

Image generation has achieved remarkable progress with the development of large-scale text-to-image models, especially diffusion-based models. However, generating human images with plausible details, such as faces or hands, remains challenging due to insufficient supervision of local regions during training. To address this issue, we propose FairHuman, a multi-objective fine-tuning approach designed to enhance both global and local generation quality fairly. Specifically, we first construct three learning objectives: a global objective derived from the default diffusion objective function and two local objectives for hands and faces based on pre-annotated positional priors. Subsequently, we derive the optimal parameter updating strategy under the guidance of the Minimum Potential Delay (MPD) criterion, thereby attaining fairness-ware optimization for this multi-objective problem. Based on this, our proposed method can achieve significant improvements in generating challenging local details while maintaining overall quality. Extensive experiments showcase the effectiveness of our method in improving the performance of human image generation under different scenarios.