FaceScore: Benchmarking and Enhancing Face Quality in Human Generation

TL;DR

This paper introduces FaceScore, a new metric for evaluating face quality in images generated by diffusion models, and demonstrates how it can be used to improve face realism in generated images.

Contribution

The paper develops FaceScore, a novel face quality metric aligned with human judgment, and uses it to enhance diffusion models for better face generation.

Findings

FaceScore aligns better with human judgments than existing metrics.

Using FaceScore, diffusion models' face quality can be significantly improved.

Extensive experiments validate the effectiveness of the proposed approach.

Abstract

Diffusion models (DMs) have achieved significant success in generating imaginative images given textual descriptions. However, they are likely to fall short when it comes to real-life scenarios with intricate details. The low-quality, unrealistic human faces in text-to-image generation are one of the most prominent issues, hindering the wide application of DMs in practice. Targeting addressing such an issue, we first assess the face quality of generations from popular pre-trained DMs with the aid of human annotators and then evaluate the alignment between existing metrics with human judgments. Observing that existing metrics can be unsatisfactory for quantifying face quality, we develop a novel metric named FaceScore (FS) by fine-tuning the widely used ImageReward on a dataset of (win, loss) face pairs cheaply crafted by an inpainting pipeline of DMs. Extensive studies reveal FS enjoys a superior alignment with humans. On the other hand, FS opens up the door for enhancing DMs for better face generation. With FS offering image ratings, we can easily perform preference learning algorithms to refine DMs like SDXL. Comprehensive experiments verify the efficacy of our approach for improving face quality. The code is released at https://github.com/OPPO-Mente-Lab/FaceScore.