Turn That Frown Upside Down: FaceID Customization via Cross-Training Data

TL;DR

This paper introduces CrossFaceID, a large-scale dataset designed to enhance FaceID customization models' ability to generate varied facial images of the same person, addressing previous limitations due to lack of diverse training data.

Contribution

The creation of CrossFaceID, the first large-scale dataset enabling improved facial modifications in FaceID customization models, along with demonstrating its effectiveness through experiments.

Findings

Models fine-tuned on CrossFaceID retain FaceID fidelity.

Fine-tuned models significantly improve face customization capabilities.

Public release of code, datasets, and trained models.

Abstract

Existing face identity (FaceID) customization methods perform well but are limited to generating identical faces as the input, while in real-world applications, users often desire images of the same person but with variations, such as different expressions (e.g., smiling, angry) or angles (e.g., side profile). This limitation arises from the lack of datasets with controlled input-output facial variations, restricting models' ability to learn effective modifications. To address this issue, we propose CrossFaceID, the first large-scale, high-quality, and publicly available dataset specifically designed to improve the facial modification capabilities of FaceID customization models. Specifically, CrossFaceID consists of 40,000 text-image pairs from approximately 2,000 persons, with each person represented by around 20 images showcasing diverse facial attributes such as poses, expressions, angles, and adornments. During the training stage, a specific face of a person is used as input, and the FaceID customization model is forced to generate another image of the same person but with altered facial features. This allows the FaceID customization model to acquire the ability to personalize and modify known facial features during the inference stage. Experiments show that models fine-tuned on the CrossFaceID dataset retain its performance in preserving FaceID fidelity while significantly improving its face customization capabilities. To facilitate further advancements in the FaceID customization field, our code, constructed datasets, and trained models are fully available to the public.