Learning Feature-Preserving Portrait Editing from Generated Pairs

TL;DR

This paper introduces a training-based portrait editing method that uses auto-generated paired data and a multi-conditioned diffusion model to achieve high-quality edits while preserving subject identity features.

Contribution

It presents a novel data generation process and a multi-conditioned diffusion model for feature-preserving portrait editing, addressing limitations of existing techniques.

Findings

Achieves state-of-the-art quality in costume and cartoon expression editing.

Effectively preserves subject features during editing.

Demonstrates superior quantitative and qualitative results.

Abstract

Portrait editing is challenging for existing techniques due to difficulties in preserving subject features like identity. In this paper, we propose a training-based method leveraging auto-generated paired data to learn desired editing while ensuring the preservation of unchanged subject features. Specifically, we design a data generation process to create reasonably good training pairs for desired editing at low cost. Based on these pairs, we introduce a Multi-Conditioned Diffusion Model to effectively learn the editing direction and preserve subject features. During inference, our model produces accurate editing mask that can guide the inference process to further preserve detailed subject features. Experiments on costume editing and cartoon expression editing show that our method achieves state-of-the-art quality, quantitatively and qualitatively.