TruePose: Human-Parsing-guided Attention Diffusion for Full-ID Preserving Pose Transfer

TL;DR

TruePose introduces a novel human-parsing-guided attention diffusion method that significantly improves the preservation of clothing and facial details in pose transfer, addressing limitations of existing diffusion models.

Contribution

The paper proposes a human-parsing-aware Siamese network with fusion and CLIP-guided attention modules to enhance clothing and facial detail preservation in pose-guided image synthesis.

Findings

Outperforms 13 baseline methods in clothing and facial detail preservation

Effective in both in-shop and in-the-wild datasets

Achieves high-quality pose transfer with identity preservation

Abstract

Pose-Guided Person Image Synthesis (PGPIS) generates images that maintain a subject's identity from a source image while adopting a specified target pose (e.g., skeleton). While diffusion-based PGPIS methods effectively preserve facial features during pose transformation, they often struggle to accurately maintain clothing details from the source image throughout the diffusion process. This limitation becomes particularly problematic when there is a substantial difference between the source and target poses, significantly impacting PGPIS applications in the fashion industry where clothing style preservation is crucial for copyright protection. Our analysis reveals that this limitation primarily stems from the conditional diffusion model's attention modules failing to adequately capture and preserve clothing patterns. To address this limitation, we propose human-parsing-guided attention diffusion, a novel approach that effectively preserves both facial and clothing appearance while generating high-quality results. We propose a human-parsing-aware Siamese network that consists of three key components: dual identical UNets (TargetNet for diffusion denoising and SourceNet for source image embedding extraction), a human-parsing-guided fusion attention (HPFA), and a CLIP-guided attention alignment (CAA). The HPFA and CAA modules can embed the face and clothes patterns into the target image generation adaptively and effectively. Extensive experiments on both the in-shop clothes retrieval benchmark and the latest in-the-wild human editing dataset demonstrate our method's significant advantages over 13 baseline approaches for preserving both facial and clothes appearance in the source image.