Cross Attention Based Style Distribution for Controllable Person Image Synthesis

TL;DR

This paper introduces a cross attention style distribution module for controllable person image synthesis, enabling explicit control over pose and appearance with improved style transfer accuracy.

Contribution

It proposes a novel cross attention based style distribution mechanism that effectively aligns source styles with target poses for enhanced image synthesis.

Findings

Improves pose transfer quality both quantitatively and qualitatively.

Effectively routes color and texture based on attention between source styles and target pose.

Validated on pose transfer and virtual try-on tasks.

Abstract

Controllable person image synthesis task enables a wide range of applications through explicit control over body pose and appearance. In this paper, we propose a cross attention based style distribution module that computes between the source semantic styles and target pose for pose transfer. The module intentionally selects the style represented by each semantic and distributes them according to the target pose. The attention matrix in cross attention expresses the dynamic similarities between the target pose and the source styles for all semantics. Therefore, it can be utilized to route the color and texture from the source image, and is further constrained by the target parsing map to achieve a clearer objective. At the same time, to encode the source appearance accurately, the self attention among different semantic styles is also added. The effectiveness of our model is validated quantitatively and qualitatively on pose transfer and virtual try-on tasks.