Deep Image Spatial Transformation for Person Image Generation

TL;DR

This paper introduces a differentiable global-flow local-attention framework for pose-guided person image generation, enabling effective spatial transformations at the feature level, outperforming previous methods in quality and applicability.

Contribution

The paper proposes a novel global-flow local-attention mechanism that improves spatial transformation capabilities in person image generation tasks.

Findings

Superior subjective and objective results compared to existing methods

Effective in video animation and view synthesis tasks

Demonstrates the model's versatility in spatial transformation applications

Abstract

Pose-guided person image generation is to transform a source person image to a target pose. This task requires spatial manipulations of source data. However, Convolutional Neural Networks are limited by the lack of ability to spatially transform the inputs. In this paper, we propose a differentiable global-flow local-attention framework to reassemble the inputs at the feature level. Specifically, our model first calculates the global correlations between sources and targets to predict flow fields. Then, the flowed local patch pairs are extracted from the feature maps to calculate the local attention coefficients. Finally, we warp the source features using a content-aware sampling method with the obtained local attention coefficients. The results of both subjective and objective experiments demonstrate the superiority of our model. Besides, additional results in video animation and view synthesis show that our model is applicable to other tasks requiring spatial transformation. Our source code is available at https://github.com/RenYurui/Global-Flow-Local-Attention.