Two-Stream Appearance Transfer Network for Person Image Generation

TL;DR

This paper introduces a two-stream appearance transfer network (2s-ATN) for pose-guided person image generation, effectively handling large deformations and occlusions to produce realistic images.

Contribution

The paper presents a novel multi-stage two-stream architecture with dense correspondence and feature fusion modules for improved pose-guided person image synthesis.

Findings

Outperforms previous methods on benchmark datasets.

Effectively manages large spatial deformations and occlusions.

Retains detailed appearance information in generated images.

Abstract

Pose guided person image generation means to generate a photo-realistic person image conditioned on an input person image and a desired pose. This task requires spatial manipulation of the source image according to the target pose. However, the generative adversarial networks (GANs) widely used for image generation and translation rely on spatially local and translation equivariant operators, i.e., convolution, pooling and unpooling, which cannot handle large image deformation. This paper introduces a novel two-stream appearance transfer network (2s-ATN) to address this challenge. It is a multi-stage architecture consisting of a source stream and a target stream. Each stage features an appearance transfer module and several two-stream feature fusion modules. The former finds the dense correspondence between the two-stream feature maps and then transfers the appearance information from the source stream to the target stream. The latter exchange local information between the two streams and supplement the non-local appearance transfer. Both quantitative and qualitative results indicate the proposed 2s-ATN can effectively handle large spatial deformation and occlusion while retaining the appearance details. It outperforms prior states of the art on two widely used benchmarks.