Towards Using Clothes Style Transfer for Scenario-aware Person Video Generation

TL;DR

This paper introduces a novel clothes style transfer framework for person video generation that enhances detail preservation and temporal consistency, enabling better scenario adaptation and outperforming existing methods.

Contribution

A new disentangled multi-branch encoder and inner-frame discriminator are proposed to improve detail, coherence, and scenario adaptation in clothes style transfer for videos.

Findings

Outperforms state-of-the-art in image quality

Achieves superior video coherence

Demonstrates effective scenario adaptation

Abstract

Clothes style transfer for person video generation is a challenging task, due to drastic variations of intra-person appearance and video scenarios. To tackle this problem, most recent AdaIN-based architectures are proposed to extract clothes and scenario features for generation. However, these approaches suffer from being short of fine-grained details and are prone to distort the origin person. To further improve the generation performance, we propose a novel framework with disentangled multi-branch encoders and a shared decoder. Moreover, to pursue the strong video spatio-temporal consistency, an inner-frame discriminator is delicately designed with input being cross-frame difference. Besides, the proposed framework possesses the property of scenario adaptation. Extensive experiments on the TEDXPeople benchmark demonstrate the superiority of our method over state-of-the-art approaches in terms of image quality and video coherence.