Adaptive Appearance Rendering

TL;DR

This paper introduces an adaptive appearance rendering method that generates realistic images and videos of people by disentangling pose and appearance, using neural networks to encode and combine these features effectively.

Contribution

It presents a novel neural network architecture that encodes pose and appearance separately and combines them for high-quality image and video synthesis.

Findings

Outperforms state-of-the-art methods in image quality

Handles pose-guided appearance rendering in videos

Demonstrates effective disentanglement of pose and appearance

Abstract

We propose an approach to generate images of people given a desired appearance and pose. Disentangled representations of pose and appearance are necessary to handle the compound variability in the resulting generated images. Hence, we develop an approach based on intermediate representations of poses and appearance: our pose-guided appearance rendering network firstly encodes the targets' poses using an encoder-decoder neural network. Then the targets' appearances are encoded by learning adaptive appearance filters using a fully convolutional network. Finally, these filters are placed in the encoder-decoder neural networks to complete the rendering. We demonstrate that our model can generate images and videos that are superior to state-of-the-art methods, and can handle pose guided appearance rendering in both image and video generation.