Pose Guided Image Generation from Misaligned Sources via Residual Flow Based Correction

TL;DR

This paper introduces a unified method for pose-guided image generation from misaligned sources, effectively handling diverse variations like view angles and expressions across different datasets, outperforming existing techniques.

Contribution

A novel general framework that models multiple variations among source images, enabling robust image generation from misaligned inputs across various data types.

Findings

Outperforms state-of-the-art methods in qualitative and quantitative evaluations.

Effective across diverse datasets including humans, faces, city scenes, and 3D objects.

Handles large misalignments without strong assumptions on camera or object.

Abstract

Generating new images with desired properties (e.g. new view/poses) from source images has been enthusiastically pursued recently, due to its wide range of potential applications. One way to ensure high-quality generation is to use multiple sources with complementary information such as different views of the same object. However, as source images are often misaligned due to the large disparities among the camera settings, strong assumptions have been made in the past with respect to the camera(s) or/and the object in interest, limiting the application of such techniques. Therefore, we propose a new general approach which models multiple types of variations among sources, such as view angles, poses, facial expressions, in a unified framework, so that it can be employed on datasets of vastly different nature. We verify our approach on a variety of data including humans bodies, faces, city scenes and 3D objects. Both the qualitative and quantitative results demonstrate the better performance of our method than the state of the art.