Bipartite Graph Reasoning GANs for Person Pose and Facial Image Synthesis

TL;DR

This paper introduces BiGraphGAN, a novel GAN architecture utilizing bipartite graph reasoning to improve person pose and facial image synthesis by modeling complex pose-to-pose and pose-to-image relations.

Contribution

It proposes new bipartite graph reasoning blocks and interaction-aggregation mechanisms to better model pose transformations and local part details in image synthesis.

Findings

Outperforms existing methods on multiple datasets

Achieves higher quantitative scores in pose and facial synthesis

Produces more realistic and detailed generated images

Abstract

We present a novel bipartite graph reasoning Generative Adversarial Network (BiGraphGAN) for two challenging tasks: person pose and facial image synthesis. The proposed graph generator consists of two novel blocks that aim to model the pose-to-pose and pose-to-image relations, respectively. Specifically, the proposed bipartite graph reasoning (BGR) block aims to reason the long-range cross relations between the source and target pose in a bipartite graph, which mitigates some of the challenges caused by pose deformation. Moreover, we propose a new interaction-and-aggregation (IA) block to effectively update and enhance the feature representation capability of both a person's shape and appearance in an interactive way. To further capture the change in pose of each part more precisely, we propose a novel part-aware bipartite graph reasoning (PBGR) block to decompose the task of reasoning the global structure transformation with a bipartite graph into learning different local transformations for different semantic body/face parts. Experiments on two challenging generation tasks with three public datasets demonstrate the effectiveness of the proposed methods in terms of objective quantitative scores and subjective visual realness. The source code and trained models are available at https://github.com/Ha0Tang/BiGraphGAN.