GRPose: Learning Graph Relations for Human Image Generation with Pose Priors

TL;DR

GRPose introduces a graph-based approach to human image generation with pose priors, leveraging a hierarchical graph integrator and pose perception loss to improve pose consistency and image quality in diffusion models.

Contribution

It proposes a novel graph relation framework and a Progressive Graph Integrator for better pose control in diffusion-based human image synthesis.

Findings

Significant performance improvements over benchmarks.

Effective capture of pose-part relationships.

Enhanced pose alignment and image quality.

Abstract

Recent methods using diffusion models have made significant progress in human image generation with various control signals such as pose priors. However, existing efforts are still struggling to generate high-quality images with consistent pose alignment, resulting in unsatisfactory output. In this paper, we propose a framework that delves into the graph relations of pose priors to provide control information for human image generation. The main idea is to establish a graph topological structure between the pose priors and latent representation of diffusion models to capture the intrinsic associations between different pose parts. A Progressive Graph Integrator (PGI) is designed to learn the spatial relationships of the pose priors with the graph structure, adopting a hierarchical strategy within an Adapter to gradually propagate information across different pose parts. Besides, a pose perception loss is introduced based on a pretrained pose estimation network to minimize the pose differences. Extensive qualitative and quantitative experiments conducted on the Human-Art and LAION-Human datasets clearly demonstrate that our model can achieve significant performance improvement over the latest benchmark models. The code is available at \url{https://xiangchenyin.github.io/GRPose/}.