Learning Spatial Context with Graph Neural Network for Multi-Person Pose Grouping

TL;DR

This paper introduces a graph neural network that incorporates spatial and appearance information to improve multi-person pose grouping, significantly outperforming existing methods on benchmark datasets.

Contribution

We propose a Geometry-aware Association GNN that leverages spatial configuration of keypoints for more accurate grouping in multi-person pose estimation.

Findings

Outperforms existing appearance-only grouping methods on benchmarks.

Utilizes spatial context to improve keypoint association accuracy.

Fuses spatial and appearance affinities for robust pose grouping.

Abstract

Bottom-up approaches for image-based multi-person pose estimation consist of two stages: (1) keypoint detection and (2) grouping of the detected keypoints to form person instances. Current grouping approaches rely on learned embedding from only visual features that completely ignore the spatial configuration of human poses. In this work, we formulate the grouping task as a graph partitioning problem, where we learn the affinity matrix with a Graph Neural Network (GNN). More specifically, we design a Geometry-aware Association GNN that utilizes spatial information of the keypoints and learns local affinity from the global context. The learned geometry-based affinity is further fused with appearance-based affinity to achieve robust keypoint association. Spectral clustering is used to partition the graph for the formation of the pose instances. Experimental results on two benchmark datasets show that our proposed method outperforms existing appearance-only grouping frameworks, which shows the effectiveness of utilizing spatial context for robust grouping. Source code is available at: https://github.com/jiahaoLjh/PoseGrouping.