3D Human Pose Lifting with Grid Convolution

TL;DR

This paper introduces Grid Convolution, a novel approach for 3D human pose lifting from 2D poses, using a regular grid representation to enable convolutional feature learning, outperforming existing graph-based methods.

Contribution

The paper proposes GridConv with Semantic Grid Transformation for 3D human pose lifting, including handcrafted and learnable SGT, and an attention module to enhance contextual encoding.

Findings

Outperforms state-of-the-art methods on Human3.6M and MPI-INF-3DHP datasets.

Learnable SGT achieves better results than handcrafted design.

Attention-enhanced GridConv improves encoding of contextual cues.

Abstract

Existing lifting networks for regressing 3D human poses from 2D single-view poses are typically constructed with linear layers based on graph-structured representation learning. In sharp contrast to them, this paper presents Grid Convolution (GridConv), mimicking the wisdom of regular convolution operations in image space. GridConv is based on a novel Semantic Grid Transformation (SGT) which leverages a binary assignment matrix to map the irregular graph-structured human pose onto a regular weave-like grid pose representation joint by joint, enabling layer-wise feature learning with GridConv operations. We provide two ways to implement SGT, including handcrafted and learnable designs. Surprisingly, both designs turn out to achieve promising results and the learnable one is better, demonstrating the great potential of this new lifting representation learning formulation. To improve the ability of GridConv to encode contextual cues, we introduce an attention module over the convolutional kernel, making grid convolution operations input-dependent, spatial-aware and grid-specific. We show that our fully convolutional grid lifting network outperforms state-of-the-art methods with noticeable margins under (1) conventional evaluation on Human3.6M and (2) cross-evaluation on MPI-INF-3DHP. Code is available at https://github.com/OSVAI/GridConv