3D Human Pose and Shape Estimation via HybrIK-Transformer

TL;DR

This paper introduces an improved 3D human pose estimation method called HybrIK-Transformer, which replaces the deconvolution component with a Transformer to enhance accuracy and efficiency in monocular images.

Contribution

The paper proposes replacing the deconvolution module in HybrIK with a Transformer, significantly improving 3D pose estimation accuracy and computational efficiency.

Findings

Enhanced accuracy on H36M, PW3D, COCO, and HP3D datasets.

Improved computational efficiency over the original HybrIK.

Open-source implementation available.

Abstract

HybrIK relies on a combination of analytical inverse kinematics and deep learning to produce more accurate 3D pose estimation from 2D monocular images. HybrIK has three major components: (1) pretrained convolution backbone, (2) deconvolution to lift 3D pose from 2D convolution features, (3) analytical inverse kinematics pass correcting deep learning prediction using learned distribution of plausible twist and swing angles. In this paper we propose an enhancement of the 2D to 3D lifting module, replacing deconvolution with Transformer, resulting in accuracy and computational efficiency improvement relative to the original HybrIK method. We demonstrate our results on commonly used H36M, PW3D, COCO and HP3D datasets. Our code is publicly available https://github.com/boreshkinai/hybrik-transformer.