Multi-hop graph transformer network for 3D human pose estimation

TL;DR

This paper presents a multi-hop graph transformer network that effectively captures spatio-temporal dependencies for 3D human pose estimation from videos, addressing occlusion and depth ambiguity challenges.

Contribution

It introduces a novel architecture combining multi-head self-attention and multi-hop graph convolutional networks with disentangled neighborhoods for improved 3D pose estimation.

Findings

Achieves competitive results on benchmark datasets.

Effectively models long-range spatio-temporal dependencies.

Demonstrates strong generalization ability.

Abstract

Accurate 3D human pose estimation is a challenging task due to occlusion and depth ambiguity. In this paper, we introduce a multi-hop graph transformer network designed for 2D-to-3D human pose estimation in videos by leveraging the strengths of multi-head self-attention and multi-hop graph convolutional networks with disentangled neighborhoods to capture spatio-temporal dependencies and handle long-range interactions. The proposed network architecture consists of a graph attention block composed of stacked layers of multi-head self-attention and graph convolution with learnable adjacency matrix, and a multi-hop graph convolutional block comprised of multi-hop convolutional and dilated convolutional layers. The combination of multi-head self-attention and multi-hop graph convolutional layers enables the model to capture both local and global dependencies, while the integration of dilated convolutional layers enhances the model's ability to handle spatial details required for accurate localization of the human body joints. Extensive experiments demonstrate the effectiveness and generalization ability of our model, achieving competitive performance on benchmark datasets.