HDFormer: High-order Directed Transformer for 3D Human Pose Estimation

TL;DR

HDFormer introduces a high-order attention transformer that models complex joint relationships for more accurate and efficient 3D human pose estimation, especially in occlusion-heavy scenarios, outperforming existing methods.

Contribution

The paper proposes a novel high-order attention mechanism within a transformer architecture for 3D pose estimation, effectively capturing complex joint interactions and reducing computational costs.

Findings

Outperforms state-of-the-art on Human3.6M and MPI-INF-3DHP datasets.

Uses only 1/10 of the parameters of comparable models.

Enables real-time 3D human pose estimation.

Abstract

Human pose estimation is a challenging task due to its structured data sequence nature. Existing methods primarily focus on pair-wise interaction of body joints, which is insufficient for scenarios involving overlapping joints and rapidly changing poses. To overcome these issues, we introduce a novel approach, the High-order Directed Transformer (HDFormer), which leverages high-order bone and joint relationships for improved pose estimation. Specifically, HDFormer incorporates both self-attention and high-order attention to formulate a multi-order attention module. This module facilitates first-order "joint$\leftrightarrow$joint", second-order "bone$\leftrightarrow$joint", and high-order "hyperbone$\leftrightarrow$joint" interactions, effectively addressing issues in complex and occlusion-heavy situations. In addition, modern CNN techniques are integrated into the transformer-based architecture, balancing the trade-off between performance and efficiency. HDFormer significantly outperforms state-of-the-art (SOTA) models on Human3.6M and MPI-INF-3DHP datasets, requiring only 1/10 of the parameters and significantly lower computational costs. Moreover, HDFormer demonstrates broad real-world applicability, enabling real-time, accurate 3D pose estimation. The source code is in https://github.com/hyer/HDFormer