A Structure-aware and Motion-adaptive Framework for 3D Human Pose Estimation with Mamba

TL;DR

This paper introduces SAMA, a novel framework for 3D human pose estimation that models joint topology and motion dynamics independently, improving accuracy and efficiency over previous methods.

Contribution

The paper proposes a structure-aware and motion-adaptive framework with two modules, SSI and MSM, to better capture joint relationships and motion characteristics in 3D pose estimation.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Reduces computational costs compared to existing methods.

Effectively models joint topology and motion dynamics independently.

Abstract

Recent Mamba-based methods for the pose-lifting task tend to model joint dependencies by 2D-to-1D mapping with diverse scanning strategies. Though effective, they struggle to model intricate joint connections and uniformly process all joint motion trajectories while neglecting the intrinsic differences across motion characteristics. In this work, we propose a structure-aware and motion-adaptive framework to capture spatial joint topology along with diverse motion dynamics independently, named as SAMA. Specifically, SAMA consists of a Structure-aware State Integrator (SSI) and a Motion-adaptive State Modulator (MSM). The Structure-aware State Integrator is tasked with leveraging dynamic joint relationships to fuse information at both the joint feature and state levels in the state space, based on pose topology rather than sequential state transitions. The Motion-adaptive State Modulator is responsible for joint-specific motion characteristics recognition, thus applying tailored adjustments to diverse motion patterns across different joints. Through the above key modules, our algorithm enables structure-aware and motion-adaptive pose lifting. Extensive experiments across multiple benchmarks demonstrate that our algorithm achieves advanced results with fewer computational costs.