A3GC-IP: Attention-Oriented Adjacency Adaptive Recurrent Graph Convolutions for Human Pose Estimation from Sparse Inertial Measurements

TL;DR

This paper introduces A3GC-LSTM, a novel graph convolutional LSTM network with attention and adjacency adaptivity, enabling accurate human pose estimation from only six sparse IMUs by effectively modeling spatial and temporal dependencies.

Contribution

The paper presents a new attention-oriented, adjacency adaptive graph convolutional LSTM network that improves human pose estimation from sparse IMU data by capturing complex joint dependencies.

Findings

Outperforms state-of-the-art methods on pose estimation accuracy.

Efficiently models long-term temporal dependencies with fewer sensors.

Incorporates spatial attention to enhance joint dependency learning.

Abstract

Conventional methods for human pose estimation either require a high degree of instrumentation, by relying on many inertial measurement units (IMUs), or constraint the recording space, by relying on extrinsic cameras. These deficits are tackled through the approach of human pose estimation from sparse IMU data. We define attention-oriented adjacency adaptive graph convolutional long-short term memory networks (A3GC-LSTM), to tackle human pose estimation based on six IMUs, through incorporating the human body graph structure directly into the network. The A3GC-LSTM combines both spatial and temporal dependency in a single network operation, more memory efficiently than previous approaches. The recurrent graph learning on arbitrarily long sequences is made possible by equipping graph convolutions with adjacency adaptivity, which eliminates the problem of information loss in deep or recurrent graph networks, while it also allows for learning unknown dependencies between the human body joints. To further boost accuracy, a spatial attention formalism is incorporated into the recurrent LSTM cell. With our presented approach, we are able to utilize the inherent graph nature of the human body, and thus can outperform the state of the art for human pose estimation from sparse IMU data.