Joint Temporal Pooling for Improving Skeleton-based Action Recognition

TL;DR

This paper introduces JMAP, a novel temporal pooling method that adaptively emphasizes discriminative motion segments in skeleton-based action recognition, outperforming traditional methods.

Contribution

The paper proposes a new Joint Motion Adaptive Temporal Pooling (JMAP) method with frame-wise and joint-wise variants for enhanced action recognition.

Findings

JMAP improves recognition accuracy on NTU RGB+D 120 and PKU-MMD datasets.

JMAP effectively captures discriminative motion segments.

Experimental results demonstrate the superiority of JMAP over conventional pooling methods.

Abstract

In skeleton-based human action recognition, temporal pooling is a critical step for capturing spatiotemporal relationship of joint dynamics. Conventional pooling methods overlook the preservation of motion information and treat each frame equally. However, in an action sequence, only a few segments of frames carry discriminative information related to the action. This paper presents a novel Joint Motion Adaptive Temporal Pooling (JMAP) method for improving skeleton-based action recognition. Two variants of JMAP, frame-wise pooling and joint-wise pooling, are introduced. The efficacy of JMAP has been validated through experiments on the popular NTU RGB+D 120 and PKU-MMD datasets.