Multi-person Pose Tracking using Sequential Monte Carlo with Probabilistic Neural Pose Predictor

TL;DR

This paper introduces a probabilistic neural pose predictor integrated with Sequential Monte Carlo to improve multi-person pose tracking in videos, effectively handling uncertainty and multiple hypotheses to reduce tracking errors.

Contribution

It proposes a novel SMC-based framework with a probabilistic neural predictor that models uncertainty and generates diverse pose hypotheses for better tracking accuracy.

Findings

Achieves state-of-the-art MOTA score on PoseTrack2018 dataset.

Reduces approximately 50% of tracking errors compared to previous methods.

Effectively manages pose disappearance and reappearance scenarios.

Abstract

It is an effective strategy for the multi-person pose tracking task in videos to employ prediction and pose matching in a frame-by-frame manner. For this type of approach, uncertainty-aware modeling is essential because precise prediction is impossible. However, previous studies have relied on only a single prediction without incorporating uncertainty, which can cause critical tracking errors if the prediction is unreliable. This paper proposes an extension to this approach with Sequential Monte Carlo (SMC). This naturally reformulates the tracking scheme to handle multiple predictions (or hypotheses) of poses, thereby mitigating the negative effect of prediction errors. An important component of SMC, i.e., a proposal distribution, is designed as a probabilistic neural pose predictor, which can propose diverse and plausible hypotheses by incorporating epistemic uncertainty and heteroscedastic aleatoric uncertainty. In addition, a recurrent architecture is introduced to our neural modeling to utilize time-sequence information of poses to manage difficult situations, such as the frequent disappearance and reappearances of poses. Compared to existing baselines, the proposed method achieves a state-of-the-art MOTA score on the PoseTrack2018 validation dataset by reducing approximately 50% of tracking errors from a state-of-the art baseline method.