Self-Supervision and Spatial-Sequential Attention Based Loss for Multi-Person Pose Estimation

TL;DR

This paper introduces a novel loss function and attention mechanism for multi-person pose estimation, significantly improving accuracy by better supervision and feature extraction, validated on the MS COCO dataset.

Contribution

It proposes a new loss organization with self-supervised heatmaps and spatial-sequential attention, enhancing feature utilization and prediction consistency in pose estimation.

Findings

Outperforms baseline by over 5.5% mAP on COCO dataset

Reduces prediction contradictions with self-supervised heatmaps

Enhances feature extraction using spatial-sequential attention

Abstract

Bottom-up based multi-person pose estimation approaches use heatmaps with auxiliary predictions to estimate joint positions and belonging at one time. Recently, various combinations between auxiliary predictions and heatmaps have been proposed for higher performance, these predictions are supervised by the corresponding L2 loss function directly. However, the lack of more explicit supervision results in low features utilization and contradictions between predictions in one model. To solve these problems, this paper proposes (i) a new loss organization method which uses self-supervised heatmaps to reduce prediction contradictions and spatial-sequential attention to enhance networks' features extraction; (ii) a new combination of predictions composed by heatmaps, Part Affinity Fields (PAFs) and our block-inside offsets to fix pixel-level joints positions and further demonstrates the effectiveness of proposed loss function. Experiments are conducted on the MS COCO keypoint dataset and adopting OpenPose as the baseline model. Our method outperforms the baseline overall. On the COCO verification dataset, the mAP of OpenPose trained with our proposals outperforms the OpenPose baseline by over 5.5%.