How Robust is 3D Human Pose Estimation to Occlusion?

TL;DR

This paper investigates the robustness of state-of-the-art 3D human pose estimation methods to occlusions, revealing their sensitivity and proposing data augmentation with synthetic occlusions to improve robustness and regularization.

Contribution

It provides a systematic study of occlusion effects on 3D pose estimation and introduces a data augmentation technique to enhance robustness against occlusions.

Findings

State-of-the-art methods are sensitive to even low levels of occlusion.

Synthetic occlusion augmentation improves robustness and acts as a regularizer.

Augmentation benefits extend to non-occluded test cases.

Abstract

Occlusion is commonplace in realistic human-robot shared environments, yet its effects are not considered in standard 3D human pose estimation benchmarks. This leaves the question open: how robust are state-of-the-art 3D pose estimation methods against partial occlusions? We study several types of synthetic occlusions over the Human3.6M dataset and find a method with state-of-the-art benchmark performance to be sensitive even to low amounts of occlusion. Addressing this issue is key to progress in applications such as collaborative and service robotics. We take a first step in this direction by improving occlusion-robustness through training data augmentation with synthetic occlusions. This also turns out to be an effective regularizer that is beneficial even for non-occluded test cases.