Occlusion Handling in 3D Human Pose Estimation with Perturbed Positional Encoding

TL;DR

This paper introduces PerturbPE, a novel positional encoding method for GCNs in 3D human pose estimation, improving robustness and accuracy especially under occlusion scenarios by handling missing graph edges.

Contribution

The paper proposes PerturbPE, a new eigenbasis-based positional encoding that uses perturbations to improve GCN robustness in occlusion scenarios for 3D human pose estimation.

Findings

Up to 12% performance improvement on Human3.6M with occlusion.

Significant enhancement when two edges are missing.

Sets new benchmarks for occlusion robustness.

Abstract

Understanding human behavior fundamentally relies on accurate 3D human pose estimation. Graph Convolutional Networks (GCNs) have recently shown promising advancements, delivering state-of-the-art performance with rather lightweight architectures. In the context of graph-structured data, leveraging the eigenvectors of the graph Laplacian matrix for positional encoding is effective. Yet, the approach does not specify how to handle scenarios where edges in the input graph are missing. To this end, we propose a novel positional encoding technique, PerturbPE, that extracts consistent and regular components from the eigenbasis. Our method involves applying multiple perturbations and taking their average to extract the consistent and regular component from the eigenbasis. PerturbPE leverages the Rayleigh-Schrodinger Perturbation Theorem (RSPT) for calculating the perturbed eigenvectors. Employing this labeling technique enhances the robustness and generalizability of the model. Our results support our theoretical findings, e.g. our experimental analysis observed a performance enhancement of up to $12\%$ on the Human3.6M dataset in instances where occlusion resulted in the absence of one edge. Furthermore, our novel approach significantly enhances performance in scenarios where two edges are missing, setting a new benchmark for state-of-the-art.