Body Knowledge and Uncertainty Modeling for Monocular 3D Human Body Reconstruction

TL;DR

The paper introduces KNOWN, a framework that leverages body knowledge and uncertainty modeling to improve monocular 3D human body reconstruction without relying on extensive 3D supervision, addressing data noise and imbalance.

Contribution

KNOWN utilizes generic body constraints and a probabilistic uncertainty framework to enable effective training without 3D data and handle dataset inconsistencies.

Findings

Outperforms prior weakly-supervised methods

Effective on minority images with unusual poses

Utilizes uncertainty to guide model refinement

Abstract

While 3D body reconstruction methods have made remarkable progress recently, it remains difficult to acquire the sufficiently accurate and numerous 3D supervisions required for training. In this paper, we propose \textbf{KNOWN}, a framework that effectively utilizes body \textbf{KNOW}ledge and u\textbf{N}certainty modeling to compensate for insufficient 3D supervisions. KNOWN exploits a comprehensive set of generic body constraints derived from well-established body knowledge. These generic constraints precisely and explicitly characterize the reconstruction plausibility and enable 3D reconstruction models to be trained without any 3D data. Moreover, existing methods typically use images from multiple datasets during training, which can result in data noise (\textit{e.g.}, inconsistent joint annotation) and data imbalance (\textit{e.g.}, minority images representing unusual poses or captured from challenging camera views). KNOWN solves these problems through a novel probabilistic framework that models both aleatoric and epistemic uncertainty. Aleatoric uncertainty is encoded in a robust Negative Log-Likelihood (NLL) training loss, while epistemic uncertainty is used to guide model refinement. Experiments demonstrate that KNOWN's body reconstruction outperforms prior weakly-supervised approaches, particularly on the challenging minority images.