Divide and Fuse: Body Part Mesh Recovery from Partially Visible Human Images

TL;DR

This paper presents a bottom-up human body mesh reconstruction approach that effectively handles partial visibility and occlusion by independently reconstructing body parts and then fusing them, outperforming traditional top-down methods.

Contribution

The introduction of the Divide and Fuse strategy with Human Part Parametric Models offers a robust solution for partial human mesh recovery, addressing limitations of existing methods.

Findings

Effective reconstruction of partially visible humans in challenging scenarios.

Superior performance over traditional methods in occlusion-heavy images.

Established benchmark datasets for partial human mesh recovery.

Abstract

We introduce a novel bottom-up approach for human body mesh reconstruction, specifically designed to address the challenges posed by partial visibility and occlusion in input images. Traditional top-down methods, relying on whole-body parametric models like SMPL, falter when only a small part of the human is visible, as they require visibility of most of the human body for accurate mesh reconstruction. To overcome this limitation, our method employs a "Divide and Fuse (D&F)" strategy, reconstructing human body parts independently before fusing them, thereby ensuring robustness against occlusions. We design Human Part Parametric Models (HPPM) that independently reconstruct the mesh from a few shape and global-location parameters, without inter-part dependency. A specially designed fusion module then seamlessly integrates the reconstructed parts, even when only a few are visible. We harness a large volume of ground-truth SMPL data to train our parametric mesh models. To facilitate the training and evaluation of our method, we have established benchmark datasets featuring images of partially visible humans with HPPM annotations. Our experiments, conducted on these benchmark datasets, demonstrate the effectiveness of our D&F method, particularly in scenarios with substantial invisibility, where traditional approaches struggle to maintain reconstruction quality.