Multi-person Implicit Reconstruction from a Single Image

TL;DR

This paper introduces an end-to-end, model-free deep learning framework for detailed 3D reconstruction of multiple clothed humans from a single image, overcoming limitations of prior methods.

Contribution

It presents the first end-to-end learning approach for multi-person implicit 3D reconstruction from a single image, handling occlusions and loose clothing without manual intervention.

Findings

Achieves high-resolution, accurate multi-human reconstructions with complex occlusions.

Outperforms existing methods in accuracy and completeness on synthetic and real datasets.

Introduces a new synthetic dataset for multi-human 3D reconstruction.

Abstract

We present a new end-to-end learning framework to obtain detailed and spatially coherent reconstructions of multiple people from a single image. Existing multi-person methods suffer from two main drawbacks: they are often model-based and therefore cannot capture accurate 3D models of people with loose clothing and hair; or they require manual intervention to resolve occlusions or interactions. Our method addresses both limitations by introducing the first end-to-end learning approach to perform model-free implicit reconstruction for realistic 3D capture of multiple clothed people in arbitrary poses (with occlusions) from a single image. Our network simultaneously estimates the 3D geometry of each person and their 6DOF spatial locations, to obtain a coherent multi-human reconstruction. In addition, we introduce a new synthetic dataset that depicts images with a varying number of inter-occluded humans and a variety of clothing and hair styles. We demonstrate robust, high-resolution reconstructions on images of multiple humans with complex occlusions, loose clothing and a large variety of poses and scenes. Our quantitative evaluation on both synthetic and real-world datasets demonstrates state-of-the-art performance with significant improvements in the accuracy and completeness of the reconstructions over competing approaches.