Occupancy Planes for Single-view RGB-D Human Reconstruction

TL;DR

This paper introduces occupancy planes (OPlanes), a novel representation for single-view RGB-D human reconstruction that improves accuracy by capturing correlations across neighboring locations, especially in occluded or partial views.

Contribution

The authors propose OPlanes, a flexible plane-based occupancy representation that enhances reconstruction accuracy over traditional per-point classification methods.

Findings

OPlanes outperform voxel grids in reconstruction tasks.

The method handles occlusions and partial views effectively.

Results on S3D data show significant improvements.

Abstract

Single-view RGB-D human reconstruction with implicit functions is often formulated as per-point classification. Specifically, a set of 3D locations within the view-frustum of the camera are first projected independently onto the image and a corresponding feature is subsequently extracted for each 3D location. The feature of each 3D location is then used to classify independently whether the corresponding 3D point is inside or outside the observed object. This procedure leads to sub-optimal results because correlations between predictions for neighboring locations are only taken into account implicitly via the extracted features. For more accurate results we propose the occupancy planes (OPlanes) representation, which enables to formulate single-view RGB-D human reconstruction as occupancy prediction on planes which slice through the camera's view frustum. Such a representation provides more flexibility than voxel grids and enables to better leverage correlations than per-point classification. On the challenging S3D data we observe a simple classifier based on the OPlanes representation to yield compelling results, especially in difficult situations with partial occlusions due to other objects and partial visibility, which haven't been addressed by prior work.