Concise and Effective Network for 3D Human Modeling from Orthogonal Silhouettes

TL;DR

This paper introduces a new CNN-based method for 3D human modeling from orthogonal silhouettes, achieving higher accuracy with fewer parameters by leveraging a novel architecture and transfer learning.

Contribution

A novel CNN architecture for 3D human modeling from silhouettes that reduces parameter count and improves accuracy using transfer learning on limited datasets.

Findings

High-accuracy 3D models generated from only 2.4M coefficients

The proposed network outperforms existing CNN approaches in parameter efficiency

Transfer learning enables gender- and region-specific modeling

Abstract

In this paper, we revisit the problem of 3D human modeling from two orthogonal silhouettes of individuals (i.e., front and side views). Different from our prior work, a supervised learning approach based on convolutional neural network (CNN) is investigated to solve the problem by establishing a mapping function that can effectively extract features from two silhouettes and fuse them into coefficients in the shape space of human bodies. A new CNN structure is proposed in our work to exact not only the discriminative features of front and side views and also their mixed features for the mapping function. 3D human models with high accuracy are synthesized from coefficients generated by the mapping function. Existing CNN approaches for 3D human modeling usually learn a large number of parameters (from 8.5M to 355.4M) from two binary images. Differently, we investigate a new network architecture and conduct the samples on silhouettes as input. As a consequence, more accurate models can be generated by our network with only 2.4M coefficients. The training of our network is conducted on samples obtained by augmenting a publicly accessible dataset. Learning transfer by using datasets with a smaller number of scanned models is applied to our network to enable the function of generating results with gender-oriented (or geographical) patterns.