Multi-Garment Net: Learning to Dress 3D People from Images

TL;DR

This paper introduces Multi-Garment Network (MGN), a novel method for predicting detailed 3D clothing and body shape from a few video frames, enabling realistic dressing of diverse body shapes and poses.

Contribution

The paper presents MGN, a new approach that predicts layered clothing on 3D human models from limited video frames, utilizing a large digital wardrobe and a novel registration method.

Findings

MGN achieves high-quality garment prediction from few frames.

The digital wardrobe enables diverse clothing transfer.

The model allows dressing arbitrary body shapes and poses.

Abstract

We present Multi-Garment Network (MGN), a method to predict body shape and clothing, layered on top of the SMPL model from a few frames (1-8) of a video. Several experiments demonstrate that this representation allows higher level of control when compared to single mesh or voxel representations of shape. Our model allows to predict garment geometry, relate it to the body shape, and transfer it to new body shapes and poses. To train MGN, we leverage a digital wardrobe containing 712 digital garments in correspondence, obtained with a novel method to register a set of clothing templates to a dataset of real 3D scans of people in different clothing and poses. Garments from the digital wardrobe, or predicted by MGN, can be used to dress any body shape in arbitrary poses. We will make publicly available the digital wardrobe, the MGN model, and code to dress SMPL with the garments.