Garment Avatars: Realistic Cloth Driving using Pattern Registration

TL;DR

This paper introduces an end-to-end system for creating realistic, drivable clothing models using multi-view pattern registration, enabling high-fidelity virtual garment animation for telepresence and customization.

Contribution

We develop a novel multi-view patterned cloth tracking algorithm and build a fully drivable garment avatar that produces realistic animations from limited views.

Findings

High-accuracy cloth deformation capture

Realistic garment reconstruction from two views

Superior quality compared to state-of-the-art methods

Abstract

Virtual telepresence is the future of online communication. Clothing is an essential part of a person's identity and self-expression. Yet, ground truth data of registered clothes is currently unavailable in the required resolution and accuracy for training telepresence models for realistic cloth animation. Here, we propose an end-to-end pipeline for building drivable representations for clothing. The core of our approach is a multi-view patterned cloth tracking algorithm capable of capturing deformations with high accuracy. We further rely on the high-quality data produced by our tracking method to build a Garment Avatar: an expressive and fully-drivable geometry model for a piece of clothing. The resulting model can be animated using a sparse set of views and produces highly realistic reconstructions which are faithful to the driving signals. We demonstrate the efficacy of our pipeline on a realistic virtual telepresence application, where a garment is being reconstructed from two views, and a user can pick and swap garment design as they wish. In addition, we show a challenging scenario when driven exclusively with body pose, our drivable garment avatar is capable of producing realistic cloth geometry of significantly higher quality than the state-of-the-art.