TemporalUV: Capturing Loose Clothing with Temporally Coherent UV Coordinates

TL;DR

This paper introduces a method for generating temporally stable UV coordinates for loose clothing and hair, enabling efficient and high-quality garment synthesis without relying on human body outlines.

Contribution

It presents a novel differentiable pipeline that learns UV mappings across sequences, improving temporal coherence and generalization for loose garments.

Findings

Produces high-quality, temporally coherent UV coordinates

Generalizes well to new poses and styles

Reduces computational workload significantly

Abstract

We propose a novel approach to generate temporally coherent UV coordinates for loose clothing. Our method is not constrained by human body outlines and can capture loose garments and hair. We implemented a differentiable pipeline to learn UV mapping between a sequence of RGB inputs and textures via UV coordinates. Instead of treating the UV coordinates of each frame separately, our data generation approach connects all UV coordinates via feature matching for temporal stability. Subsequently, a generative model is trained to balance the spatial quality and temporal stability. It is driven by supervised and unsupervised losses in both UV and image spaces. Our experiments show that the trained models output high-quality UV coordinates and generalize to new poses. Once a sequence of UV coordinates has been inferred by our model, it can be used to flexibly synthesize new looks and modified visual styles. Compared to existing methods, our approach reduces the computational workload to animate new outfits by several orders of magnitude.