A Pixel-Based Framework for Data-Driven Clothing

TL;DR

This paper introduces a pixel-based computational framework that models 3D cloth deformation as 2D RGB images, enabling the use of CNNs for realistic virtual clothing animation without requiring precise body shape data.

Contribution

The novel approach recasts 3D cloth deformation as 2D images, simplifying the learning process and eliminating the need for accurate body shape or skinning techniques.

Findings

Effective cloth deformation modeling using CNNs in image space

Framework does not require accurate unclothed body shapes

Supports advanced image processing techniques like GANs

Abstract

With the aim of creating virtual cloth deformations more similar to real world clothing, we propose a new computational framework that recasts three dimensional cloth deformation as an RGB image in a two dimensional pattern space. Then a three dimensional animation of cloth is equivalent to a sequence of two dimensional RGB images, which in turn are driven/choreographed via animation parameters such as joint angles. This allows us to leverage popular CNNs to learn cloth deformations in image space. The two dimensional cloth pixels are extended into the real world via standard body skinning techniques, after which the RGB values are interpreted as texture offsets and displacement maps. Notably, we illustrate that our approach does not require accurate unclothed body shapes or robust skinning techniques. Additionally, we discuss how standard image based techniques such as image partitioning for higher resolution, GANs for merging partitioned image regions back together, etc., can readily be incorporated into our framework.