Leveraging Intrinsic Properties for Non-Rigid Garment Alignment

TL;DR

This paper introduces a novel two-stage method leveraging intrinsic manifold properties and neural deformation fields to achieve high-precision non-rigid garment alignment at wrinkle and texture levels, outperforming traditional extrinsic and isometric methods.

Contribution

It proposes a coarse-to-fine approach combining intrinsic manifold properties with neural deformation fields for improved non-rigid garment alignment.

Findings

Achieves accurate wrinkle-level and texture-level alignment.

Effective on challenging garment types like long coats.

Outperforms existing extrinsic and isometric methods.

Abstract

We address the problem of aligning real-world 3D data of garments, which benefits many applications such as texture learning, physical parameter estimation, generative modeling of garments, etc. Existing extrinsic methods typically perform non-rigid iterative closest point and struggle to align details due to incorrect closest matches and rigidity constraints. While intrinsic methods based on functional maps can produce high-quality correspondences, they work under isometric assumptions and become unreliable for garment deformations which are highly non-isometric. To achieve wrinkle-level as well as texture-level alignment, we present a novel coarse-to-fine two-stage method that leverages intrinsic manifold properties with two neural deformation fields, in the 3D space and the intrinsic space, respectively. The coarse stage performs a 3D fitting, where we leverage intrinsic manifold properties to define a manifold deformation field. The coarse fitting then induces a functional map that produces an alignment of intrinsic embeddings. We further refine the intrinsic alignment with a second neural deformation field for higher accuracy. We evaluate our method with our captured garment dataset, GarmCap. The method achieves accurate wrinkle-level and texture-level alignment and works for difficult garment types such as long coats. Our project page is https://jsnln.github.io/iccv2023_intrinsic/index.html.