Patch-based Representation and Learning for Efficient Deformation Modeling

TL;DR

This paper introduces PolyFit, a patch-based surface representation that enables efficient deformation modeling, learning from data, and rapid application in tasks like shape deformation and garment draping.

Contribution

The paper proposes PolyFit, a novel patch-based surface representation that allows fast, data-driven surface deformation applicable to various tasks in vision and graphics.

Findings

PolyFit enables faster deformation than traditional methods.

Test-time optimization with PolyFit achieves competitive accuracy.

The garment model generalizes across resolutions and types.

Abstract

In this paper, we present a patch-based representation of surfaces, PolyFit, which is obtained by fitting jet functions locally on surface patches. Such a representation can be learned efficiently in a supervised fashion from both analytic functions and real data. Once learned, it can be generalized to various types of surfaces. Using PolyFit, the surfaces can be efficiently deformed by updating a compact set of jet coefficients rather than optimizing per-vertex degrees of freedom for many downstream tasks in computer vision and graphics. We demonstrate the capabilities of our proposed methodologies with two applications: 1) Shape-from-template (SfT): where the goal is to deform the input 3D template of an object as seen in image/video. Using PolyFit, we adopt test-time optimization that delivers competitive accuracy while being markedly faster than offline physics-based solvers, and outperforms recent physics-guided neural simulators in accuracy at modest additional runtime. 2) Garment draping. We train a self-supervised, mesh- and garment-agnostic model that generalizes across resolutions and garment types, delivering up to an order-of-magnitude faster inference than strong baselines.