NRST: Non-rigid Surface Tracking from Monocular Video

TL;DR

This paper introduces an efficient monocular video-based non-rigid surface tracking method that leverages micro-structural texture orientation to improve accuracy on uniform textured objects like fabrics.

Contribution

It presents a novel texture term exploiting micro-structure orientations for better tracking of uniform textured surfaces in monocular videos.

Findings

Effective tracking of fabrics with micro-structures.

Improved accuracy over traditional photometric methods.

Applicable to various non-rigid objects.

Abstract

We propose an efficient method for non-rigid surface tracking from monocular RGB videos. Given a video and a template mesh, our algorithm sequentially registers the template non-rigidly to each frame. We formulate the per-frame registration as an optimization problem that includes a novel texture term specifically tailored towards tracking objects with uniform texture but fine-scale structure, such as the regular micro-structural patterns of fabric. Our texture term exploits the orientation information in the micro-structures of the objects, e.g., the yarn patterns of fabrics. This enables us to accurately track uniformly colored materials that have these high frequency micro-structures, for which traditional photometric terms are usually less effective. The results demonstrate the effectiveness of our method on both general textured non-rigid objects and monochromatic fabrics.