Learning Correspondence for Deformable Objects

TL;DR

This paper compares classical and learning-based pixelwise correspondence methods for deformable objects like cloth and rope, introduces a new learning approach extending Dense Object Nets, and provides a standardized evaluation framework.

Contribution

It presents a synthetic data generation framework, a novel extension of Dense Object Nets, and a comprehensive comparison of state-of-the-art correspondence methods.

Findings

Dense Object Nets outperform classical feature-matching methods.

The proposed extension of Dense Object Nets performs similarly to the original.

Synthetic data transfer demonstrates potential for real-world applications.

Abstract

We investigate the problem of pixelwise correspondence for deformable objects, namely cloth and rope, by comparing both classical and learning-based methods. We choose cloth and rope because they are traditionally some of the most difficult deformable objects to analytically model with their large configuration space, and they are meaningful in the context of robotic tasks like cloth folding, rope knot-tying, T-shirt folding, curtain closing, etc. The correspondence problem is heavily motivated in robotics, with wide-ranging applications including semantic grasping, object tracking, and manipulation policies built on top of correspondences. We present an exhaustive survey of existing classical methods for doing correspondence via feature-matching, including SIFT, SURF, and ORB, and two recently published learning-based methods including TimeCycle and Dense Object Nets. We make three main contributions: (1) a framework for simulating and rendering synthetic images of deformable objects, with qualitative results demonstrating transfer between our simulated and real domains (2) a new learning-based correspondence method extending Dense Object Nets, and (3) a standardized comparison across state-of-the-art correspondence methods. Our proposed method provides a flexible, general formulation for learning temporally and spatially continuous correspondences for nonrigid (and rigid) objects. We report root mean squared error statistics for all methods and find that Dense Object Nets outperforms baseline classical methods for correspondence, and our proposed extension of Dense Object Nets performs similarly.