Localization and Tracking of User-Defined Points on Deformable Objects for Robotic Manipulation

TL;DR

This paper presents a real-time method for localizing and tracking user-defined points on deformable objects using a discretized deformation model and a multi-step solver, suitable for industrial robotics.

Contribution

It introduces a novel online localization and tracking approach for deformable objects using a high-dimensional energy minimization with regularization, enabling efficient perception in robotics.

Findings

Capable of online localization and tracking in real-time.

Uses a discretized deformation field estimated via a multi-step solver.

Suitable for perception tasks in industrial manufacturing.

Abstract

This paper introduces an efficient procedure to localize user-defined points on the surface of deformable objects and track their positions in 3D space over time. To cope with a deformable object's infinite number of DOF, we propose a discretized deformation field, which is estimated during runtime using a multi-step non-linear solver pipeline. The resulting high-dimensional energy minimization problem describes the deviation between an offline-defined reference model and a pre-processed camera image. An additional regularization term allows for assumptions about the object's hidden areas and increases the solver's numerical stability. Our approach is capable of solving the localization problem online in a data-parallel manner, making it ideally suitable for the perception of non-rigid objects in industrial manufacturing processes.