Object-centric Task Representation and Transfer using Diffused Orientation Fields

TL;DR

This paper introduces Diffused Orientation Fields (DOF), a smooth local reference frame representation that enables effective transfer of manipulation tasks across curved objects in robotics, addressing the challenge posed by their complex geometries.

Contribution

The paper presents a novel method using DOF for task transfer across curved objects, computed online from point clouds, improving robustness and applicability in robotic manipulation.

Findings

Successful transfer of tasks like inspection, slicing, and peeling across varied objects.

DOF demonstrates robustness under geometric, topological, and localization perturbations.

Open-source implementation provided for reproducibility and further research.

Abstract

Curved objects pose a fundamental challenge for skill transfer in robotics: unlike planar surfaces, they do not admit a global reference frame. As a result, task-relevant directions such as "toward" or "along" the surface vary with position and geometry, making object-centric tasks difficult to transfer across shapes. To address this, we introduce an approach using Diffused Orientation Fields (DOF), a smooth representation of local reference frames, for transfer learning of tasks across curved objects. By expressing manipulation tasks in these smoothly varying local frames, we reduce the problem of transferring tasks across curved objects to establishing sparse keypoint correspondences. DOF is computed online from raw point cloud data using diffusion processes governed by partial differential equations, conditioned on keypoints. We evaluate DOF under geometric, topological, and localization perturbations, and demonstrate successful transfer of tasks requiring continuous physical interaction such as inspection, slicing, and peeling across varied objects. We provide our open-source codes at our website https://github.com/idiap/diffused_fields_robotics