Perceiving Extrinsic Contacts from Touch Improves Learning Insertion Policies

TL;DR

This paper enhances Neural Contact Fields for better sim-to-real transfer, enabling robots to perceive extrinsic contacts during object insertion, which significantly improves success rates and speed in real-world tasks.

Contribution

The authors improve NCF to NCF-v2 for real-world applicability and demonstrate its effectiveness in robotic insertion tasks with substantial performance gains.

Findings

NCF-v2 accurately estimates extrinsic contacts in real-world scenarios.

Insertion policies with NCF-v2 outperform those without, with up to 33% higher success.

Policies using NCF-v2 are 1.36x and 1.27x faster in respective tasks.

Abstract

Robotic manipulation tasks such as object insertion typically involve interactions between object and environment, namely extrinsic contacts. Prior work on Neural Contact Fields (NCF) use intrinsic tactile sensing between gripper and object to estimate extrinsic contacts in simulation. However, its effectiveness and utility in real-world tasks remains unknown. In this work, we improve NCF to enable sim-to-real transfer and use it to train policies for mug-in-cupholder and bowl-in-dishrack insertion tasks. We find our model NCF-v2, is capable of estimating extrinsic contacts in the real-world. Furthermore, our insertion policy with NCF-v2 outperforms policies without it, achieving 33% higher success and 1.36x faster execution on mug-in-cupholder, and 13% higher success and 1.27x faster execution on bowl-in-dishrack.