Stiffness Copilot: An Impedance Policy for Contact-Rich Teleoperation

TL;DR

This paper introduces Stiffness Copilot, a vision-based policy that adaptively adjusts robot impedance during contact-rich teleoperation, enhancing safety and efficiency through learned stiffness control from visual input.

Contribution

It presents a novel vision-based impedance policy trained in simulation and transferred to real-world teleoperation, enabling adaptive stiffness control without additional sensors.

Findings

Achieved safety comparable to low stiffness control.

Matched efficiency of high stiffness control.

Enabled zero-shot transfer from simulation to real-world images.

Abstract

In teleoperation of contact-rich manipulation tasks, selecting robot impedance is critical but difficult. The robot must be compliant to avoid damaging the environment, but stiff to remain responsive and to apply force when needed. In this paper, we present Stiffness Copilot, a vision-based policy for shared-control teleoperation in which the operator commands robot pose and the policy adjusts robot impedance online. To train Stiffness Copilot, we first infer direction-dependent stiffness matrices in simulation using privileged contact information. We then use these matrices to supervise a lightweight vision policy that predicts robot stiffness from wrist-camera images and transfers zero-shot to real images at runtime. In a human-subject study, Stiffness Copilot achieved safety comparable to using a constant low stiffness while matching the efficiency of using a constant high stiffness.