Learning Hybrid-Control Policies for High-Precision In-Contact Manipulation Under Uncertainty

TL;DR

This paper introduces hybrid position-force control policies learned via reinforcement learning, enabling high-precision in-contact manipulation under uncertainty, with improved success rates and reduced damage in peg-in-hole tasks.

Contribution

It proposes Mode-Aware Training for Contact Handling (MATCH), a novel method for efficiently learning hybrid control policies that explicitly switch between force and position control modes.

Findings

MATCH outperforms pose-only policies with up to 10% higher success rates.

MATCH reduces peg breakage by 5x compared to pose-only policies.

In real-world tests, MATCH succeeds twice as often as pose policies in high noise conditions.

Abstract

Reinforcement learning-based control policies have been frequently demonstrated to be more effective than analytical techniques for many manipulation tasks. Commonly, these methods learn neural control policies that predict end-effector pose changes directly from observed state information. For tasks like inserting delicate connectors which induce force constraints, pose-based policies have limited explicit control over force and rely on carefully tuned low-level controllers to avoid executing damaging actions. In this work, we present hybrid position-force control policies that learn to dynamically select when to use force or position control in each control dimension. To improve learning efficiency of these policies, we introduce Mode-Aware Training for Contact Handling (MATCH) which adjusts policy action probabilities to explicitly mirror the mode selection behavior in hybrid control. We validate MATCH's learned policy effectiveness using fragile peg-in-hole tasks under extreme localization uncertainty. We find MATCH substantially outperforms pose-control policies -- solving these tasks with up to 10% higher success rates and 5x fewer peg breaks than pose-only policies under common types of state estimation error. MATCH also demonstrates data efficiency equal to pose-control policies, despite learning in a larger and more complex action space. In over 1600 sim-to-real experiments, we find MATCH succeeds twice as often as pose policies in high noise settings (33% vs.~68%) and applies ~30% less force on average compared to variable impedance policies on a Franka FR3 in laboratory conditions.