Learning Preconditions of Hybrid Force-Velocity Controllers for Contact-Rich Manipulation

TL;DR

This paper introduces a method for planning contact-rich manipulation tasks using learned preconditions for hybrid force-velocity controllers, enabling robots to operate effectively in constrained environments with occlusions.

Contribution

It presents a novel framework that relaxes the need for precise models and feedback in HFVCs, using learned preconditions from point clouds to improve task success in contact-rich manipulation.

Findings

Achieved 73.2% task success rate in shelf manipulation tasks.

Outperformed baseline methods without learned preconditions.

Precondition transfer from simulation to real-world without fine-tuning.

Abstract

Robots need to manipulate objects in constrained environments like shelves and cabinets when assisting humans in everyday settings like homes and offices. These constraints make manipulation difficult by reducing grasp accessibility, so robots need to use non-prehensile strategies that leverage object-environment contacts to perform manipulation tasks. To tackle the challenge of planning and controlling contact-rich behaviors in such settings, this work uses Hybrid Force-Velocity Controllers (HFVCs) as the skill representation and plans skill sequences with learned preconditions. While HFVCs naturally enable robust and compliant contact-rich behaviors, solvers that synthesize them have traditionally relied on precise object models and closed-loop feedback on object pose, which are difficult to obtain in constrained environments due to occlusions. We first relax HFVCs' need for precise models and feedback with our HFVC synthesis framework, then learn a point-cloud-based precondition function to classify where HFVC executions will still be successful despite modeling inaccuracies. Finally, we use the learned precondition in a search-based task planner to complete contact-rich manipulation tasks in a shelf domain. Our method achieves a task success rate of $73.2\%$, outperforming the $51.5\%$ achieved by a baseline without the learned precondition. While the precondition function is trained in simulation, it can also transfer to a real-world setup without further fine-tuning. See supplementary materials and videos at https://sites.google.com/view/constrained-manipulation/