Hybrid Learning- and Model-Based Planning and Control of In-Hand Manipulation

TL;DR

This paper introduces a hierarchical hybrid planning and control framework for in-hand manipulation that combines learning-based grasp planning with model-based trajectory and force control, enabling robust and efficient tool use with multifingered robotic hands.

Contribution

It presents a novel hybrid approach integrating learning and model-based methods for in-hand manipulation, reducing computational costs and increasing robustness.

Findings

Successful in simulation for various tool-use tasks

Achieves robustness to trajectory and task variations

Generates grasps comparable to optimization-based methods

Abstract

This paper presents a hierarchical framework for planning and control of in-hand manipulation of a rigid object involving grasp changes using fully-actuated multifingered robotic hands. While the framework can be applied to the general dexterous manipulation, we focus on a more complex definition of in-hand manipulation, where at the goal pose the hand has to reach a grasp suitable for using the object as a tool. The high level planner determines the object trajectory as well as the grasp changes, i.e. adding, removing, or sliding fingers, to be executed by the low-level controller. While the grasp sequence is planned online by a learning-based policy to adapt to variations, the trajectory planner and the low-level controller for object tracking and contact force control are exclusively model-based to robustly realize the plan. By infusing the knowledge about the physics of the problem and the low-level controller into the grasp planner, it learns to successfully generate grasps similar to those generated by model-based optimization approaches, obviating the high computation cost of online running of such methods to account for variations. By performing experiments in physics simulation for realistic tool use scenarios, we show the success of our method on different tool-use tasks and dexterous hand models. Additionally, we show that this hybrid method offers more robustness to trajectory and task variations compared to a model-based method.