Learning Pivoting Manipulation with Force and Vision Feedback Using Optimization-based Demonstrations

TL;DR

This paper introduces a hybrid learning framework combining model-based optimization and deep reinforcement learning for efficient, robust pivoting manipulation using force and vision feedback, with successful sim-to-real transfer.

Contribution

It proposes a novel demonstration-guided RL approach leveraging CITO for sample efficiency and a privileged training strategy for sim-to-real transfer without privileged info.

Findings

Successful pivoting tasks in simulation and real-world

Sample-efficient learning with demonstration guidance

Effective sim-to-real transfer using proprioception, vision, and force sensing

Abstract

Non-prehensile manipulation is challenging due to complex contact interactions between objects, the environment, and robots. Model-based approaches can efficiently generate complex trajectories of robots and objects under contact constraints. However, they tend to be sensitive to model inaccuracies and require access to privileged information (e.g., object mass, size, pose), making them less suitable for novel objects. In contrast, learning-based approaches are typically more robust to modeling errors but require large amounts of data. In this paper, we bridge these two approaches to propose a framework for learning closed-loop pivoting manipulation. By leveraging computationally efficient Contact-Implicit Trajectory Optimization (CITO), we design demonstration-guided deep Reinforcement Learning (RL), leading to sample-efficient learning. We also present a sim-to-real transfer approach using a privileged training strategy, enabling the robot to perform pivoting manipulation using only proprioception, vision, and force sensing without access to privileged information. Our method is evaluated on several pivoting tasks, demonstrating that it can successfully perform sim-to-real transfer. The overview of our method and the hardware experiments are shown at https://youtu.be/akjGDgfwLbM?si=QVw6ExoPy2VsU2g6