Deep Dynamics Models for Learning Dexterous Manipulation

TL;DR

This paper introduces PDDM, a method combining deep dynamics models and online planning to enable dexterous manipulation skills on a 24-DoF robotic hand using minimal real-world data.

Contribution

It presents a scalable online planning approach with improved dynamics models for complex, contact-rich manipulation tasks in real-world settings.

Findings

Achieved manipulation of multiple objects with a 24-DoF hand in 4 hours of real data.

Demonstrated effective learning of contact-rich skills with minimal data.

Showed that online planning with deep models outperforms prior methods.

Abstract

Dexterous multi-fingered hands can provide robots with the ability to flexibly perform a wide range of manipulation skills. However, many of the more complex behaviors are also notoriously difficult to control: Performing in-hand object manipulation, executing finger gaits to move objects, and exhibiting precise fine motor skills such as writing, all require finely balancing contact forces, breaking and reestablishing contacts repeatedly, and maintaining control of unactuated objects. Learning-based techniques provide the appealing possibility of acquiring these skills directly from data, but current learning approaches either require large amounts of data and produce task-specific policies, or they have not yet been shown to scale up to more complex and realistic tasks requiring fine motor skills. In this work, we demonstrate that our method of online planning with deep dynamics models (PDDM) addresses both of these limitations; we show that improvements in learned dynamics models, together with improvements in online model-predictive control, can indeed enable efficient and effective learning of flexible contact-rich dexterous manipulation skills -- and that too, on a 24-DoF anthropomorphic hand in the real world, using just 4 hours of purely real-world data to learn to simultaneously coordinate multiple free-floating objects. Videos can be found at https://sites.google.com/view/pddm/