Learning Dexterous Manipulation Policies from Experience and Imitation

TL;DR

This paper presents a method for learning dexterous manipulation policies using local controllers derived from trajectory optimization and imitation, demonstrating effective generalization and robustness in complex manipulation tasks.

Contribution

It introduces a framework combining trajectory optimization, imitation learning, and interpolation methods to develop robust, generalizable dexterous manipulation controllers from limited data.

Findings

Nearest neighbors interpolation outperforms deep learning in generalization.

Neural network policies operate without visual feedback and are time-invariant.

Local controllers can be constructed from small amounts of data.

Abstract

We explore learning-based approaches for feedback control of a dexterous five-finger hand performing non-prehensile manipulation. First, we learn local controllers that are able to perform the task starting at a predefined initial state. These controllers are constructed using trajectory optimization with respect to locally-linear time-varying models learned directly from sensor data. In some cases, we initialize the optimizer with human demonstrations collected via teleoperation in a virtual environment. We demonstrate that such controllers can perform the task robustly, both in simulation and on the physical platform, for a limited range of initial conditions around the trained starting state. We then consider two interpolation methods for generalizing to a wider range of initial conditions: deep learning, and nearest neighbors. We find that nearest neighbors achieve higher performance. Nevertheless, the neural network has its advantages: it uses only tactile and proprioceptive feedback but no visual feedback about the object (i.e. it performs the task blind) and learns a time-invariant policy. In contrast, the nearest neighbors method switches between time-varying local controllers based on the proximity of initial object states sensed via motion capture. While both generalization methods leave room for improvement, our work shows that (i) local trajectory-based controllers for complex non-prehensile manipulation tasks can be constructed from surprisingly small amounts of training data, and (ii) collections of such controllers can be interpolated to form more global controllers. Results are summarized in the supplementary video: https://youtu.be/E0wmO6deqjo