Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations

TL;DR

This paper introduces a method for learning multiview visuomotor manipulation policies from demonstrations, enabling robots to perform contact-rich tasks from multiple viewpoints, both in simulation and real-world settings.

Contribution

It demonstrates that multiview policies can be learned via imitation learning from diverse viewpoints, improving robustness and generalization in robotic manipulation tasks.

Findings

Multiview policies perform well across various tasks and viewpoints.

Learning from multiview data does not reduce performance on fixed-view tasks.

Multiview policies implicitly learn spatially correlated visual features.

Abstract

Learned visuomotor policies have shown considerable success as an alternative to traditional, hand-crafted frameworks for robotic manipulation. Surprisingly, an extension of these methods to the multiview domain is relatively unexplored. A successful multiview policy could be deployed on a mobile manipulation platform, allowing the robot to complete a task regardless of its view of the scene. In this work, we demonstrate that a multiview policy can be found through imitation learning by collecting data from a variety of viewpoints. We illustrate the general applicability of the method by learning to complete several challenging multi-stage and contact-rich tasks, from numerous viewpoints, both in a simulated environment and on a real mobile manipulation platform. Furthermore, we analyze our policies to determine the benefits of learning from multiview data compared to learning with data collected from a fixed perspective. We show that learning from multiview data results in little, if any, penalty to performance for a fixed-view task compared to learning with an equivalent amount of fixed-view data. Finally, we examine the visual features learned by the multiview and fixed-view policies. Our results indicate that multiview policies implicitly learn to identify spatially correlated features.