Robust Imitation Learning for Mobile Manipulator Focusing on Task-Related Viewpoints and Regions

TL;DR

This paper introduces a robust imitation learning approach for mobile manipulators that leverages multiple viewpoints and attention mechanisms to handle occlusion and domain shift, significantly improving success rates across tasks.

Contribution

It proposes a novel multi-viewpoint policy with attention mechanisms that enhances robustness to occlusion and domain shift in mobile manipulation tasks.

Findings

Success rate improved by up to 29.3 points.

Robustness to occlusion increased by learning task-related viewpoints.

Domain shift resilience improved by focusing on task-related regions.

Abstract

We study how to generalize the visuomotor policy of a mobile manipulator from the perspective of visual observations. The mobile manipulator is prone to occlusion owing to its own body when only a single viewpoint is employed and a significant domain shift when deployed in diverse situations. However, to the best of the authors' knowledge, no study has been able to solve occlusion and domain shift simultaneously and propose a robust policy. In this paper, we propose a robust imitation learning method for mobile manipulators that focuses on task-related viewpoints and their spatial regions when observing multiple viewpoints. The multiple viewpoint policy includes attention mechanism, which is learned with an augmented dataset, and brings optimal viewpoints and robust visual embedding against occlusion and domain shift. Comparison of our results for different tasks and environments with those of previous studies revealed that our proposed method improves the success rate by up to 29.3 points. We also conduct ablation studies using our proposed method. Learning task-related viewpoints from the multiple viewpoints dataset increases robustness to occlusion than using a uniquely defined viewpoint. Focusing on task-related regions contributes to up to a 33.3-point improvement in the success rate against domain shift.