Imitation Learning for Active Neck Motion Enabling Robot Manipulation beyond the Field of View

TL;DR

This paper introduces a system enabling robots to actively move their necks during imitation learning, expanding task scope beyond fixed viewpoints and achieving high success rates in complex, dynamic scenarios.

Contribution

It presents a novel dataset collection method and a new network model that incorporate active neck motion for improved imitation learning in robots.

Findings

Achieved around 90% success rate despite viewpoint distractions.

Effective in scenarios with objects at the periphery or beyond the field of view.

Enhanced dataset collection efficiency and applicability to complex tasks.

Abstract

Most prior research in deep imitation learning has predominantly utilized fixed cameras for image input, which constrains task performance to the predefined field of view. However, enabling a robot to actively maneuver its neck can significantly expand the scope of imitation learning to encompass a wider variety of tasks and expressive actions such as neck gestures. To facilitate imitation learning in robots capable of neck movement while simultaneously performing object manipulation, we propose a teaching system that systematically collects datasets incorporating neck movements while minimizing discomfort caused by dynamic viewpoints during teleoperation. In addition, we present a novel network model for learning manipulation tasks including active neck motion. Experimental results showed that our model can achieve a high success rate of around 90\%, regardless of the distraction from the viewpoint variations by active neck motion. Moreover, the proposed model proved particularly effective in challenging scenarios, such as when objects were situated at the periphery or beyond the standard field of view, where traditional models struggled. The proposed approach contributes to the efficiency of dataset collection and extends the applicability of imitation learning to more complex and dynamic scenarios.