Gaze-based dual resolution deep imitation learning for high-precision dexterous robot manipulation

TL;DR

This paper introduces a gaze-inspired dual resolution deep imitation learning approach for high-precision robot manipulation, effectively combining peripheral and foveated vision to enhance accuracy and efficiency in tasks like needle threading.

Contribution

It presents a novel gaze-based dual resolution visuomotor control system that mimics human vision, enabling precise and efficient manipulation with a general-purpose robot.

Findings

Successfully performed needle threading task using the proposed method

Improved computational efficiency over traditional methods

Demonstrated generalization to different manipulation tasks

Abstract

A high-precision manipulation task, such as needle threading, is challenging. Physiological studies have proposed connecting low-resolution peripheral vision and fast movement to transport the hand into the vicinity of an object, and using high-resolution foveated vision to achieve the accurate homing of the hand to the object. The results of this study demonstrate that a deep imitation learning based method, inspired by the gaze-based dual resolution visuomotor control system in humans, can solve the needle threading task. First, we recorded the gaze movements of a human operator who was teleoperating a robot. Then, we used only a high-resolution image around the gaze to precisely control the thread position when it was close to the target. We used a low-resolution peripheral image to reach the vicinity of the target. The experimental results obtained in this study demonstrate that the proposed method enables precise manipulation tasks using a general-purpose robot manipulator and improves computational efficiency. Data from this and related works are available at: https://sites.google.com/view/multi-task-fine.