Imitation Learning-based Direct Visual Servoing using the Large Projection Formulation

TL;DR

This paper presents a novel imitation learning approach for direct visual servoing that integrates deep learning-based perception with a large projection task priority formulation, enabling robots to perform complex tasks in unstructured environments.

Contribution

It introduces a dynamical system-based imitation learning framework for visual servoing that combines deep perception modules with a large projection formulation, enhancing robustness and task complexity.

Findings

Successfully performed complex manipulation tasks with a robotic arm.

Robust feature extraction from raw images using deep learning modules.

Effective integration of imitation learning with visual feedback control.

Abstract

Today robots must be safe, versatile, and user-friendly to operate in unstructured and human-populated environments. Dynamical system-based imitation learning enables robots to perform complex tasks stably and without explicit programming, greatly simplifying their real-world deployment. To exploit the full potential of these systems it is crucial to implement closed loops that use visual feedback. Vision permits to cope with environmental changes, but is complex to handle due to the high dimension of the image space. This study introduces a dynamical system-based imitation learning for direct visual servoing. It leverages off-the-shelf deep learning-based perception modules to extract robust features from the raw input image, and an imitation learning strategy to execute sophisticated robot motions. The learning blocks are integrated using the large projection task priority formulation. As demonstrated through extensive experimental analysis, the proposed method realizes complex tasks with a robotic manipulator.