Toward Scalable Visual Servoing Using Deep Reinforcement Learning and Optimal Control

TL;DR

This paper introduces a hybrid visual servoing approach combining deep reinforcement learning and optimal control to improve scalability, convergence, and accuracy in robotic manipulation across diverse scenes and real-world conditions.

Contribution

It presents a novel hybrid strategy that enhances scalability and generalization in visual servoing by integrating DRL with optimal control, addressing limitations of traditional and deep learning methods.

Findings

Achieves high convergence rates and low end-positioning errors with a 7-DOF manipulator.

Demonstrates scalability across over 1000 scenes and generalization to unseen datasets.

Shows real-world applicability with domain transfer learning in noisy and occluded environments.

Abstract

Classical pixel-based Visual Servoing (VS) approaches offer high accuracy but suffer from a limited convergence area due to optimization nonlinearity. Modern deep learning-based VS methods overcome traditional vision issues but lack scalability, requiring training on limited scenes. This paper proposes a hybrid VS strategy utilizing Deep Reinforcement Learning (DRL) and optimal control to enhance both convergence area and scalability. The DRL component of our approach separately handles representation and policy learning to enhance scalability, generalizability, learning efficiency and ease domain adaptation. Moreover, the optimal control part ensures high end-point accuracy. Our method showcases remarkable achievements in terms of high convergence rates and minimal end-positioning errors using a 7-DOF manipulator. Importantly, it exhibits scalability across more than 1000 distinct scenes. Furthermore, we demonstrate its capacity for generalization to previously unseen datasets. Lastly, we illustrate the real-world applicability of our approach, highlighting its adaptability through single-shot domain transfer learning in environments with noise and occlusions. Real-robot experiments can be found at \url{https://sites.google.com/view/vsls}.