Deep Direct Visual Servoing of Tendon-Driven Continuum Robots

TL;DR

This paper introduces a deep learning-based direct visual servoing method for tendon-driven continuum robots, eliminating the need for feature extraction and pose estimation, and demonstrating robustness in various visual conditions.

Contribution

It presents a novel deep learning approach using a modified VGG-16 network for direct visual servoing of continuum robots, bypassing traditional feature-based methods.

Findings

Effective control in normal, shadowed, and occluded scenes

Robust convergence and accuracy demonstrated

Implementation successful in simulation and real robot

Abstract

Vision-based control provides a significant potential for the end-point positioning of continuum robots under physical sensing limitations. Traditional visual servoing requires feature extraction and tracking followed by full or partial pose estimation, limiting the controller's efficiency. We hypothesize that employing deep learning models and implementing direct visual servoing can effectively resolve the issue by eliminating such intermediate steps, enabling control of a continuum robot without requiring an exact system model. This paper presents the control of a single-section tendon-driven continuum robot using a modified VGG-16 deep learning network and an eye-in-hand direct visual servoing approach. The proposed algorithm is first developed in Blender software using only one input image of the target and then implemented on a real robot. The convergence and accuracy of the results in normal, shadowed, and occluded scenes demonstrate the effectiveness and robustness of the proposed controller.