Robot Agnostic Visual Servoing considering kinematic constraints enabled by a decoupled network trajectory planner structure

TL;DR

This paper introduces a robot-agnostic visual servoing approach that combines a detection network with a kinematic-aware trajectory planner, enabling safe, stable control across different robotic systems without retraining.

Contribution

The method's decoupled detection and planning components allow transferability to various robots, considering kinematic constraints for improved control stability.

Findings

Achieved <0.5 mm position error and <1° orientation error.

Successfully transferred to a different robotic system.

Demonstrated effectiveness in cluttered and uncluttered environments.

Abstract

We propose a visual servoing method consisting of a detection network and a velocity trajectory planner. First, the detection network estimates the objects position and orientation in the image space. Furthermore, these are normalized and filtered. The direction and orientation is then the input to the trajectory planner, which considers the kinematic constrains of the used robotic system. This allows safe and stable control, since the kinematic boundary values are taken into account in planning. Also, by having direction estimation and velocity planner separated, the learning part of the method does not directly influence the control value. This also enables the transfer of the method to different robotic systems without retraining, therefore being robot agnostic. We evaluate our method on different visual servoing tasks with and without clutter on two different robotic systems. Our method achieved mean absolute position errors of <0.5 mm and orientation errors of <1{\deg}. Additionally, we transferred the method to a new system which differs in robot and camera, emphasizing robot agnostic capability of our method.