AI-Enabled Image-Based Hybrid Vision/Force Control of Tendon-Driven Aerial Continuum Manipulators

TL;DR

This paper introduces an AI-enabled hybrid vision/force control framework for tendon-driven aerial continuum manipulators, integrating neural networks and graph-based visual features for robust autonomous interaction.

Contribution

It proposes a novel cascaded control strategy combining sliding mode control and neural networks with graph-based visual features for aerial continuum manipulation.

Findings

Demonstrates rapid online learning of uncertainties without offline training.

Shows improved robustness over traditional rigid-arm methods in simulations and experiments.

Achieves stable, autonomous manipulation with enhanced feature tracking and force regulation.

Abstract

This paper presents an AI-enabled cascaded hybrid vision/force control framework for tendon-driven aerial continuum manipulators based on constant-strain modeling in $SE(3)$ as a coupled system. The proposed controller is designed to enable autonomous, physical interaction with a static environment while stabilizing the image feature error. The developed strategy combines the cascaded fast fixed-time sliding mode control and a radial basis function neural network to cope with the uncertainties in the image acquired by the eye-in-hand monocular camera and the measurements from the force sensing apparatus. This ensures rapid, online learning of the vision- and force-related uncertainties without requiring offline training. Furthermore, the features are extracted via a state-of-the-art graph neural network architecture employed by a visual servoing framework using line features, rather than relying on heuristic geometric line extractors, to concurrently contribute to tracking the desired normal interaction force during contact and regulating the image feature error. A comparative study benchmarks the proposed controller against established rigid-arm aerial manipulation methods, evaluating robustness across diverse scenarios and feature extraction strategies. The simulation and experimental results showcase the effectiveness of the proposed methodology under various initial conditions and demonstrate robust performance in executing manipulation tasks.