Towards Safe Control of Continuum Manipulator Using Shielded Multiagent Reinforcement Learning

TL;DR

This paper demonstrates that a shielded multiagent deep Q network can safely and accurately control continuum surgical robots under complex interaction scenarios, improving control performance without explicit modeling.

Contribution

It introduces a shielded MADQN framework for safe, model-free control of continuum robots, addressing nonlinearities and external interactions effectively.

Findings

Achieved submillimeter trajectory tracking accuracy.

Enabled safe control under external loads and collisions.

Validated on both 2-DoF and miniature continuum robots.

Abstract

Continuum robotic manipulators are increasingly adopted in minimal invasive surgery. However, their nonlinear behavior is challenging to model accurately, especially when subject to external interaction, potentially leading to poor control performance. In this letter, we investigate the feasibility of adopting a model-free multiagent reinforcement learning (RL), namely multiagent deep Q network (MADQN), to control a 2-degree of freedom (DoF) cable-driven continuum surgical manipulator. The control of the robot is formulated as a one-DoF, one agent problem in the MADQN framework to improve the learning efficiency. Combined with a shielding scheme that enables dynamic variation of the action set boundary, MADQN leads to efficient and importantly safer control of the robot. Shielded MADQN enabled the robot to perform point and trajectory tracking with submillimeter root mean square errors under external loads, soft obstacles, and rigid collision, which are common interaction scenarios encountered by surgical manipulators. The controller was further proven to be effective in a miniature continuum robot with high structural nonlinearitiy, achieving trajectory tracking with submillimeter accuracy under external payload.