Toward Synergic Learning for Autonomous Manipulation of Deformable Tissues via Surgical Robots: An Approximate Q-Learning Approach

TL;DR

This paper introduces a synergic learning algorithm using approximate Q-learning for robotic manipulation of deformable tissues, demonstrating successful simulation results without prior tissue or camera knowledge.

Contribution

It presents a novel synergic learning approach combining human knowledge and approximate Q-learning for tissue manipulation in surgical robotics.

Findings

Successful simulation of tissue manipulation in four scenarios

Effective learning without prior tissue dynamics knowledge

Simple feature selection enables optimal policy learning

Abstract

In this paper, we present a synergic learning algorithm to address the task of indirect manipulation of an unknown deformable tissue. Tissue manipulation is a common yet challenging task in various surgical interventions, which makes it a good candidate for robotic automation. We propose using a linear approximate Q-learning method in which human knowledge contributes to selecting useful yet simple features of tissue manipulation while the algorithm learns to take optimal actions and accomplish the task. The algorithm is implemented and evaluated on a simulation using the OpenCV and CHAI3D libraries. Successful simulation results for four different configurations which are based on realistic tissue manipulation scenarios are presented. Results indicate that with a careful selection of relatively simple and intuitive features, the developed Q-learning algorithm can successfully learn an optimal policy without any prior knowledge of tissue dynamics or camera intrinsic/extrinsic calibration parameters.