Incorporating Prior Knowledge into Reinforcement Learning for Soft Tissue Manipulation with Autonomous Grasping Point Selection

TL;DR

This paper introduces a deep reinforcement learning framework that incorporates prior knowledge for soft tissue manipulation, enabling obstacle avoidance and handling unknown constraints, with improved training speed and generalization over existing methods.

Contribution

The paper presents a novel RL framework that integrates prior knowledge and a regulator mechanism for effective soft tissue manipulation under unknown constraints.

Findings

Successfully manipulates soft tissue with obstacle avoidance

Accelerates training compared to SAC algorithm

Enhances generalization in soft tissue manipulation tasks

Abstract

Previous soft tissue manipulation studies assumed that the grasping point was known and the target deformation can be achieved. During the operation, the constraints are supposed to be constant, and there is no obstacles around the soft tissue. To go beyond these assumptions, a deep reinforcement learning framework with prior knowledge is proposed for soft tissue manipulation under unknown constraints, such as the force applied by fascia. The prior knowledge is represented through an intuitive manipulation strategy. As an action of the agent, a regulator factor is used to coordinate the intuitive approach and the deliberate network. A reward function is designed to balance the exploration and exploitation for large deformation. Successful simulation results verify that the proposed framework can manipulate the soft tissue while avoiding obstacles and adding new position constraints. Compared with the soft actor-critic (SAC) algorithm, the proposed framework can accelerate the training procedure and improve the generalization.