A Q-learning Control Method for a Soft Robotic Arm Utilizing Training Data from a Rough Simulator

TL;DR

This paper introduces a Q-learning control method for a soft robotic arm that leverages pre-trained models from a rough simulator, significantly reducing real-world training data and enhancing control performance.

Contribution

The novel approach applies pre-trained models from simulation to improve Q-learning control of a soft robot, addressing sample efficiency issues.

Findings

Pre-trained models reduce real-world training data needed.

The method improves control accuracy.

The approach accelerates convergence rate.

Abstract

It is challenging to control a soft robot, where reinforcement learning methods have been applied with promising results. However, due to the poor sample efficiency, reinforcement learning methods require a large collection of training data, which limits their applications. In this paper, we propose a Q-learning controller for a physical soft robot, in which pre-trained models using data from a rough simulator are applied to improve the performance of the controller. We implement the method on our soft robot, i.e., Honeycomb Pneumatic Network (HPN) arm. The experiments show that the usage of pre-trained models can not only reduce the amount of the real-world training data, but also greatly improve its accuracy and convergence rate.