Survivable Hyper-Redundant Robotic Arm with Bayesian Policy Morphing

TL;DR

This paper introduces a Bayesian reinforcement learning framework called Bayesian Policy Morphing that enables robotic arms to adaptively recover from mechanical failures by self-modifying their policies, enhancing survivability.

Contribution

The paper proposes a novel Bayesian Policy Morphing method that extends actor-critic algorithms for adaptive policy recovery in damaged robotic manipulators.

Findings

Robotic arm maintains functionality despite joint damages.

Bayesian Policy Morphing improves learning efficiency with fewer samples.

Successful demonstration on an 8-DOF robotic arm with various joint damages.

Abstract

In this paper we present a Bayesian reinforcement learning framework that allows robotic manipulators to adaptively recover from random mechanical failures autonomously, hence being survivable. To this end, we formulate the framework of Bayesian Policy Morphing (BPM) that enables a robot agent to self-modify its learned policy after the diminution of its maneuvering dimensionality. We build upon existing actor-critic framework, and extend it to perform policy gradient updates as posterior learning, taking past policy updates as prior distributions. We show that policy search, in the direction biased by prior experience, significantly improves learning efficiency in terms of sampling requirements. We demonstrate our results on an 8-DOF robotic arm with our algorithm of BPM, while intentionally disabling random joints with different damage types like unresponsive joints, constant offset errors and angular imprecision. Our results have shown that, even with physical damages, the robotic arm can still successfully maintain its functionality to accurately locate and grasp a given target object.