Survivable Robotic Control through Guided Bayesian Policy Search with Deep Reinforcement Learning

TL;DR

This paper introduces Survivable Robotic Learning (SRL), a Bayesian policy gradient method enabling robots to adapt and maintain manipulation skills despite mechanical failures, improving sample efficiency and success rates.

Contribution

The paper presents a novel Bayesian policy gradient approach using Gaussian process priors for robot survivability and adaptation after mechanical failures, enhancing learning efficiency and robustness.

Findings

SRL outperforms DDPG in sample efficiency.

SRL achieves higher success ratios across failure modes.

Method effectively adapts to mechanical loss in hexapod robots.

Abstract

Many robot manipulation skills can be represented with deterministic characteristics and there exist efficient techniques for learning parameterized motor plans for those skills. However, one of the active research challenge still remains to sustain manipulation capabilities in situation of a mechanical failure. Ideally, like biological creatures, a robotic agent should be able to reconfigure its control policy by adapting to dynamic adversaries. In this paper, we propose a method that allows an agent to survive in a situation of mechanical loss, and adaptively learn manipulation with compromised degrees of freedom -- we call our method Survivable Robotic Learning (SRL). Our key idea is to leverage Bayesian policy gradient by encoding knowledge bias in posterior estimation, which in turn alleviates future policy search explorations, in terms of sample efficiency and when compared to random exploration based policy search methods. SRL represents policy priors as Gaussian process, which allows tractable computation of approximate posterior (when true gradient is intractable), by incorporating guided bias as proxy from prior replays. We evaluate our proposed method against off-the-shelf model free learning algorithm (DDPG), testing on a hexapod robot platform which encounters incremental failure emulation, and our experiments show that our method improves largely in terms of sample requirement and quantitative success ratio in all failure modes. A demonstration video of our experiments can be viewed at: https://sites.google.com/view/survivalrl