Reinforcement Learning Using Expectation Maximization Based Guided Policy Search for Stochastic Dynamics

TL;DR

This paper introduces a novel reinforcement learning method combining expectation maximization with guided policy search to optimize policies for stochastic dynamical systems, demonstrating improved performance over existing baselines.

Contribution

It extends trajectory optimization to unknown noisy systems using EM, leading to lower-variance, more robust policies with theoretical and empirical validation.

Findings

Learnt policies exhibit reduced noise and variance.

The approach outperforms baseline methods on autonomous system tasks.

Theoretical analysis supports improved policy robustness.

Abstract

Guided policy search algorithms have been proven to work with incredible accuracy for not only controlling a complicated dynamical system, but also learning optimal policies from various unseen instances. One assumes true nature of the states in almost all of the well known policy search and learning algorithms. This paper deals with a trajectory optimization procedure for an unknown dynamical system subject to measurement noise using expectation maximization and extends it to learning (optimal) policies which have less noise because of lower variance in the optimal trajectories. Theoretical and empirical evidence of learnt optimal policies of the new approach is depicted in comparison to some well known baselines which are evaluated on an autonomous system with widely used performance metrics.