Robust Reinforcement Learning under model misspecification

TL;DR

This paper introduces a new framework for robust reinforcement learning that uses trajectory history and POMDP modeling to handle environment differences, along with an adversarial attack method to improve training robustness.

Contribution

It presents a novel approach combining trajectory history, POMDP modeling, and adversarial attacks to enhance robustness in reinforcement learning under model misspecification.

Findings

Framework effectively handles environment transition differences.

Experimental validation in four gym domains shows improved robustness.

Adversarial attack method aids in training robustness.

Abstract

Reinforcement learning has achieved remarkable performance in a wide range of tasks these days. Nevertheless, some unsolved problems limit its applications in real-world control. One of them is model misspecification, a situation where an agent is trained and deployed in environments with different transition dynamics. We propose an novel framework that utilize history trajectory and Partial Observable Markov Decision Process Modeling to deal with this dilemma. Additionally, we put forward an efficient adversarial attack method to assist robust training. Our experiments in four gym domains validate the effectiveness of our framework.