Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learning

TL;DR

This paper presents a risk-aware model-based reinforcement learning method that manages epistemic and aleatoric uncertainties separately, improving safety and performance in uncertain control environments.

Contribution

It introduces a simple approach combining probabilistic safety constraints with uncertainty separation in ensemble neural networks for better risk management.

Findings

Separation of uncertainties improves safety in control tasks.

The method outperforms traditional approaches in safety-critical environments.

Effective balancing of optimism and pessimism enhances learning stability.

Abstract

We introduce a simple but effective method for managing risk in model-based reinforcement learning with trajectory sampling that involves probabilistic safety constraints and balancing of optimism in the face of epistemic uncertainty and pessimism in the face of aleatoric uncertainty of an ensemble of stochastic neural networks.Various experiments indicate that the separation of uncertainties is essential to performing well with data-driven MPC approaches in uncertain and safety-critical control environments.