UDUC: An Uncertainty-driven Approach for Learning-based Robust Control

TL;DR

This paper introduces UDUC, a novel uncertainty-driven loss function for probabilistic ensemble models that enhances robustness in learning-based control under environmental uncertainties, validated on challenging benchmarks.

Contribution

The paper proposes the UDUC loss function, inspired by contrastive learning, to improve robustness of PE models in control tasks with environment mismatches.

Findings

UDUC improves robustness against environment variations.

Enhanced performance on the RWRL benchmark.

Addresses mode collapse in probabilistic ensemble models.

Abstract

Learning-based techniques have become popular in both model predictive control (MPC) and reinforcement learning (RL). Probabilistic ensemble (PE) models offer a promising approach for modelling system dynamics, showcasing the ability to capture uncertainty and scalability in high-dimensional control scenarios. However, PE models are susceptible to mode collapse, resulting in non-robust control when faced with environments slightly different from the training set. In this paper, we introduce the $\textbf{u}$ncertainty-$\textbf{d}$riven rob$\textbf{u}$st $\textbf{c}$ontrol (UDUC) loss as an alternative objective for training PE models, drawing inspiration from contrastive learning. We analyze the robustness of UDUC loss through the lens of robust optimization and evaluate its performance on the challenging Real-world Reinforcement Learning (RWRL) benchmark, which involves significant environmental mismatches between the training and testing environments.