Minimax Iterative Dynamic Game: Application to Nonlinear Robot Control Tasks

TL;DR

This paper introduces a minimax iterative dynamic game framework to design robust policies for nonlinear robot control tasks, improving resilience against disturbances and uncertainties in high-dimensional decision spaces.

Contribution

It presents a novel minimax iterative dynamic game approach for enhancing policy robustness in complex control scenarios with disturbances.

Findings

The framework successfully improves policy robustness against adversarial disturbances.

It is adaptable for deep and meta-learning policies in high-risk control tasks.

Experimental results on a mecanum-wheeled robot demonstrate effectiveness.

Abstract

Multistage decision policies provide useful control strategies in high-dimensional state spaces, particularly in complex control tasks. However, they exhibit weak performance guarantees in the presence of disturbance, model mismatch, or model uncertainties. This brittleness limits their use in high-risk scenarios. We present how to quantify the sensitivity of such policies in order to inform of their robustness capacity. We also propose a minimax iterative dynamic game framework for designing robust policies in the presence of disturbance/uncertainties. We test the quantification hypothesis on a carefully designed deep neural network policy; we then pose a minimax iterative dynamic game (iDG) framework for improving policy robustness in the presence of adversarial disturbances. We evaluate our iDG framework on a mecanum-wheeled robot, whose goal is to find a ocally robust optimal multistage policy that achieve a given goal-reaching task. The algorithm is simple and adaptable for designing meta-learning/deep policies that are robust against disturbances, model mismatch, or model uncertainties, up to a disturbance bound. Videos of the results are on the author's website, http://ecs.utdallas.edu/~opo140030/iros18/iros2018.html, while the codes for reproducing our experiments are on github, https://github.com/lakehanne/youbot/tree/rilqg. A self-contained environment for reproducing our results is on docker, https://hub.docker.com/r/lakehanne/youbotbuntu14/