On the Benefits of Inducing Local Lipschitzness for Robust Generative Adversarial Imitation Learning

TL;DR

This paper introduces a regularization technique to induce local Lipschitzness in GAIL's discriminator and generator, significantly enhancing policy robustness against observation noise in robotics tasks.

Contribution

The paper proposes a novel regularization method to enforce local Lipschitzness in GAIL, improving robustness of learned policies against noisy observations.

Findings

Robust policies outperform state-of-the-art GAIL in noisy environments

Training a Lipschitz discriminator induces a Lipschitz generator

Method demonstrates significant robustness improvements in MuJoCo environments

Abstract

We explore methodologies to improve the robustness of generative adversarial imitation learning (GAIL) algorithms to observation noise. Towards this objective, we study the effect of local Lipschitzness of the discriminator and the generator on the robustness of policies learned by GAIL. In many robotics applications, the learned policies by GAIL typically suffer from a degraded performance at test time since the observations from the environment might be corrupted by noise. Hence, robustifying the learned policies against the observation noise is of critical importance. To this end, we propose a regularization method to induce local Lipschitzness in the generator and the discriminator of adversarial imitation learning methods. We show that the modified objective leads to learning significantly more robust policies. Moreover, we demonstrate -- both theoretically and experimentally -- that training a locally Lipschitz discriminator leads to a locally Lipschitz generator, thereby improving the robustness of the resultant policy. We perform extensive experiments on simulated robot locomotion environments from the MuJoCo suite that demonstrate the proposed method learns policies that significantly outperform the state-of-the-art generative adversarial imitation learning algorithm when applied to test scenarios with noise-corrupted observations.