Robust Offline Reinforcement Learning -- Certify the Confidence Interval

TL;DR

This paper introduces a method to certify the robustness of offline reinforcement learning policies using random smoothing, providing theoretical guarantees and validating effectiveness through experiments.

Contribution

It develops a novel algorithm that certifies policy robustness in offline RL with rigorous theoretical analysis, unlike prior empirical approaches.

Findings

Algorithm effectively certifies policy robustness.

The method is computationally efficient.

Experimental results confirm correctness across environments.

Abstract

Currently, reinforcement learning (RL), especially deep RL, has received more and more attention in the research area. However, the security of RL has been an obvious problem due to the attack manners becoming mature. In order to defend against such adversarial attacks, several practical approaches are developed, such as adversarial training, data filtering, etc. However, these methods are mostly based on empirical algorithms and experiments, without rigorous theoretical analysis of the robustness of the algorithms. In this paper, we develop an algorithm to certify the robustness of a given policy offline with random smoothing, which could be proven and conducted as efficiently as ones without random smoothing. Experiments on different environments confirm the correctness of our algorithm.