Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agents

TL;DR

This paper introduces S-DQN and S-PPO algorithms that significantly improve robustness and clean rewards in smoothed deep reinforcement learning agents, outperforming existing methods and attacks across benchmarks.

Contribution

The study presents novel algorithms S-DQN and S-PPO that enhance robustness and rewards in smoothed DRL, filling a performance gap in current approaches.

Findings

S-DQN and S-PPO outperform existing smoothed agents by over 2x under strong attacks.

The new algorithms achieve higher clean rewards and robustness guarantees.

Smoothed Attack is nearly twice as effective in reducing rewards of smoothed agents.

Abstract

Robustness remains a paramount concern in deep reinforcement learning (DRL), with randomized smoothing emerging as a key technique for enhancing this attribute. However, a notable gap exists in the performance of current smoothed DRL agents, often characterized by significantly low clean rewards and weak robustness. In response to this challenge, our study introduces innovative algorithms aimed at training effective smoothed robust DRL agents. We propose S-DQN and S-PPO, novel approaches that demonstrate remarkable improvements in clean rewards, empirical robustness, and robustness guarantee across standard RL benchmarks. Notably, our S-DQN and S-PPO agents not only significantly outperform existing smoothed agents by an average factor of $2.16\times$ under the strongest attack, but also surpass previous robustly-trained agents by an average factor of $2.13\times$. This represents a significant leap forward in the field. Furthermore, we introduce Smoothed Attack, which is $1.89\times$ more effective in decreasing the rewards of smoothed agents than existing adversarial attacks.