Soft-Robust Actor-Critic Policy-Gradient
Esther Derman, Daniel J. Mankowitz, Timothy A. Mann, Shie Mannor

TL;DR
This paper introduces a Soft-Robust Actor-Critic algorithm that balances robustness to model uncertainty with reduced conservativeness, improving policy learning in uncertain environments.
Contribution
The paper proposes a novel SR-AC algorithm that learns policies considering a distribution over uncertainties, avoiding the overly conservative nature of traditional robust methods.
Findings
SR-AC converges reliably in tested domains.
SR-AC outperforms standard and robust methods in efficiency.
The approach effectively balances robustness and performance.
Abstract
Robust Reinforcement Learning aims to derive optimal behavior that accounts for model uncertainty in dynamical systems. However, previous studies have shown that by considering the worst case scenario, robust policies can be overly conservative. Our soft-robust framework is an attempt to overcome this issue. In this paper, we present a novel Soft-Robust Actor-Critic algorithm (SR-AC). It learns an optimal policy with respect to a distribution over an uncertainty set and stays robust to model uncertainty but avoids the conservativeness of robust strategies. We show the convergence of SR-AC and test the efficiency of our approach on different domains by comparing it against regular learning methods and their robust formulations.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Advanced Memory and Neural Computing · Simulation Techniques and Applications
