Adversarial Reinforcement Learning: A Duality-Based Approach To Solving Optimal Control Problems

TL;DR

This paper introduces ADRL, an adversarial reinforcement learning method based on duality principles, to effectively solve high-dimensional stochastic control problems with improved performance and tight dual gaps.

Contribution

It presents a novel ADRL algorithm that reformulates stochastic control as a min-max problem using information relaxation duality, enhancing solution robustness.

Findings

ADRL outperforms existing methods in numerical experiments.

ADRL achieves tight dual gaps in high-dimensional control problems.

The approach demonstrates robustness in simulation-based optimization.

Abstract

We propose an adversarial deep reinforcement learning (ADRL) algorithm for high-dimensional stochastic control problems. Inspired by the information relaxation duality, ADRL reformulates the control problem as a min-max optimization between policies and adversarial penalties, enforcing non-anticipativity while preserving optimality. Numerical experiments demonstrate ADRL's superior performance to yield tight dual gaps. Our results highlight the potential of ADRL as a robust computational framework for high-dimensional stochastic control in simulation-based optimization contexts.