Dual Behavior Regularized Reinforcement Learning

TL;DR

This paper introduces a dual, advantage-based behavior policy using counterfactual regret minimization in reinforcement learning, enhancing flexibility and performance across various environments.

Contribution

It presents a novel dual behavior regularized reinforcement learning algorithm that adapts to multiple contexts and outperforms strong baselines.

Findings

Outperforms baseline models in continuous environments

Demonstrates adaptability to online and varied contexts

Provides insights through ablation studies

Abstract

Reinforcement learning has been shown to perform a range of complex tasks through interaction with an environment or collected leveraging experience. However, many of these approaches presume optimal or near optimal experiences or the presence of a consistent environment. In this work we propose dual, advantage-based behavior policy based on counterfactual regret minimization. We demonstrate the flexibility of this approach and how it can be adapted to online contexts where the environment is available to collect experiences and a variety of other contexts. We demonstrate this new algorithm can outperform several strong baseline models in different contexts based on a range of continuous environments. Additional ablations provide insights into how our dual behavior regularized reinforcement learning approach is designed compared with other plausible modifications and demonstrates its ability to generalize.