Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning

TL;DR

AIRS is an adaptive method that dynamically selects intrinsic rewards to improve exploration in deep reinforcement learning, leading to better performance across diverse tasks.

Contribution

The paper introduces AIRS, a novel adaptive intrinsic reward shaping approach that selects shaping functions based on estimated returns, enhancing exploration efficiency.

Findings

AIRS outperforms benchmark schemes on multiple RL tasks.

The intrinsic reward toolkit enables efficient implementation of diverse approaches.

AIRS achieves superior performance with simple architecture.

Abstract

We present AIRS: Automatic Intrinsic Reward Shaping that intelligently and adaptively provides high-quality intrinsic rewards to enhance exploration in reinforcement learning (RL). More specifically, AIRS selects shaping function from a predefined set based on the estimated task return in real-time, providing reliable exploration incentives and alleviating the biased objective problem. Moreover, we develop an intrinsic reward toolkit to provide efficient and reliable implementations of diverse intrinsic reward approaches. We test AIRS on various tasks of MiniGrid, Procgen, and DeepMind Control Suite. Extensive simulation demonstrates that AIRS can outperform the benchmarking schemes and achieve superior performance with simple architecture.