Deep Reinforcement Learning with Hybrid Intrinsic Reward Model

TL;DR

This paper introduces HIRE, a flexible framework for combining multiple intrinsic rewards in reinforcement learning, which improves exploration efficiency and skill acquisition in complex environments.

Contribution

The paper presents HIRE, a novel framework for hybrid intrinsic rewards, systematically analyzing its effectiveness across various benchmarks and settings.

Findings

HIRE significantly improves exploration efficiency.

HIRE enhances diversity and skill acquisition.

HIRE outperforms single intrinsic reward methods.

Abstract

Intrinsic reward shaping has emerged as a prevalent approach to solving hard-exploration and sparse-rewards environments in reinforcement learning (RL). While single intrinsic rewards, such as curiosity-driven or novelty-based methods, have shown effectiveness, they often limit the diversity and efficiency of exploration. Moreover, the potential and principle of combining multiple intrinsic rewards remains insufficiently explored. To address this gap, we introduce HIRE (Hybrid Intrinsic REward), a flexible and elegant framework for creating hybrid intrinsic rewards through deliberate fusion strategies. With HIRE, we conduct a systematic analysis of the application of hybrid intrinsic rewards in both general and unsupervised RL across multiple benchmarks. Extensive experiments demonstrate that HIRE can significantly enhance exploration efficiency and diversity, as well as skill acquisition in complex and dynamic settings.