Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning

TL;DR

This paper introduces an entropy-aware model initialization method for deep reinforcement learning, which improves exploration efficiency by addressing initial entropy biases, leading to better performance and stability.

Contribution

It proposes a novel entropy-aware initialization strategy that effectively enhances exploration and reduces learning failures in deep reinforcement learning.

Findings

Reduces learning failures significantly.

Improves exploration, stability, and learning speed.

Enhances overall performance in experiments.

Abstract

Encouraging exploration is a critical issue in deep reinforcement learning. We investigate the effect of initial entropy that significantly influences the exploration, especially at the earlier stage. Our main observations are as follows: 1) low initial entropy increases the probability of learning failure, and 2) this initial entropy is biased towards a low value that inhibits exploration. Inspired by the investigations, we devise entropy-aware model initialization, a simple yet powerful learning strategy for effective exploration. We show that the devised learning strategy significantly reduces learning failures and enhances performance, stability, and learning speed through experiments.