ISL: A novel approach for deep exploration

TL;DR

The paper introduces ISL, a new deep exploration algorithm that combines regularization with RL objectives, deriving learning and exploration strategies simultaneously, and demonstrates superior performance on challenging benchmarks.

Contribution

We propose the ISL algorithm, a novel deep exploration method that jointly derives learning and exploration strategies from a well-posed optimization problem.

Findings

State-of-the-art performance on deep exploration benchmarks

Efficient deep exploration through regularized RL

Unified derivation of learning and exploration strategies

Abstract

In this article we explore an alternative approach to address deep exploration and we introduce the ISL algorithm, which is efficient at performing deep exploration. Similarly to maximum entropy RL, we derive the algorithm by augmenting the traditional RL objective with a novel regularization term. A distinctive feature of our approach is that, as opposed to other works that tackle the problem of deep exploration, in our derivation both the learning equations and the exploration-exploitation strategy are derived in tandem as the solution to a well-posed optimization problem whose minimization leads to the optimal value function. Empirically we show that our method exhibits state of the art performance on a range of challenging deep-exploration benchmarks.