A Novel Approach to Curiosity and Explainable Reinforcement Learning via Interpretable Sub-Goals

TL;DR

This paper introduces a curiosity-driven, interpretable reinforcement learning approach that decomposes complex tasks into subgoals using GAN-based curiosity and subgoal networks, improving learning in stochastic environments and enhancing explainability.

Contribution

The paper presents a novel GAN-based curiosity mechanism combined with subgoal generation to improve learning and interpretability in reinforcement learning without manual task decomposition.

Findings

Outperforms state-of-the-art methods on stochastic, procedurally-generated tasks.

Enhances explainability by decomposing tasks into interpretable subgoals.

Robust to environment stochasticity and sparse rewards.

Abstract

Two key challenges within Reinforcement Learning involve improving (a) agent learning within environments with sparse extrinsic rewards and (b) the explainability of agent actions. We describe a curious subgoal focused agent to address both these challenges. We use a novel method for curiosity produced from a Generative Adversarial Network (GAN) based model of environment transitions that is robust to stochastic environment transitions. Additionally, we use a subgoal generating network to guide navigation. The explainability of the agent's behavior is increased by decomposing complex tasks into a sequence of interpretable subgoals that do not require any manual design. We show that this method also enables the agent to solve challenging procedurally-generated tasks that contain stochastic transitions above other state-of-the-art methods.