Exploring Variational Deep Q Networks

TL;DR

This paper analyzes and refines Variational Deep Q Networks, introducing a Double Variational Deep Q Network to enhance stability and robustness in exploration for complex environments, with empirical evaluation and discussion.

Contribution

It provides a detailed analysis, a refined implementation, and introduces the Double Variational Deep Q Network for improved stability in inference-based reinforcement learning.

Findings

Double Variational Deep Q Network improves stability

Refined implementation enhances exploration efficiency

Evaluation shows competitive performance

Abstract

This study provides both analysis and a refined, research-ready implementation of Tang and Kucukelbir's Variational Deep Q Network, a novel approach to maximising the efficiency of exploration in complex learning environments using Variational Bayesian Inference. Alongside reference implementations of both Traditional and Double Deep Q Networks, a small novel contribution is presented - the Double Variational Deep Q Network, which incorporates improvements to increase the stability and robustness of inference-based learning. Finally, an evaluation and discussion of the effectiveness of these approaches is discussed in the wider context of Bayesian Deep Learning.