GAN Q-learning

TL;DR

This paper introduces GAN Q-learning, a novel distributional reinforcement learning method utilizing GANs, demonstrating its effectiveness in simple environments and OpenAI Gym, offering a flexible deep learning-based alternative.

Contribution

The paper proposes GAN Q-learning, integrating GANs into distributional RL, and analyzes its performance, providing a new approach for complex MDPs with deep learning.

Findings

Effective in simple tabular environments

Performs well in OpenAI Gym tasks

Offers a flexible deep learning alternative

Abstract

Distributional reinforcement learning (distributional RL) has seen empirical success in complex Markov Decision Processes (MDPs) in the setting of nonlinear function approximation. However, there are many different ways in which one can leverage the distributional approach to reinforcement learning. In this paper, we propose GAN Q-learning, a novel distributional RL method based on generative adversarial networks (GANs) and analyze its performance in simple tabular environments, as well as OpenAI Gym. We empirically show that our algorithm leverages the flexibility and blackbox approach of deep learning models while providing a viable alternative to traditional methods.