Double A3C: Deep Reinforcement Learning on OpenAI Gym Games

TL;DR

This paper introduces Double A3C, an improved deep reinforcement learning algorithm combining Double Q-learning and A3C, demonstrating enhanced performance on OpenAI Gym Atari games.

Contribution

The paper proposes a novel Double A3C algorithm that integrates strengths of Double Q-learning and A3C for better game-playing performance.

Findings

Outperforms baseline A3C on Atari games

Reduces overestimation bias in value estimates

Achieves higher scores in benchmark tests

Abstract

Reinforcement Learning (RL) is an area of machine learning figuring out how agents take actions in an unknown environment to maximize its rewards. Unlike classical Markov Decision Process (MDP) in which agent has full knowledge of its state, rewards, and transitional probability, reinforcement learning utilizes exploration and exploitation for the model uncertainty. Under the condition that the model usually has a large state space, a neural network (NN) can be used to correlate its input state to its output actions to maximize the agent's rewards. However, building and training an efficient neural network is challenging. Inspired by Double Q-learning and Asynchronous Advantage Actor-Critic (A3C) algorithm, we will propose and implement an improved version of Double A3C algorithm which utilizing the strength of both algorithms to play OpenAI Gym Atari 2600 games to beat its benchmarks for our project.