Design of Artificial Intelligence Agents for Games using Deep Reinforcement Learning

TL;DR

This paper evaluates deep reinforcement learning algorithms, including biologically-inspired methods, for training AI agents in game environments, highlighting potential for neuromorphic hardware implementation.

Contribution

It introduces and compares biologically plausible feedback alignment methods with traditional backpropagation in deep reinforcement learning for game AI.

Findings

Biologically-inspired feedback alignment methods perform comparably to backpropagation.

Deep RL algorithms successfully trained agents on classic control and Atari games.

Potential for implementing these algorithms on neuromorphic hardware is demonstrated.

Abstract

In order perform a large variety of tasks and to achieve human-level performance in complex real-world environments, Artificial Intelligence (AI) Agents must be able to learn from their past experiences and gain both knowledge and an accurate representation of their environment from raw sensory inputs. Traditionally, AI agents have suffered from difficulties in using only sensory inputs to obtain a good representation of their environment and then mapping this representation to an efficient control policy. Deep reinforcement learning algorithms have provided a solution to this issue. In this study, the performance of different conventional and novel deep reinforcement learning algorithms was analysed. The proposed method utilises two types of algorithms, one trained with a variant of Q-learning (DQN) and another trained with SARSA learning (DSN) to assess the feasibility of using direct feedback alignment, a novel biologically plausible method for back-propagating the error. These novel agents, alongside two similar agents trained with the conventional backpropagation algorithm, were tested by using the OpenAI Gym toolkit on several classic control theory problems and Atari 2600 video games. The results of this investigation open the way into new, biologically-inspired deep reinforcement learning algorithms, and their implementation on neuromorphic hardware.