Pathfinding in Random Partially Observable Environments with Vision-Informed Deep Reinforcement Learning

TL;DR

This paper demonstrates that a Deep Q-Network (DQN) can effectively perform pathfinding in partially observable environments using visual inputs, outperforming recurrent models like DQN-GRU and DQN-LSTM under similar conditions.

Contribution

It introduces a comparison of DQN, DQN-GRU, and DQN-LSTM for visual pathfinding in partially observable environments, showing DQN's superior performance.

Findings

DQN outperforms recurrent models in pathfinding tasks.

Visual input-based DQN can operate effectively in partial observability.

Recurrent models do not necessarily improve performance in this setting.

Abstract

Deep reinforcement learning is a technique for solving problems in a variety of environments, ranging from Atari video games to stock trading. This method leverages deep neural network models to make decisions based on observations of a given environment with the goal of maximizing a reward function that can incorporate cost and rewards for reaching goals. With the aim of pathfinding, reward conditions can include reaching a specified target area along with costs for movement. In this work, multiple Deep Q-Network (DQN) agents are trained to operate in a partially observable environment with the goal of reaching a target zone in minimal travel time. The agent operates based on a visual representation of its surroundings, and thus has a restricted capability to observe the environment. A comparison between DQN, DQN-GRU, and DQN-LSTM is performed to examine each models capabilities with two different types of input. Through this evaluation, it is been shown that with equivalent training and analogous model architectures, a DQN model is able to outperform its recurrent counterparts.