Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learning

TL;DR

This study uses deep reinforcement learning to simulate and analyze navigation strategies in a virtual Morris Water Maze, revealing similarities with biological learning patterns and internal neural representations.

Contribution

It introduces a novel simulation framework for studying navigation strategies with deep RL, including strategy classification, auxiliary tasks, and neural representation analysis.

Findings

Artificial agents develop strategies similar to animals.

Auxiliary tasks improve learning efficiency.

Neural activations resemble biological place and head-direction cells.

Abstract

Navigation is a complex skill with a long history of research in animals and humans. In this work, we simulate the Morris Water Maze in 2D to train deep reinforcement learning agents. We perform automatic classification of navigation strategies, analyze the distribution of strategies used by artificial agents, and compare them with experimental data to show similar learning dynamics as those seen in humans and rodents. We develop environment-specific auxiliary tasks and examine factors affecting their usefulness. We suggest that the most beneficial tasks are potentially more biologically feasible for real agents to use. Lastly, we explore the development of internal representations in the activations of artificial agent neural networks. These representations resemble place cells and head-direction cells found in mouse brains, and their presence has correlation to the navigation strategies that artificial agents employ.