An Exploration of Deep Learning Methods in Hungry Geese

TL;DR

This paper evaluates various deep reinforcement learning value methods in the stochastic environment of Hungry Geese, highlighting the limitations of Deep Q Networks and suggesting potential improvements and alternative models.

Contribution

It systematically compares Deep Q, Double Q, and Dueling Q-Networks in Hungry Geese, revealing the strengths and weaknesses of each in a complex environment.

Findings

Vanilla Deep Q Network performed best among tested models.

Convergence to optimal policy was challenging due to environment stochasticity.

Deep Q Networks may not be ideal for highly stochastic environments.

Abstract

Hungry Geese is a n-player variation of the popular game snake. This paper looks at state of the art Deep Reinforcement Learning Value Methods. The goal of the paper is to aggregate research of value based methods and apply it as an exercise to other environments. A vanilla Deep Q Network, a Double Q-network and a Dueling Q-Network were all examined and tested with the Hungry Geese environment. The best performing model was the vanilla Deep Q Network due to its simple state representation and smaller network structure. Converging towards an optimal policy was found to be difficult due to random geese initialization and food generation. Therefore we show that Deep Q Networks may not be the appropriate model for such a stochastic environment and lastly we present improvements that can be made along with more suitable models for the environment.