On Reinforcement Learning for the Game of 2048

TL;DR

This paper explores reinforcement learning techniques for 2048, proposing optimistic temporal difference learning and developing a high-performing program, while also discussing educational applications of the game.

Contribution

It introduces optimistic temporal difference learning for 2048 and creates a state-of-the-art program, along with exploring advanced techniques and pedagogical uses.

Findings

Achieved an average score of 625377 points in 2048.

Reached a 72% success rate for 32768-tiles.

Proposed techniques show promise for further performance improvements.

Abstract

2048 is a single-player stochastic puzzle game. This intriguing and addictive game has been popular worldwide and has attracted researchers to develop game-playing programs. Due to its simplicity and complexity, 2048 has become an interesting and challenging platform for evaluating the effectiveness of machine learning methods. This dissertation conducts comprehensive research on reinforcement learning and computer game algorithms for 2048. First, this dissertation proposes optimistic temporal difference learning, which significantly improves the quality of learning by employing optimistic initialization to encourage exploration for 2048. Furthermore, based on this approach, a state-of-the-art program for 2048 is developed, which achieves the highest performance among all learning-based programs, namely an average score of 625377 points and a rate of 72% for reaching 32768-tiles. Second, this dissertation investigates several techniques related to 2048, including the n-tuple network ensemble learning, Monte Carlo tree search, and deep reinforcement learning. These techniques are promising for further improving the performance of the current state-of-the-art program. Finally, this dissertation discusses pedagogical applications related to 2048 by proposing course designs and summarizing the teaching experience. The proposed course designs adopt 2048-like games as materials for beginners to learn reinforcement learning and computer game algorithms. The courses have been successfully applied to graduate-level students and received well by student feedback.