Generalised agent for solving higher board states of tic tac toe using Reinforcement Learning

TL;DR

This paper introduces a generalized reinforcement learning algorithm for solving higher-dimensional tic tac toe boards, aiming to efficiently determine optimal moves faster than traditional recursive methods like Min-Max.

Contribution

The study proposes a novel approximate, learning-based algorithm that generalizes to larger tic tac toe boards, with minimal code modifications and promising results.

Findings

High win-to-draw ratio achieved with training epochs

Algorithm effectively generalizes to larger board sizes

Potential applicability to other complex board games

Abstract

Tic Tac Toe is amongst the most well-known games. It has already been shown that it is a biased game, giving more chances to win for the first player leaving only a draw or a loss as possibilities for the opponent, assuming both the players play optimally. Thus on average majority of the games played result in a draw. The majority of the latest research on how to solve a tic tac toe board state employs strategies such as Genetic Algorithms, Neural Networks, Co-Evolution, and Evolutionary Programming. But these approaches deal with a trivial board state of 3X3 and very little research has been done for a generalized algorithm to solve 4X4,5X5,6X6 and many higher states. Even though an algorithm exists which is Min-Max but it takes a lot of time in coming up with an ideal move due to its recursive nature of implementation. A Sample has been created on this link \url{https://bk-tic-tac-toe.herokuapp.com/} to prove this fact. This is the main problem that this study is aimed at solving i.e providing a generalized algorithm(Approximate method, Learning-Based) for higher board states of tic tac toe to make precise moves in a short period. Also, the code changes needed to accommodate higher board states will be nominal. The idea is to pose the tic tac toe game as a well-posed learning problem. The study and its results are promising, giving a high win to draw ratio with each epoch of training. This study could also be encouraging for other researchers to apply the same algorithm to other similar board games like Minesweeper, Chess, and GO for finding efficient strategies and comparing the results.