Variations on the Reinforcement Learning performance of Blackjack

TL;DR

This paper explores how reinforcement learning agents, specifically q-learning, perform in blackjack under various deck sizes and environment variations, analyzing convergence rates and the impact of card counting strategies.

Contribution

It introduces an analysis of q-learning convergence in blackjack with different deck sizes and examines the effects of environment variations and card counting on game outcomes.

Findings

Q-learning convergence rate varies with deck size.

Card counting can significantly increase the house's bankruptcy risk.

Environment variations influence the effectiveness of basic strategies.

Abstract

Blackjack or "21" is a popular card-based game of chance and skill. The objective of the game is to win by obtaining a hand total higher than the dealer's without exceeding 21. The ideal blackjack strategy will maximize financial return in the long run while avoiding gambler's ruin. The stochastic environment and inherent reward structure of blackjack presents an appealing problem to better understand reinforcement learning agents in the presence of environment variations. Here we consider a q-learning solution for optimal play and investigate the rate of learning convergence of the algorithm as a function of deck size. A blackjack simulator allowing for universal blackjack rules is also implemented to demonstrate the extent to which a card counter perfectly using the basic strategy and hi-lo system can bring the house to bankruptcy and how environment variations impact this outcome. The novelty of our work is to place this conceptual understanding of the impact of deck size in the context of learning agent convergence.