Analysis and Optimization of Deep Counterfactual Value Networks

TL;DR

This paper explores improved encoding methods for DeepStack's neural networks in poker, enhancing accuracy by integrating traditional abstraction techniques and unabstracted approaches to better approximate Nash equilibrium.

Contribution

It introduces novel encoding strategies for deep counterfactual value networks, combining traditional abstraction with unabstracted methods to improve predictive accuracy.

Findings

Unabstracted encoding increases network accuracy

Traditional abstraction techniques are integrated into neural network inputs and outputs

Enhanced encoding methods improve approximation of Nash equilibrium in poker

Abstract

Recently a strong poker-playing algorithm called DeepStack was published, which is able to find an approximate Nash equilibrium during gameplay by using heuristic values of future states predicted by deep neural networks. This paper analyzes new ways of encoding the inputs and outputs of DeepStack's deep counterfactual value networks based on traditional abstraction techniques, as well as an unabstracted encoding, which was able to increase the network's accuracy.