Structure and Reduction of MCTS for Explainable-AI

TL;DR

This paper presents novel information-theoretic methods to simplify and extract explainable insights from Monte Carlo Tree Search data structures, enhancing AI decision transparency with minimal additional computational cost.

Contribution

It introduces new techniques for reducing and interpreting MCTS data using information theory, facilitating human-understandable explanations of AI decisions.

Findings

Explainability quantities can be computed efficiently during MCTS

Methods enable extraction of decision reasoning from MCTS data

Approaches are applicable to real-world planning problems

Abstract

Complex sequential decision-making planning problems, covering infinite states' space have been shown to be solvable by AlphaZero type of algorithms. Such an approach that trains a neural model while simulating projection of futures with a Monte Carlo Tree Search algorithm were shown to be applicable to real life planning problems. As such, engineers and users interacting with the resulting policy of behavior might benefit from obtaining automated explanations about these planners' decisions offline or online. This paper focuses on the information within the Monte Carlo Tree Search data structure. Given its construction, this information contains much of the reasoning of the sequential decision-making algorithm and is essential for its explainability. We show novel methods using information theoretic tools for the simplification and reduction of the Monte Carlo Tree Search and the extraction of information. Such information can be directly used for the construction of human understandable explanations. We show that basic explainability quantities can be calculated with limited additional computational cost, as an integrated part of the Monte Carlo Tree Search construction process. We focus on the theoretical and algorithmic aspects and provide examples of how the methods presented here can be used in the construction of human understandable explanations.