Combining LLMs with Logic-Based Framework to Explain MCTS

TL;DR

This paper introduces a logic-guided LLM framework that enhances interpretability and factual consistency in explaining Monte Carlo Tree Search (MCTS) processes within AI planning.

Contribution

It presents a novel framework combining logic-based methods with LLMs to improve explanation accuracy and handle diverse user queries about MCTS and MDPs.

Findings

Framework achieves high accuracy in explanations

Ensures factual consistency with environmental dynamics

Handles a wide range of user inquiries

Abstract

In response to the lack of trust in Artificial Intelligence (AI) for sequential planning, we design a Computational Tree Logic-guided large language model (LLM)-based natural language explanation framework designed for the Monte Carlo Tree Search (MCTS) algorithm. MCTS is often considered challenging to interpret due to the complexity of its search trees, but our framework is flexible enough to handle a wide range of free-form post-hoc queries and knowledge-based inquiries centered around MCTS and the Markov Decision Process (MDP) of the application domain. By transforming user queries into logic and variable statements, our framework ensures that the evidence obtained from the search tree remains factually consistent with the underlying environmental dynamics and any constraints in the actual stochastic control process. We evaluate the framework rigorously through quantitative assessments, where it demonstrates strong performance in terms of accuracy and factual consistency.