Enabling MCTS Explainability for Sequential Planning Through Computation Tree Logic

TL;DR

This paper introduces a logic-based framework to explain Monte Carlo tree search (MCTS) in sequential planning, enhancing user understanding and satisfaction in real-world transportation routing applications.

Contribution

It presents a novel computation tree logic-based explainer that translates user requirements into logic, verifies MCTS states, and generates human-readable explanations, improving interpretability.

Findings

Outperforms baseline explanations in user preference surveys

Provides rigorous logic validation of MCTS states

Enhances transparency in transportation routing applications

Abstract

Monte Carlo tree search (MCTS) is one of the most capable online search algorithms for sequential planning tasks, with significant applications in areas such as resource allocation and transit planning. Despite its strong performance in real-world deployment, the inherent complexity of MCTS makes it challenging to understand for users without technical background. This paper considers the use of MCTS in transportation routing services, where the algorithm is integrated to develop optimized route plans. These plans are required to meet a range of constraints and requirements simultaneously, further complicating the task of explaining the algorithm's operation in real-world contexts. To address this critical research gap, we introduce a novel computation tree logic-based explainer for MCTS. Our framework begins by taking user-defined requirements and translating them into rigorous logic specifications through the use of language templates. Then, our explainer incorporates a logic verification and quantitative evaluation module that validates the states and actions traversed by the MCTS algorithm. The outcomes of this analysis are then rendered into human-readable descriptive text using a second set of language templates. The user satisfaction of our approach was assessed through a survey with 82 participants. The results indicated that our explanatory approach significantly outperforms other baselines in user preference.