Interpretable Local Tree Surrogate Policies

TL;DR

This paper introduces a method to create interpretable policy trees as surrogates for complex neural network policies, enhancing transparency and predictability in high-dimensional decision-making tasks.

Contribution

The paper presents a novel approach to build human-interpretable policy trees that approximate neural network policies, enabling better understanding and trust.

Findings

Policy trees accurately mimic neural network policies.

The approach improves interpretability without significant loss of performance.

Demonstrated effectiveness on simulated tasks.

Abstract

High-dimensional policies, such as those represented by neural networks, cannot be reasonably interpreted by humans. This lack of interpretability reduces the trust users have in policy behavior, limiting their use to low-impact tasks such as video games. Unfortunately, many methods rely on neural network representations for effective learning. In this work, we propose a method to build predictable policy trees as surrogates for policies such as neural networks. The policy trees are easily human interpretable and provide quantitative predictions of future behavior. We demonstrate the performance of this approach on several simulated tasks.