Unveiling the Decision-Making Process in Reinforcement Learning with Genetic Programming

TL;DR

This paper introduces a genetic programming approach to generate interpretable explanations for trained reinforcement learning agents, improving transparency without significant computational costs.

Contribution

It presents a novel genetic programming framework that produces interpretable programs to explain RL decisions, with an ablation study on domain-specific language enhancements.

Findings

Comparable performance to state-of-the-art methods

Reduced hardware resource requirements

Faster computation times

Abstract

Despite tremendous progress, machine learning and deep learning still suffer from incomprehensible predictions. Incomprehensibility, however, is not an option for the use of (deep) reinforcement learning in the real world, as unpredictable actions can seriously harm the involved individuals. In this work, we propose a genetic programming framework to generate explanations for the decision-making process of already trained agents by imitating them with programs. Programs are interpretable and can be executed to generate explanations of why the agent chooses a particular action. Furthermore, we conduct an ablation study that investigates how extending the domain-specific language by using library learning alters the performance of the method. We compare our results with the previous state of the art for this problem and show that we are comparable in performance but require much less hardware resources and computation time.