SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks

TL;DR

SkillTree introduces an explainable hierarchical framework that reduces complex control tasks into skill spaces, combining decision trees with reinforcement learning to improve transparency without sacrificing performance.

Contribution

It presents a novel hierarchical approach integrating differentiable decision trees for explainable skill-based reinforcement learning in long-horizon control tasks.

Findings

Achieves performance comparable to neural network-based methods.

Provides skill-level explanations enhancing transparency.

Effective in complex robotic arm control domains.

Abstract

Deep reinforcement learning (DRL) has achieved remarkable success in various research domains. However, its reliance on neural networks results in a lack of transparency, which limits its practical applications. To achieve explainability, decision trees have emerged as a popular and promising alternative to neural networks. Nonetheless, due to their limited expressiveness, traditional decision trees struggle with high-dimensional long-horizon continuous control tasks. In this paper, we proposes SkillTree, a novel framework that reduces complex continuous action spaces into discrete skill spaces. Our hierarchical approach integrates a differentiable decision tree within the high-level policy to generate skill embeddings, which subsequently guide the low-level policy in executing skills. By making skill decisions explainable, we achieve skill-level explainability, enhancing the understanding of the decision-making process in complex tasks. Experimental results demonstrate that our method achieves performance comparable to skill-based neural networks in complex robotic arm control domains. Furthermore, SkillTree offers explanations at the skill level, thereby increasing the transparency of the decision-making process.