Interpretable reinforcement learning for heat pump control through asymmetric differentiable decision trees

TL;DR

This paper introduces an asymmetric soft differential decision tree method for reinforcement learning, enhancing interpretability and efficiency in home energy management systems while maintaining high performance.

Contribution

It proposes a novel asymmetric soft DDT construction that adaptively expands nodes, improving interpretability and performance over traditional symmetric trees.

Findings

Improved decision transparency in heat pump control.

Enhanced efficiency with adaptive tree expansion.

Comparable or superior performance to existing methods.

Abstract

In recent years, deep reinforcement learning (DRL) algorithms have gained traction in home energy management systems. However, their adoption by energy management companies remains limited due to the black-box nature of DRL, which fails to provide transparent decision-making feedback. To address this, explainable reinforcement learning (XRL) techniques have emerged, aiming to make DRL decisions more transparent. Among these, soft differential decision tree (DDT) distillation provides a promising approach due to the clear decision rules they are based on, which can be efficiently computed. However, achieving high performance often requires deep, and completely full, trees, which reduces interpretability. To overcome this, we propose a novel asymmetric soft DDT construction method. Unlike traditional soft DDTs, our approach adaptively constructs trees by expanding nodes only when necessary. This improves the efficient use of decision nodes, which require a predetermined depth to construct full symmetric trees, enhancing both interpretability and performance. We demonstrate the potential of asymmetric DDTs to provide transparent, efficient, and high-performing decision-making in home energy management systems.