XDQN: Inherently Interpretable DQN through Mimicking

TL;DR

XDQN introduces an inherently interpretable deep Q-network by mimicking an interpretable policy model, enabling explanations in complex multi-agent congestion management tasks without sacrificing performance.

Contribution

The paper presents XDQN, a novel interpretable DQN variant trained through mimicking, applied successfully to multi-agent congestion problems in air traffic management.

Findings

XDQN achieves comparable performance to DQN.

XDQN provides global and local interpretability.

Effective in multi-agent congestion scenarios.

Abstract

Although deep reinforcement learning (DRL) methods have been successfully applied in challenging tasks, their application in real-world operational settings is challenged by methods' limited ability to provide explanations. Among the paradigms for explainability in DRL is the interpretable box design paradigm, where interpretable models substitute inner constituent models of the DRL method, thus making the DRL method "inherently" interpretable. In this paper we explore this paradigm and we propose XDQN, an explainable variation of DQN, which uses an interpretable policy model trained through mimicking. XDQN is challenged in a complex, real-world operational multi-agent problem, where agents are independent learners solving congestion problems. Specifically, XDQN is evaluated in three MARL scenarios, pertaining to the demand-capacity balancing problem of air traffic management. XDQN achieves performance similar to that of DQN, while its abilities to provide global models' interpretations and interpretations of local decisions are demonstrated.