Real-Time Counterfactual Explanations For Robotic Systems With Multiple Continuous Outputs

TL;DR

This paper presents a method using linear model trees to generate real-time counterfactual explanations for robotic systems with multiple continuous inputs and outputs, enhancing interpretability and trust.

Contribution

It introduces a novel approach for producing counterfactual explanations in robotic systems with continuous variables, addressing a gap in explainability for complex models.

Findings

Linear model trees can generate counterfactual explanations for robotic systems.

The method handles multiple continuous inputs and outputs effectively.

Infeasibility issues related to physical laws are discussed.

Abstract

Although many machine learning methods, especially from the field of deep learning, have been instrumental in addressing challenges within robotic applications, we cannot take full advantage of such methods before these can provide performance and safety guarantees. The lack of trust that impedes the use of these methods mainly stems from a lack of human understanding of what exactly machine learning models have learned, and how robust their behaviour is. This is the problem the field of explainable artificial intelligence aims to solve. Based on insights from the social sciences, we know that humans prefer contrastive explanations, i.e.\ explanations answering the hypothetical question "what if?". In this paper, we show that linear model trees are capable of producing answers to such questions, so-called counterfactual explanations, for robotic systems, including in the case of multiple, continuous inputs and outputs. We demonstrate the use of this method to produce counterfactual explanations for two robotic applications. Additionally, we explore the issue of infeasibility, which is of particular interest in systems governed by the laws of physics.