Robustness of Explainable Artificial Intelligence in Industrial Process Modelling

TL;DR

This paper evaluates the robustness of various XAI methods in industrial process modeling using a simulated Electric Arc Furnace, revealing differences in their ability to accurately reflect true process sensitivities.

Contribution

It introduces a novel scoring methodology to assess XAI methods' correctness against ground-truth sensitivities in a complex industrial simulation.

Findings

XAI methods vary in accuracy of sensitivity prediction

Model correctness correlates with explainability accuracy

SHAP, LIME, ALE, and SG show different robustness levels

Abstract

eXplainable Artificial Intelligence (XAI) aims at providing understandable explanations of black box models. In this paper, we evaluate current XAI methods by scoring them based on ground truth simulations and sensitivity analysis. To this end, we used an Electric Arc Furnace (EAF) model to better understand the limits and robustness characteristics of XAI methods such as SHapley Additive exPlanations (SHAP), Local Interpretable Model-agnostic Explanations (LIME), as well as Averaged Local Effects (ALE) or Smooth Gradients (SG) in a highly topical setting. These XAI methods were applied to various types of black-box models and then scored based on their correctness compared to the ground-truth sensitivity of the data-generating processes using a novel scoring evaluation methodology over a range of simulated additive noise. The resulting evaluation shows that the capability of the Machine Learning (ML) models to capture the process accurately is, indeed, coupled with the correctness of the explainability of the underlying data-generating process. We furthermore show the differences between XAI methods in their ability to correctly predict the true sensitivity of the modeled industrial process.