On Feature Relevance Uncertainty: A Monte Carlo Dropout Sampling Approach

TL;DR

This paper introduces Monte Carlo Relevance Propagation (MCRP), a method to estimate uncertainty in feature relevance for neural network explanations, enhancing interpretability and understanding of model reasoning.

Contribution

The paper presents MCRP, a novel Monte Carlo-based approach for quantifying uncertainty in feature relevance, addressing a gap in existing explanation techniques.

Findings

MCRP provides uncertainty scores for feature relevance.

The method improves interpretability of neural network decisions.

It enables deeper insights into model perception.

Abstract

Understanding decisions made by neural networks is key for the deployment of intelligent systems in real world applications. However, the opaque decision making process of these systems is a disadvantage where interpretability is essential. Many feature-based explanation techniques have been introduced over the last few years in the field of machine learning to better understand decisions made by neural networks and have become an important component to verify their reasoning capabilities. However, existing methods do not allow statements to be made about the uncertainty regarding a feature's relevance for the prediction. In this paper, we introduce Monte Carlo Relevance Propagation (MCRP) for feature relevance uncertainty estimation. A simple but powerful method based on Monte Carlo estimation of the feature relevance distribution to compute feature relevance uncertainty scores that allow a deeper understanding of a neural network's perception and reasoning.