Identifying Drivers of Predictive Aleatoric Uncertainty
Pascal Iversen, Simon Witzke, Katharina Baum, Bernhard Y. Renard
TL;DR
This paper introduces a simple, effective method for explaining predictive aleatoric uncertainty in neural networks, enhancing transparency in AI models without complex modifications.
Contribution
The authors propose a straightforward approach to explain aleatoric uncertainty using existing explainers on neural networks with Gaussian outputs, outperforming complex methods.
Findings
Reliable uncertainty explanations demonstrated in synthetic data
Outperforms complex methods in most benchmark settings
Applicable to both tabular and image datasets
Abstract
Explainability and uncertainty quantification are key to trustable artificial intelligence. However, the reasoning behind uncertainty estimates is generally left unexplained. Identifying the drivers of uncertainty complements explanations of point predictions in recognizing model limitations and enhancing transparent decision-making. So far, explanations of uncertainties have been rarely studied. The few exceptions rely on Bayesian neural networks or technically intricate approaches, such as auxiliary generative models, thereby hindering their broad adoption. We propose a straightforward approach to explain predictive aleatoric uncertainties. We estimate uncertainty in regression as predictive variance by adapting a neural network with a Gaussian output distribution. Subsequently, we apply out-of-the-box explainers to the model's variance output. This approach can explain uncertainty…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
- The paper proposes a novel method for explaining predictive aleatoric uncertainty by explaining the variance output in a heteroscedastic regression model. - The paper empirically demonstrates that the proposed method VFA-SHAP identifies noise features driving the model's aleatoric uncertainty.
- A limitation of this method is its computational complexity; can the authors elaborate on the computational cost of their method, relative to existing approaches? - For the real-world dataset evaluation, the authors show the output explanations of VFA. It is hard to understand the performance of the method on this dataset, since the ground-truth explanations are unknown (which I understand is not available) and only the performance of VFA is presented. It would be useful to show the performanc
The idea of understanding reasons behind the aleatoric uncertainty is interesting. The paper picks a specific controllable case and studies it. Multiple evaluation metrics are introduced and both synthetic and real-world datasets are considered for evaluation. In total, the paper compares five different methods, including two baselines.
1. The paper introduction is built around understanding aleatoric uncertainty; however, it is not clear from the paper what "understanding uncertainty" is. In general, one would expect that understanding uncertainty means identifying the sources of it. For the case of heteroscedastic regression, the explanations are built by identifying features that contribute to output uncertainty. More discussion is needed on why this method is effective and scalable. The worry is that the methods can be misl
The method is mathematically straightforward and simple to grasp. Figure 1 is a nice conceptual overview. The methods and metrics are well explained (and therefore replicable), and the provided formulas and definitions are appropriately conceptual for understanding the method. In general, the paper organization is very good. The experimental results, especially on the synthetic data where the noise process is known, are strong compared to the other methods. Results figures are also well-present
My main concern is about originality: how to quantify a contribution that is simply combining existing concepts from explainability and uncertainty quantification, albeit in an appealingly simple and well-presented way. Is good presentation, fairly complete experiments, and mathematical appeal sufficient to call this a major contribution? I’ll say yes because of the performance improvement in identifying noise features and the usability/accessibility of the approach. There may be a few more ex
- The setup of the method (VFA) and experiments are easy to follow. It is fairly clear what the metrics are measuring and that relevance-accuracy, faithfulness and robustness are desirable properties of an explanation. - Applying various explainability methods for VFA gives a nice overview of advisable ways to do VFA. - The language is consistently appropriate and has little to no grammatical/spelling errors. - The results on the tabular data are persuasive and show strongly that VFA-based metho
- The proposed benchmark only concerns regression with tabular data. I suspect the metrics do not extend well to high dimensional problems such as the age regression example. Additionally, the method (VFA) may not extend very well to classification (though formulations exist, they have limitations), which gives a limitation to the scope of the findings. From your results the conclusion of the comparative analysis is limited to regression on tabular data (where faithfulness is specifically even o
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsExplainable Artificial Intelligence (XAI) · Machine Learning in Healthcare · Age of Information Optimization