Black-Box Anomaly Attribution
Tsuyoshi Id\'e, Naoki Abe

TL;DR
This paper introduces a likelihood-based anomaly attribution method called 'likelihood compensation' for black-box models, addressing the challenge of explaining deviations without access to training data.
Contribution
It proposes a novel likelihood compensation framework for anomaly attribution in black-box models, highlighting limitations of existing explanation methods and validating the approach with real-world data.
Findings
Likelihood compensation effectively identifies responsible input variables for anomalies.
Existing explanation methods are deviation-agnostic and less suitable for anomaly explanation.
The approach is validated on public datasets and a real-world energy prediction case study.
Abstract
When the prediction of a black-box machine learning model deviates from the true observation, what can be said about the reason behind that deviation? This is a fundamental and ubiquitous question that the end user in a business or industrial AI application often asks. The deviation may be due to a sub-optimal black-box model, or it may be simply because the sample in question is an outlier. In either case, one would ideally wish to obtain some form of attribution score -- a value indicative of the extent to which an input variable is responsible for the anomaly. In the present paper we address this task of ``anomaly attribution,'' particularly in the setting in which the model is black-box and the training data are not available. Specifically, we propose a novel likelihood-based attribution framework we call the ``likelihood compensation (LC),'' in which the responsibility score is…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Explainable Artificial Intelligence (XAI) · Machine Learning in Materials Science
