X-MAP: eXplainable Misclassification Analysis and Profiling for Spam and Phishing Detection

TL;DR

X-MAP is an explainable framework that identifies and profiles misclassifications in spam and phishing detection, significantly improving detection accuracy and interpretability by analyzing semantic patterns behind errors.

Contribution

It introduces X-MAP, combining SHAP and NMF to reveal topic-level semantic patterns behind model failures, enhancing detection and interpretability.

Findings

Misclassified messages have at least twice the divergence of correctly classified ones.

X-MAP achieves up to 0.98 AUROC in detection.

It recovers up to 97% of falsely rejected correct predictions.

Abstract

Misclassifications in spam and phishing detection are very harmful, as false negatives expose users to attacks while false positives degrade trust. Existing uncertainty-based detectors can flag potential errors, but possibly be deceived and offer limited interpretability. This paper presents X-MAP, an eXplainable Misclassification Analysis and Profilling framework that reveals topic-level semantic patterns behind model failures. X-MAP combines SHAP-based feature attributions with non-negative matrix factorization to build interpretable topic profiles for reliably classified spam/phishing and legitimate messages, and measures each message's deviation from these profiles using Jensen-Shannon divergence. Experiments on SMS and phishing datasets show that misclassified messages exhibit at least two times larger divergence than correctly classified ones. As a detector, X-MAP achieves up to 0.98 AUROC and lowers the false-rejection rate at 95% TRR to 0.089 on positive predictions. When used as a repair layer on base detectors, it recovers up to 97% of falsely rejected correct predictions with moderate leakage. These results demonstrate X-MAP's effectiveness and interpretability for improving spam and phishing detection.