Comparative Analysis of Black-Box and White-Box Machine Learning Model in Phishing Detection

TL;DR

This study compares black-box and white-box machine learning models for phishing detection, evaluating their accuracy and explainability to recommend suitable approaches based on application needs.

Contribution

It provides a systematic analysis of the pros and cons of black-box and white-box models in phishing detection, validated through experiments on public datasets.

Findings

White-box models like EBM offer better explainability.

Both models perform similarly in accuracy and interpretability.

Model choice depends on specific application requirements.

Abstract

Background: Explainability in phishing detection model can support a further solution of phishing attack mitigation by increasing trust and understanding how phishing can be detected. Objective: The aims of this study to determine and best recommendation to apply an approach which has several components with abilities to fulfil the critical needs Methods: A methodology starting with analyzing both black-box and white-box models to get the pros and cons specifically in phishing detection. The conclusion of the analysis will be validated by experiment using a set of well-known algorithms and public phishing datasets. Experimental metrics covers 3 measurements such as predictive accuracy and explainability metrics. Conclusion: Both models are comparable in terms of interpretability and consistency, with room for improvement in diverse datasets. EBM as an example of white-box model is generally better suited for applications requiring explainability and actionable insights. Finally, each model, white-box and black-box model has positive and negative aspects both for performance metric and for explainable metric. It is important to consider the objective of model usage.