A Proactive Insider Threat Management Framework Using Explainable Machine Learning

TL;DR

This paper introduces the IT-XML framework that combines explainable machine learning and process models to proactively identify and mitigate insider threats in organizations, enhancing security decision-making.

Contribution

It presents a novel framework integrating CRISP-DM, HMM, and explainability techniques for proactive insider threat management using survey data and pattern recognition.

Findings

Classified organizations at developing security maturity with 97-98% confidence

Achieved 91.7% classification accuracy in threat pattern recognition

Identified audit log access limits as critical controls for insider threat mitigation

Abstract

Over the years, the technological landscape has evolved, reshaping the security posture of organisations and increasing their exposure to cybersecurity threats, many originating from within. Insider threats remain a major challenge, particularly in sectors where cybersecurity infrastructure, expertise, and regulations are still developing. This study proposes the Insider Threat Explainable Machine Learning (IT-XML) framework, which integrates the Cross-Industry Standard Process for Data Mining (CRISP-DM) with Hidden Markov Models (HMM) to enhance proactive insider threat management and decision-making. A quantitative approach is adopted using an online questionnaire to assess employees' knowledge of insider threat patterns, access control, privacy practices, and existing policies across three large data-sensitive organisations. The IT-XML framework provides assessment capabilities through survey-based data, HMM-driven pattern recognition for security maturity classification, and evidence-based recommendations for proactive threat mitigation. The framework classified all organisations at the developing security maturity level with 97-98% confidence and achieved a classification accuracy of 91.7%, identifying audit log access limits as the most critical control. Random Forest analysis highlighted vendor breach notifications (0.081) and regular audit log reviews (0.052) as key determinants of resilience. Explainability methods such as SHAP and LIME improved model transparency and interpretability, demonstrating the framework's potential to strengthen insider threat management practices.