Evaluating the Cybersecurity Risk of Real World, Machine Learning Production Systems

TL;DR

This paper presents a comprehensive threat analysis framework for ML production systems, introducing a novel risk scoring system and an extension to attack graph tools to help security practitioners evaluate cyber risks.

Contribution

It develops a systematic methodology for assessing ML system security risks, including a new severity scoring system and an attack graph extension for practical risk analysis.

Findings

Identified key assets and threats in ML production environments.

Proposed a severity scoring system for AML attacks using AHP.

Extended attack graph tools to include ML-specific cyberattacks.

Abstract

Although cyberattacks on machine learning (ML) production systems can be harmful, today, security practitioners are ill equipped, lacking methodologies and tactical tools that would allow them to analyze the security risks of their ML-based systems. In this paper, we performed a comprehensive threat analysis of ML production systems. In this analysis, we follow the ontology presented by NIST for evaluating enterprise network security risk and apply it to ML-based production systems. Specifically, we (1) enumerate the assets of a typical ML production system, (2) describe the threat model (i.e., potential adversaries, their capabilities, and their main goal), (3) identify the various threats to ML systems, and (4) review a large number of attacks, demonstrated in previous studies, which can realize these threats. In addition, to quantify the risk of adversarial machine learning (AML) threat, we introduce a novel scoring system, which assign a severity score to different AML attacks. The proposed scoring system utilizes the analytic hierarchy process (AHP) for ranking, with the assistance of security experts, various attributes of the attacks. Finally, we developed an extension to the MulVAL attack graph generation and analysis framework to incorporate cyberattacks on ML production systems. Using the extension, security practitioners can apply attack graph analysis methods in environments that include ML components; thus, providing security practitioners with a methodological and practical tool for evaluating the impact and quantifying the risk of a cyberattack targeting an ML production system.