Risk Management Framework for Machine Learning Security

TL;DR

This paper proposes a comprehensive risk management framework for organizations using machine learning models, aiming to evaluate and mitigate security threats including adversarial attacks and traditional vulnerabilities.

Contribution

It introduces a novel framework with evaluation factors and a method to assess security states, helping organizations identify weak links and missing security measures.

Findings

Framework effectively identifies security weak points

Method quantifies security state based on evaluation factors

Guides organizations in prioritizing security measures

Abstract

Adversarial attacks for machine learning models have become a highly studied topic both in academia and industry. These attacks, along with traditional security threats, can compromise confidentiality, integrity, and availability of organization's assets that are dependent on the usage of machine learning models. While it is not easy to predict the types of new attacks that might be developed over time, it is possible to evaluate the risks connected to using machine learning models and design measures that help in minimizing these risks. In this paper, we outline a novel framework to guide the risk management process for organizations reliant on machine learning models. First, we define sets of evaluation factors (EFs) in the data domain, model domain, and security controls domain. We develop a method that takes the asset and task importance, sets the weights of EFs' contribution to confidentiality, integrity, and availability, and based on implementation scores of EFs, it determines the overall security state in the organization. Based on this information, it is possible to identify weak links in the implemented security measures and find out which measures might be missing completely. We believe our framework can help in addressing the security issues related to usage of machine learning models in organizations and guide them in focusing on the adequate security measures to protect their assets.