Designing Secure AI-based Systems: a Multi-Vocal Literature Review

TL;DR

This paper presents 16 security guidelines for designing AI-based systems, addressing the lack of specific security guidance and supporting practitioners in developing secure AI applications.

Contribution

It introduces a set of 16 architectural security guidelines derived from a multi-vocal literature review for AI-based systems.

Findings

High coverage of guidelines across AI system components

Guidelines provide actionable security advice

Support for secure AI system development

Abstract

AI-based systems leverage recent advances in the field of AI/ML by combining traditional software systems with AI components. Applications are increasingly being developed in this way. Software engineers can usually rely on a plethora of supporting information on how to use and implement any given technology. For AI-based systems, however, such information is scarce. Specifically, guidance on how to securely design the architecture is not available to the extent as for other systems. We present 16 architectural security guidelines for the design of AI-based systems that were curated via a multi-vocal literature review. The guidelines could support practitioners with actionable advice on the secure development of AI-based systems. Further, we mapped the guidelines to typical components of AI-based systems and observed a high coverage where 6 out of 8 generic components have at least one guideline associated to them.