Asset-centric Threat Modeling for AI-based Systems

TL;DR

This paper introduces ThreatFinderAI, a novel tool and approach for automating and guiding threat modeling specifically for AI-based systems, addressing a gap in existing security methodologies.

Contribution

It presents ThreatFinderAI, the first comprehensive method and tool for AI-specific threat modeling that supports automation, evidence-based validation, and practical application.

Findings

Participants successfully recreated expert threat models.

The approach effectively identified strategic risks in LLM applications.

Usability was rated positively by users.

Abstract

Threat modeling is a popular method to securely develop systems by achieving awareness of potential areas of future damage caused by adversaries. However, threat modeling for systems relying on Artificial Intelligence is still not well explored. While conventional threat modeling methods and tools did not address AI-related threats, research on this amalgamation still lacks solutions capable of guiding and automating the process, as well as providing evidence that the methods hold up in practice. Consequently, this paper presents ThreatFinderAI, an approach and tool providing guidance and automation to model AI-related assets, threats, countermeasures, and quantify residual risks. To evaluate the practicality of the approach, participants were tasked to recreate a threat model developed by cybersecurity experts of an AI-based healthcare platform. Secondly, the approach was used to identify and discuss strategic risks in an LLM-based application through a case study. Overall, the solution's usability was well-perceived and effectively supports threat identification and risk discussion.