Threat Modelling and Risk Analysis for Large Language Model (LLM)-Powered Applications

TL;DR

This paper presents a threat modeling framework for LLM-powered applications, identifying key security risks like data poisoning and prompt injection, and proposing mitigation strategies to enhance AI system security.

Contribution

It introduces a combined STRIDE and DREAD framework tailored for LLM security threat assessment and demonstrates its application through a detailed case study.

Findings

Identified key threats such as data poisoning and prompt injection.

Developed a tailored threat model for LLM applications.

Proposed mitigation strategies to improve security.

Abstract

The advent of Large Language Models (LLMs) has revolutionized various applications by providing advanced natural language processing capabilities. However, this innovation introduces new cybersecurity challenges. This paper explores the threat modeling and risk analysis specifically tailored for LLM-powered applications. Focusing on potential attacks like data poisoning, prompt injection, SQL injection, jailbreaking, and compositional injection, we assess their impact on security and propose mitigation strategies. We introduce a framework combining STRIDE and DREAD methodologies for proactive threat identification and risk assessment. Furthermore, we examine the feasibility of an end-to-end threat model through a case study of a custom-built LLM-powered application. This model follows Shostack's Four Question Framework, adjusted for the unique threats LLMs present. Our goal is to propose measures that enhance the security of these powerful AI tools, thwarting attacks, and ensuring the reliability and integrity of LLM-integrated systems.