garak: A Framework for Security Probing Large Language Models

TL;DR

Garak is a comprehensive framework designed to systematically evaluate the security vulnerabilities of large language models through structured probing, aiding in understanding weaknesses and informing deployment policies.

Contribution

The paper introduces garak, a novel framework for holistic and structured security probing of LLMs, addressing the dynamic and context-dependent nature of model vulnerabilities.

Findings

Garak effectively identifies potential security weaknesses in LLMs.

The framework supports context-aware vulnerability assessment.

It facilitates informed discussions on LLM safety and deployment policies.

Abstract

As Large Language Models (LLMs) are deployed and integrated into thousands of applications, the need for scalable evaluation of how models respond to adversarial attacks grows rapidly. However, LLM security is a moving target: models produce unpredictable output, are constantly updated, and the potential adversary is highly diverse: anyone with access to the internet and a decent command of natural language. Further, what constitutes a security weak in one context may not be an issue in a different context; one-fits-all guardrails remain theoretical. In this paper, we argue that it is time to rethink what constitutes ``LLM security'', and pursue a holistic approach to LLM security evaluation, where exploration and discovery of issues are central. To this end, this paper introduces garak (Generative AI Red-teaming and Assessment Kit), a framework which can be used to discover and identify vulnerabilities in a target LLM or dialog system. garak probes an LLM in a structured fashion to discover potential vulnerabilities. The outputs of the framework describe a target model's weaknesses, contribute to an informed discussion of what composes vulnerabilities in unique contexts, and can inform alignment and policy discussions for LLM deployment.