An Empirical Study on the Security Vulnerabilities of GPTs

TL;DR

This paper empirically investigates security vulnerabilities in GPT-based systems, analyzing attack surfaces, demonstrating potential exploits, and proposing defense mechanisms to enhance their security and responsible use.

Contribution

It provides a comprehensive attack surface analysis and systematic attack suite for GPTs, along with defense strategies, filling a gap in empirical security research on large language models.

Findings

Identification of key security vulnerabilities in GPT components

Demonstration of information leakage and tool misuse attacks

Proposed defense mechanisms to mitigate identified vulnerabilities

Abstract

Equipped with various tools and knowledge, GPTs, one kind of customized AI agents based on OpenAI's large language models, have illustrated great potential in many fields, such as writing, research, and programming. Today, the number of GPTs has reached three millions, with the range of specific expert domains becoming increasingly diverse. However, given the consistent framework shared among these LLM agent applications, systemic security vulnerabilities may exist and remain underexplored. To fill this gap, we present an empirical study on the security vulnerabilities of GPTs. Building upon prior research on LLM security, we first adopt a platform-user perspective to conduct a comprehensive attack surface analysis across different system components. Then, we design a systematic and multidimensional attack suite with the explicit objectives of information leakage and tool misuse based on the attack surface analysis, thereby concretely demonstrating the security vulnerabilities that various components of GPT-based systems face. Finally, we accordingly propose defense mechanisms to address the aforementioned security vulnerabilities. By increasing the awareness of these vulnerabilities and offering critical insights into their implications, this study seeks to facilitate the secure and responsible application of GPTs while contributing to developing robust defense mechanisms that protect users and systems against malicious attacks.