Is the System Message Really Important to Jailbreaks in Large Language Models?

TL;DR

This paper investigates the impact of system message configurations on the susceptibility of large language models to jailbreak prompts, proposing an evolutionary algorithm to enhance system message robustness against malicious prompts.

Contribution

It introduces the System Messages Evolutionary Algorithm (SMEA) to generate more resistant system messages, and provides experimental evidence on how message length and content affect jailbreak resistance.

Findings

Different system messages have varying resistance to jailbreaks.

SMEA can generate robust system messages with minimal length changes.

Transferability of jailbreaks varies across models with different system messages.

Abstract

The rapid evolution of Large Language Models (LLMs) has rendered them indispensable in modern society. While security measures are typically to align LLMs with human values prior to release, recent studies have unveiled a concerning phenomenon named "Jailbreak". This term refers to the unexpected and potentially harmful responses generated by LLMs when prompted with malicious questions. Most existing research focus on generating jailbreak prompts but system message configurations vary significantly in experiments. In this paper, we aim to answer a question: Is the system message really important for jailbreaks in LLMs? We conduct experiments in mainstream LLMs to generate jailbreak prompts with varying system messages: short, long, and none. We discover that different system messages have distinct resistances to jailbreaks. Therefore, we explore the transferability of jailbreaks across LLMs with different system messages. Furthermore, we propose the System Messages Evolutionary Algorithm (SMEA) to generate system messages that are more resistant to jailbreak prompts, even with minor changes. Through SMEA, we get a robust system messages population with little change in the length of system messages. Our research not only bolsters LLMs security but also raises the bar for jailbreaks, fostering advancements in this field of study.