RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts

TL;DR

RICoTA is a Korean dataset of 609 user prompts designed to test large language models' resilience against jailbreak attempts and in-the-wild interactions, aiding in safer chatbot design.

Contribution

The paper introduces RICoTA, a novel dataset capturing real-world jailbreak and testing prompts from Korean users, to evaluate and improve LLM safety measures.

Findings

RICoTA enables effective evaluation of jailbreak resistance.

User prompts reveal common testing strategies against LLMs.

Dataset supports development of safer conversational AI systems.

Abstract

User interactions with conversational agents (CAs) evolve in the era of heavily guardrailed large language models (LLMs). As users push beyond programmed boundaries to explore and build relationships with these systems, there is a growing concern regarding the potential for unauthorized access or manipulation, commonly referred to as "jailbreaking." Moreover, with CAs that possess highly human-like qualities, users show a tendency toward initiating intimate sexual interactions or attempting to tame their chatbots. To capture and reflect these in-the-wild interactions into chatbot designs, we propose RICoTA, a Korean red teaming dataset that consists of 609 prompts challenging LLMs with in-the-wild user-made dialogues capturing jailbreak attempts. We utilize user-chatbot conversations that were self-posted on a Korean Reddit-like community, containing specific testing and gaming intentions with a social chatbot. With these prompts, we aim to evaluate LLMs' ability to identify the type of conversation and users' testing purposes to derive chatbot design implications for mitigating jailbreaking risks. Our dataset will be made publicly available via GitHub.