AnswerCarefully: A Dataset for Improving the Safety of Japanese LLM Output

TL;DR

AnswerCarefully is a Japanese dataset designed to enhance the safety and appropriateness of LLM outputs, reflecting socio-cultural nuances, and serving as a benchmark for safety evaluation across multiple models.

Contribution

The paper introduces a novel Japanese safety dataset, AnswerCarefully, with original culturally contextual data, and demonstrates its effectiveness in improving LLM safety and benchmarking.

Findings

Fine-tuning with the dataset improves safety without reducing utility.

The dataset serves as a benchmark for evaluating Japanese LLM safety.

English translations and annotations facilitate dataset adaptation.

Abstract

In this paper we present AnswerCarefully, a dataset for promoting the safety and appropriateness of Japanese LLM outputs. The dataset consists of 1,800 pairs of questions and reference answers, where the questions require special attention in answering. It covers a wide range of risk categories established in prior English-language datasets, but the data samples are original in that they are manually created to reflect the socio-cultural context of LLM usage in Japan. We show that using this dataset for instruction to fine-tune a Japanese LLM led to improved output safety without compromising the utility of general responses. We also report the results of a safety evaluation of 12 Japanese LLMs using this dataset as a benchmark. Finally, we describe the latest update on the dataset which provides English translations and annotations of the questions, aimed at facilitating the derivation of similar datasets in different languages and regions.