SafeLawBench: Towards Safe Alignment of Large Language Models

TL;DR

This paper introduces SafeLawBench, a comprehensive legal-based safety evaluation benchmark for large language models, revealing current models' safety limitations and proposing a majority voting enhancement.

Contribution

It presents the first legal perspective-based safety benchmark for LLMs, including extensive evaluation and insights into safety performance and reasoning stability.

Findings

Leading models like GPT-4o score below 80.5% accuracy.

Average LLM safety accuracy is 68.8%.

Majority voting improves safety evaluation performance.

Abstract

With the growing prevalence of large language models (LLMs), the safety of LLMs has raised significant concerns. However, there is still a lack of definitive standards for evaluating their safety due to the subjective nature of current safety benchmarks. To address this gap, we conducted the first exploration of LLMs' safety evaluation from a legal perspective by proposing the SafeLawBench benchmark. SafeLawBench categorizes safety risks into three levels based on legal standards, providing a systematic and comprehensive framework for evaluation. It comprises 24,860 multi-choice questions and 1,106 open-domain question-answering (QA) tasks. Our evaluation included 2 closed-source LLMs and 18 open-source LLMs using zero-shot and few-shot prompting, highlighting the safety features of each model. We also evaluated the LLMs' safety-related reasoning stability and refusal behavior. Additionally, we found that a majority voting mechanism can enhance model performance. Notably, even leading SOTA models like Claude-3.5-Sonnet and GPT-4o have not exceeded 80.5% accuracy in multi-choice tasks on SafeLawBench, while the average accuracy of 20 LLMs remains at 68.8\%. We urge the community to prioritize research on the safety of LLMs.