SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks

TL;DR

SciSafeEval is a comprehensive benchmark designed to evaluate the safety alignment of large language models across diverse scientific tasks and languages, including molecular and genomic data, to promote responsible AI deployment in science.

Contribution

Introduces SciSafeEval, the first extensive benchmark covering multiple scientific languages and domains to assess safety and alignment of LLMs in scientific research.

Findings

LLMs show varying safety performance across scientific domains.

The benchmark reveals gaps in safety guardrails against malicious prompts.

Evaluation in multiple settings highlights strengths and weaknesses of current models.

Abstract

Large language models (LLMs) have a transformative impact on a variety of scientific tasks across disciplines including biology, chemistry, medicine, and physics. However, ensuring the safety alignment of these models in scientific research remains an underexplored area, with existing benchmarks primarily focusing on textual content and overlooking key scientific representations such as molecular, protein, and genomic languages. Moreover, the safety mechanisms of LLMs in scientific tasks are insufficiently studied. To address these limitations, we introduce SciSafeEval, a comprehensive benchmark designed to evaluate the safety alignment of LLMs across a range of scientific tasks. SciSafeEval spans multiple scientific languages-including textual, molecular, protein, and genomic-and covers a wide range of scientific domains. We evaluate LLMs in zero-shot, few-shot and chain-of-thought settings, and introduce a "jailbreak" enhancement feature that challenges LLMs equipped with safety guardrails, rigorously testing their defenses against malicious intention. Our benchmark surpasses existing safety datasets in both scale and scope, providing a robust platform for assessing the safety and performance of LLMs in scientific contexts. This work aims to facilitate the responsible development and deployment of LLMs, promoting alignment with safety and ethical standards in scientific research.