Control Risk for Potential Misuse of Artificial Intelligence in Science

TL;DR

This paper discusses the potential risks of AI misuse in scientific research, highlights real-world examples, and proposes a system called SciGuard and a benchmark SciMT-Safety to mitigate and assess these risks, advocating for responsible AI use.

Contribution

It introduces SciGuard, a system to control AI misuse in science, and SciMT-Safety, a benchmark for assessing AI safety, emphasizing the importance of responsible AI development in scientific domains.

Findings

SciGuard reduces harmful impact without sacrificing performance.

Real-world misuse examples demonstrate the need for risk management.

The proposed benchmark effectively assesses AI safety in scientific applications.

Abstract

The expanding application of Artificial Intelligence (AI) in scientific fields presents unprecedented opportunities for discovery and innovation. However, this growth is not without risks. AI models in science, if misused, can amplify risks like creation of harmful substances, or circumvention of established regulations. In this study, we aim to raise awareness of the dangers of AI misuse in science, and call for responsible AI development and use in this domain. We first itemize the risks posed by AI in scientific contexts, then demonstrate the risks by highlighting real-world examples of misuse in chemical science. These instances underscore the need for effective risk management strategies. In response, we propose a system called SciGuard to control misuse risks for AI models in science. We also propose a red-teaming benchmark SciMT-Safety to assess the safety of different systems. Our proposed SciGuard shows the least harmful impact in the assessment without compromising performance in benign tests. Finally, we highlight the need for a multidisciplinary and collaborative effort to ensure the safe and ethical use of AI models in science. We hope that our study can spark productive discussions on using AI ethically in science among researchers, practitioners, policymakers, and the public, to maximize benefits and minimize the risks of misuse.