Safety challenges of AI in medicine in the era of large language models

TL;DR

This paper reviews safety challenges posed by large language models in medicine, highlighting risks during data handling, training, and clinical use, and discusses strategies to build trust and ensure safe integration into healthcare.

Contribution

It provides a comprehensive analysis of AI safety issues specific to LLMs in medicine, proposing approaches to mitigate risks and foster trust in clinical applications.

Findings

Identifies LLM-specific safety challenges in data, training, and application.

Discusses inherent and additional safety concerns of AI in healthcare.

Highlights importance of safety for trust and adoption of AI in medicine.

Abstract

Recent advancements in artificial intelligence (AI), particularly in large language models (LLMs), have unlocked significant potential to enhance the quality and efficiency of medical care. By introducing a novel way to interact with AI and data through natural language, LLMs offer new opportunities for medical practitioners, patients, and researchers. However, as AI and LLMs become more powerful and especially achieve superhuman performance in some medical tasks, public concerns over their safety have intensified. These concerns about AI safety have emerged as the most significant obstacles to the adoption of AI in medicine. In response, this review examines emerging risks in AI utilization during the LLM era. First, we explore LLM-specific safety challenges from functional and communication perspectives, addressing issues across data collection, model training, and real-world application. We then consider inherent safety problems shared by all AI systems, along with additional complications introduced by LLMs. Last, we discussed how safety issues of using AI in clinical practice and healthcare system operation would undermine trust among patient, clinicians and the public, and how to build confidence in these systems. By emphasizing the development of safe AI, we believe these technologies can be more rapidly and reliably integrated into everyday medical practice to benefit both patients and clinicians.