Performance Gains of LLMs With Humans in a World of LLMs Versus Humans

TL;DR

This paper argues for shifting from comparing LLMs to humans towards developing strategies for effective human-LLM collaboration, especially in clinical settings, to ensure safety and leverage LLM advancements.

Contribution

It introduces the concept of symbiotic human-LLM interaction as a safer and more effective approach than traditional comparison methods.

Findings

Highlighting the risks of direct LLM-human comparisons in healthcare.

Proposing strategies for human-LLM collaboration.

Emphasizing the importance of safety in clinical LLM deployment.

Abstract

Currently, a considerable research effort is devoted to comparing LLMs to a group of human experts, where the term "expert" is often ill-defined or variable, at best, in a state of constantly updating LLM releases. Without proper safeguards in place, LLMs will threaten to cause harm to the established structure of safe delivery of patient care which has been carefully developed throughout history to keep the safety of the patient at the forefront. A key driver of LLM innovation is founded on community research efforts which, if continuing to operate under "humans versus LLMs" principles, will expedite this trend. Therefore, research efforts moving forward must focus on effectively characterizing the safe use of LLMs in clinical settings that persist across the rapid development of novel LLM models. In this communication, we demonstrate that rather than comparing LLMs to humans, there is a need to develop strategies enabling efficient work of humans with LLMs in an almost symbiotic manner.