What Patients Really Ask: Exploring the Effect of False Assumptions in Patient Information Seeking

TL;DR

This paper investigates how large language models handle real-world patient questions, highlighting challenges with false assumptions and dangerous intentions in medical inquiries sourced from Google’s People Also Ask feature.

Contribution

It introduces a dataset of real patient questions with false assumptions and analyzes LLMs' difficulties in identifying incorrect assumptions in everyday medical questions.

Findings

LLMs struggle to detect false assumptions in real patient questions.

A significant portion of questions contain dangerous or incorrect assumptions.

Question history influences the emergence of corrupted questions.

Abstract

Patients are increasingly using large language models (LLMs) to seek answers to their healthcare-related questions. However, benchmarking efforts in LLMs for question answering often focus on medical exam questions, which differ significantly in style and content from the questions patients actually raise in real life. To bridge this gap, we sourced data from Google's People Also Ask feature by querying the top 200 prescribed medications in the United States, curating a dataset of medical questions people commonly ask. A considerable portion of the collected questions contains incorrect assumptions and dangerous intentions. We demonstrate that the emergence of these corrupted questions is not uniformly random and depends heavily on the degree of incorrectness in the history of questions that led to their appearance. Current LLMs that perform strongly on other benchmarks struggle to identify incorrect assumptions in everyday questions.