Improving Clinical Decision-Making in Treating Airway Diseases With an Expert System Built Upon the Free AI Tool Google NotebookLM

TL;DR

This paper explores using Google's free AI tool, NotebookLM, to create a decision-making system for managing airway diseases in clinical settings.

Contribution

The novel contribution is building and evaluating a medical decision aid using a free AI tool without requiring coding skills.

Findings

Most specialists rated AI responses above average, with good reliability on completeness.

The system improved medical literacy and showed potential to save physicians' consultation time.

It is customizable, cost-efficient, and accessible for clinicians without coding experience.

Abstract

We used the free artificial intelligence (AI) tool Google NotebookLM, powered by the large language model Gemini 2.0, to construct a medical decision-making aid for diagnosing and managing airway diseases and subsequently evaluated its functionality and performance in a clinical workflow. After feeding this tool with relevant published clinical guidelines for these diseases, we evaluated the feasibility of the system regarding its behavior, ability, and potential, and we created simulated cases and used the system to solve associated medical problems. The test and simulation questions were designed by a pulmonologist, and the appropriateness (focusing on accuracy and completeness) of AI responses was judged by 3 pulmonologists independently. The system was then deployed in an emergency department setting, where it was tested by medical staff (n=20) to assess how it affected the process of clinical consultation. Test opinions were collected through a questionnaire. Most (56/84, 67%) of the specialists’ ratings regarding AI responses were above average. The interrater reliability was moderate for accuracy (intraclass correlation coefficient=0.612; P<.001) and good on completeness (intraclass correlation coefficient=0.773; P<.001). When deployed in an emergency department (ED) setting, this system could respond with reasonable answers, enhance the literacy of personnel about these diseases. The potential to save the time spent in consultation did not reach statistical significance (Kolmogorov-Smirnov [K-S] D=0.223, P=.24) across all participants, but it indicated a favorable outcome when we analyzed only physicians’ responses. We concluded that this system is customizable, cost efficient, and accessible to clinicians and allied health care professionals without any computer coding experience in treating airway diseases. It provides convincing guideline-based recommendations, increases the staff’s medical literacy, and potentially saves physicians’ time spent on consultation. This system warrants further evaluation in other medical disciplines and health care environments.

Figures