# Evaluation of Information Generated by ChatGPT on Preventing Peripheral Venous Catheter‐Related Infections

## TL;DR

This study evaluates how well ChatGPT models provide accurate and complete information on preventing infections from peripheral intravenous catheters.

## Contribution

The study is the first to systematically assess ChatGPT models' performance on peripheral venous catheter infection prevention guidelines.

## Key findings

- ChatGPT models scored low on accuracy and completeness for infection prevention information.
- GPT-3.5 and GPT-4o scored 5/10, while GPT-4 scored 3/10 on the information form.
- Models struggled with hand hygiene and aseptic technique questions but performed better on catheter replacement and dressing.

## Abstract

To evaluate the accuracy and completeness of information generated by ChatGPT models in preventing peripheral intravenous catheter‐related infections.

Peripheral intravenous catheters are vital for administering medication and fluids but often cause complications like life‐threatening infections. These issues increase healthcare costs and patient discomfort. Nurses are crucial in managing these catheters, yet studies show they often lack knowledge and adherence to best practices.

This descriptive study utilised a 10‐question Information Form for Preventing Peripheral Venous Catheter‐Related Infections. By presenting the form to ChatGPT models (GPT‐3.5, GPT‐4, and GPT‐4o), it was requested that each multiple‐choice question be answered and a brief explanation provided as to why that option was correct. Responses were evaluated for correctness (0–10), accuracy (1–5), and completeness (1–3) using Likert scales.

GPT‐3.5 and GPT‐4o each scored 5 out of 10 on the Information Form for Preventing Peripheral Venous Catheter‐Related Infections, while GPT‐4 scored 3 out of 10. All models correctly answered questions on catheter replacement, selection, and dressing regimens (time of replacement) but struggled with hand hygiene, aseptic technique, and catheter dressing regimens (type of dressing). Accuracy scores averaged 3.9 for GPT‐3.5, 3.5 for GPT‐4, and 3.4 for GPT‐4o. Completeness scores averaged 1.8 for GPT‐3.5, 1.6 for GPT‐4, and 1.8 for GPT‐4o. There were no significant differences in accuracy and completeness scores between the models (p > 0.05).

The findings highlight that although ChatGPT models can provide supportive information, their limitations in accuracy and completeness may pose risks for patient safety, particularly in critical domains such as aseptic technique and hand hygiene. It was emphasised that expert supervision is needed in the use of artificial intelligence tools to provide information in healthcare services.

Future improvements in artificial intelligence models are necessary to enhance their effectiveness in medical applications. It is crucial to ensure that healthcare professionals are aware of the current limitations of artificial intelligence tools and continue to rely on expert knowledge and supervision in clinical settings.

No patient or public contribution.