# Clinical evaluation of MiADE: a natural language processing system for assisting structured diagnosis recording at the point of care

## TL;DR

MiADE is an NLP system that helps doctors record diagnoses more efficiently in electronic health records, showing some improvement in outpatient settings.

## Contribution

MiADE is a novel point-of-care NLP system that assists in structured diagnosis recording and was evaluated for usability and impact in clinical settings.

## Key findings

- MiADE was associated with a 23.7% increase in outpatient diagnoses recorded per encounter.
- Half of the post-implementation survey respondents found MiADE to be 'very' or 'moderately' useful.
- No improvement in inpatient diagnosis recording was observed despite MiADE implementation.

## Abstract

To evaluate the usability, usefulness and impact of a novel point of care natural language processing (NLP) system, Medical information AI Data Extractor (MiADE), to assist structured diagnosis recording in electronic health records.

Mixed methods evaluation of the implementation of MiADE in a major National Health Service hospital, with surveys, interviews and observed outpatient consultations. The number of structured diagnoses recorded per outpatient encounter was compared before and after MiADE, and completeness of inpatient problem lists was evaluated using billing diagnoses as a gold standard.

85 clinicians consented to the study and were provided access to MiADE and 24 used MiADE to receive structured data suggestions during the study period. Baseline survey data and observations showed wide variation in structured data recording despite clinicians considering it to be important. Half of postimplementation survey respondents considered MiADE to be ‘very’ or ‘moderately’ useful. Multilevel quasi-Poisson regression of 12 309 outpatient encounters (accounting for time and clustering by clinician) estimated that the post-MiADE period was associated with 23.7% more diagnoses recorded per encounter. No improvement was seen in the inpatient setting.

Structured recording of key information such as diagnoses using a clinical terminology is essential for safe, efficient patient care, but is currently done incompletely because it is time-consuming for clinicians. MiADE was associated with an increase in outpatient structured diagnosis recording despite low uptake of the tool.

Point of care NLP using MiADE can potentially improve structured data recording, but further development and better clinician engagement are needed to maximise its impact.

ISRCTN58300671.