An Interactive Tool for Natural Language Processing on Clinical Text

TL;DR

This paper introduces an interactive visualization tool that enables clinicians and researchers to review and correct NLP outputs on clinical text, facilitating model refinement and improving accuracy through user feedback.

Contribution

The paper presents a novel prototype that integrates visualization and feedback mechanisms for NLP models in clinical settings, supporting non-expert users in model revision.

Findings

Users can quickly refine NLP models with minimal ML experience.

The tool's visualizations aid understanding and correction of NLP outputs.

User feedback highlights interface improvements for better workflow support.

Abstract

Natural Language Processing (NLP) systems often make use of machine learning techniques that are unfamiliar to end-users who are interested in analyzing clinical records. Although NLP has been widely used in extracting information from clinical text, current systems generally do not support model revision based on feedback from domain experts. We present a prototype tool that allows end users to visualize and review the outputs of an NLP system that extracts binary variables from clinical text. Our tool combines multiple visualizations to help the users understand these results and make any necessary corrections, thus forming a feedback loop and helping improve the accuracy of the NLP models. We have tested our prototype in a formative think-aloud user study with clinicians and researchers involved in colonoscopy research. Results from semi-structured interviews and a System Usability Scale (SUS) analysis show that the users are able to quickly start refining NLP models, despite having very little or no experience with machine learning. Observations from these sessions suggest revisions to the interface to better support review workflow and interpretation of results.