Bridging AI Innovation and Healthcare Needs: Lessons Learned from Incorporating Modern NLP at The BC Cancer Registry

TL;DR

This paper shares practical lessons from implementing NLP models in healthcare, emphasizing problem definition, collaboration, data quality, and iterative development to improve clinical data extraction and support patient care.

Contribution

It provides a comprehensive set of best practices and insights from real-world NLP deployment in a healthcare setting, highlighting interdisciplinary collaboration and pragmatic model choices.

Findings

Effective problem framing based on business goals is crucial.

Iterative development and collaboration enhance NLP deployment success.

Attention to data quality and human-in-the-loop validation improves model reliability.

Abstract

Automating data extraction from clinical documents offers significant potential to improve efficiency in healthcare settings, yet deploying Natural Language Processing (NLP) solutions presents practical challenges. Drawing upon our experience implementing various NLP models for information extraction and classification tasks at the British Columbia Cancer Registry (BCCR), this paper shares key lessons learned throughout the project lifecycle. We emphasize the critical importance of defining problems based on clear business objectives rather than solely technical accuracy, adopting an iterative approach to development, and fostering deep interdisciplinary collaboration and co-design involving domain experts, end-users, and ML specialists from inception. Further insights highlight the need for pragmatic model selection (including hybrid approaches and simpler methods where appropriate), rigorous attention to data quality (representativeness, drift, annotation), robust error mitigation strategies involving human-in-the-loop validation and ongoing audits, and building organizational AI literacy. These practical considerations, generalizable beyond cancer registries, provide guidance for healthcare organizations seeking to successfully implement AI/NLP solutions to enhance data management processes and ultimately improve patient care and public health outcomes.