Challenges of Testing an Evolving Cancer Registration Support System in Practice

TL;DR

This paper discusses the challenges faced in developing automated testing solutions for an evolving cancer registration system that incorporates machine learning, emphasizing the importance of data quality and privacy in healthcare software.

Contribution

It identifies key challenges in testing complex, evolving healthcare systems with ML components and shares initial solutions to address these issues.

Findings

Testing impacts long-term data quality and decision-making.

Challenges include system complexity, data privacy, and evolving ML models.

Initial solutions involve tailored automated testing approaches.

Abstract

The Cancer Registry of Norway (CRN) is a public body responsible for capturing and curating cancer patient data histories to provide a unified access to research data and statistics for doctors, patients, and policymakers. For this purpose, CRN develops and operates a complex, constantly-evolving, and socio-technical software system. Recently, machine learning (ML) algorithms have been introduced into this system to augment the manual decisions made by humans with automated decision support from learned models. To ensure that the system is correct and robust and cancer patients' data are properly handled and do not violate privacy concerns, automated testing solutions are being developed. In this paper, we share the challenges that we identified when developing automated testing solutions at CRN. Such testing potentially impacts the quality of cancer data for years to come, which is also used by the system's stakeholders to make critical decisions. The challenges identified are not specific to CRN but are also valid in the context of other healthcare registries. We also provide some details on initial solutions that we are investigating to solve the identified challenges.