RMExplorer: A Visual Analytics Approach to Explore the Performance and the Fairness of Disease Risk Models on Population Subgroups

TL;DR

RMExplorer is an interactive visualization tool that helps clinical researchers assess the performance and fairness of disease risk models across diverse patient subgroups, addressing challenges in model generalization and bias detection.

Contribution

We developed RMExplorer, a novel visual analytics system enabling detailed exploration of risk model performance and fairness across subpopulations, facilitating better model evaluation.

Findings

RMExplorer effectively visualizes model performance across subgroups.

The tool reveals biases and performance disparities in risk models.

Case study demonstrates its utility on atrial fibrillation risk models.

Abstract

Disease risk models can identify high-risk patients and help clinicians provide more personalized care. However, risk models developed on one dataset may not generalize across diverse subpopulations of patients in different datasets and may have unexpected performance. It is challenging for clinical researchers to inspect risk models across different subgroups without any tools. Therefore, we developed an interactive visualization system called RMExplorer (Risk Model Explorer) to enable interactive risk model assessment. Specifically, the system allows users to define subgroups of patients by selecting clinical, demographic, or other characteristics, to explore the performance and fairness of risk models on the subgroups, and to understand the feature contributions to risk scores. To demonstrate the usefulness of the tool, we conduct a case study, where we use RMExplorer to explore three atrial fibrillation risk models by applying them to the UK Biobank dataset of 445,329 individuals. RMExplorer can help researchers to evaluate the performance and biases of risk models on subpopulations of interest in their data.