Improving prediction models by incorporating external data with weights based on similarity

TL;DR

This paper introduces a novel weighting method that combines data set and observation similarities to improve prediction models using external data, especially in small clinical datasets with heterogeneity.

Contribution

The paper proposes a new approach that integrates data set and observation weights based on similarity, enhancing prediction accuracy in multi-center clinical studies.

Findings

Improved prediction performance when external data sets are similar.

Method effectively quantifies similarity between data sets.

Application to radiotherapy dose prediction demonstrates practical utility.

Abstract

In clinical settings, we often face the challenge of building prediction models based on small observational data sets. For example, such a data set might be from a medical center in a multi-center study. Differences between centers might be large, thus requiring specific models based on the data set from the target center. Still, we want to borrow information from the external centers, to deal with small sample sizes. There are approaches that either assign weights to each external data set or each external observation. To incorporate information on differences between data sets and observations, we propose an approach that combines both into weights that can be incorporated into a likelihood for fitting regression models. Specifically, we suggest weights at the data set level that incorporate information on how well the models that provide the observation weights distinguish between data sets. Technically, this takes the form of inverse probability weighting. We explore different scenarios where covariates and outcomes differ among data sets, informing our simulation design for method evaluation. The concept of effective sample size is used for understanding the effectiveness of our subgroup modeling approach. We demonstrate our approach through a clinical application, predicting applied radiotherapy doses for cancer patients. Generally, the proposed approach provides improved prediction performance when external data sets are similar. We thus provide a method for quantifying similarity of external data sets to the target data set and use this similarity to include external observations for improving performance in a target data set prediction modeling task with small data.