Evaluating the relative contribution of data sources in a Bayesian analysis with the application of estimating the size of hard to reach populations

TL;DR

This paper introduces a Bayesian framework using value of information methods to assess the influence of different data sources in estimating hard-to-reach populations, exemplified by estimating injection drug users in Ukraine.

Contribution

It develops computationally feasible value of information techniques for complex Bayesian models and applies them to public health data to evaluate data source contributions.

Findings

Identified key data sources influencing population estimates

Quantified the impact of each data source on model outputs

Provided recommendations for future data collection efforts

Abstract

When using multiple data sources in an analysis, it is important to understand the influence of each data source on the analysis and the consistency of the data sources with each other and the model. We suggest the use of a retrospective value of information framework in order to address such concerns. Value of information methods can be computationally difficult. We illustrate the use of computational methods that allow these methods to be applied even in relatively complicated settings. In illustrating the proposed methods, we focus on an application in estimating the size of hard to reach populations. Specifically, we consider estimating the number of injection drug users in Ukraine by combining all available data sources spanning over half a decade and numerous sub-national areas in the Ukraine. This application is of interest to public health researchers as this hard to reach population that plays a large role in the spread of HIV. We apply a Bayesian hierarchical model and evaluate the contribution of each data source in terms of absolute influence, expected influence, and level of surprise. Finally we apply value of information methods to inform suggestions on future data collection.