Methods for the inclusion of real world evidence in network meta-analysis

TL;DR

This paper explores methods to incorporate real-world evidence into network meta-analyses, assessing their impact on uncertainty and effect estimates, with a focus on Bayesian hierarchical and power prior models in a multiple sclerosis case study.

Contribution

It evaluates and compares different statistical methods for integrating RWE into NMAs, highlighting their effects on uncertainty and heterogeneity management.

Findings

Power prior models stabilize effect estimates but increase uncertainty with more RWE.

Hierarchical models effectively handle study heterogeneity but also raise uncertainty.

Inclusion of RWE can significantly influence the certainty of effect estimates.

Abstract

Background: Network Meta-Analysis (NMA) is a key component of submissions to reimbursement agencies world-wide, especially when there is limited direct head-to-head evidence for multiple technologies from randomised controlled trials (RCTs). Many NMAs include only data from RCTs. However, real-world evidence (RWE) is also becoming widely recognised as a valuable source of clinical data. We investigate methods for the inclusion of RWE in NMA and its impact on the uncertainty around the effectiveness estimates. Methods: A range of methods for inclusion of RWE in evidence synthesis, including Bayesian hierarchical and power prior models, were investigated by applying them to an example in relapsing remitting multiple sclerosis. The effect of the inclusion of RWE was investigated by varying the degree of down weighting of this part of evidence by the use of a power prior. Results: Whilst the inclusion of the RWE led to an increase in the level of uncertainty surrounding effect estimates in this example, this depended on the method of inclusion adopted for the RWE. Power prior NMA model resulted in stable effect estimates for fingolimod yet increasing the width of the credible intervals with increasing weight given to RWE data. The hierarchical NMA models were effective in allowing for heterogeneity between study designs; however, this also increased the level of uncertainty. Conclusion: The power prior approach for the inclusion of RWE in NMAs indicates that the degree to which RWE is taken into account can have a significant impact on the overall level of uncertainty. The hierarchical modelling approach further allowed for accommodating differences between study types. Consequently, further work investigating both empirical evidence for biases associated with individual RWE studies and methods of elicitation from experts on the extent of such biases is warranted.