Computing the Expected Value of Sample Information Efficiently: Expertise and Skills Required for Four Model-Based Methods

TL;DR

This paper evaluates four recent methods for efficiently computing the Expected Value of Sample Information (EVSI), providing practical guidance for analysts to choose suitable approaches based on their skills and study design needs.

Contribution

It offers a comparative analysis of four novel EVSI approximation methods, including their requirements, strengths, and limitations, to aid analysts in selecting appropriate techniques.

Findings

All methods require probabilistic sensitivity analysis output.

Some methods need model re-runs for each EVSI estimate.

Different methods excel in handling multiple outcomes and quantifying uncertainty.

Abstract

Objectives: Value of information (VOI) analyses can help policy-makers make informed decisions about whether to conduct and how to design future studies. Historically, a computationally expensive method to compute the Expected Value of Sample Information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, four EVSI approximation methods have made such analyses more feasible and accessible. We provide practical recommendations for analysts computing EVSI by evaluating these novel methods. Methods: Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, analyst's expertise and skills, and software required for four recently developed approximation methods. Information was also collected on the strengths and limitations of each approximation method. Results: All four EVSI methods require a decision-analytic model's probabilistic sensitivity analysis (PSA) output. One of the methods also requires the model to be re-run to obtain new PSA outputs for each EVSI estimation. To compute EVSI, analysts must be familiar with at least one of the following skills: advanced regression modeling, likelihood specification, and Bayesian modeling. All methods have different strengths and limitations, e.g., some methods handle evaluation of study designs with more outcomes more efficiently while others quantify uncertainty in EVSI estimates. All methods are programmed in the statistical language R and two of the methods provide online applications. Conclusion: Our paper helps to inform the choice between four efficient EVSI estimation methods, enabling analysts to assess the methods' strengths and limitations and select the most appropriate EVSI method given their situation and skills.