Microsimulation Estimates of Decision Uncertainty and Value of Information Are Biased but Consistent

TL;DR

This study compares individual-level microsimulations to cohort models in economic evaluations, revealing biases in decision uncertainty and value of information estimates that diminish with more microsimulations, impacting computational strategies.

Contribution

It provides analytical expressions and simulations showing that iSTMs produce biased but consistent estimates of decision uncertainty and VOI, highlighting the importance of sufficient microsimulations.

Findings

iSTMs estimates are biased but asymptotically consistent.

Bias in EVPI estimates is larger with negative correlation in uncertainties.

More microsimulations reduce bias in decision uncertainty and VOI estimates.

Abstract

Individual-level state-transition microsimulations (iSTMs) have proliferated for economic evaluations in place of cohort state transition models (cSTMs). Probabilistic economic evaluations quantify decision uncertainty and value of information (VOI). Prior studies show that iSTMs provide unbiased estimates of expected incremental net monetary benefits (EINMB), but statistical properties of their estimates of decision uncertainty and VOI are uncharacterized. We compare such iSTMs-produced estimates to corresponding cSTMs. For a 2-alternative decision and normally distributed incremental costs and benefits, we derive analytical expressions for the probability of being cost-effective and the expected value of perfect information (EVPI) for cSTMs and iSTMs, accounting for correlations in incremental outcomes at the population and individual levels. Numerical simulations illustrate our findings and explore relaxation of normality assumptions or having >2 decision alternatives. iSTM estimates of decision uncertainty and VOI are biased but asymptotically consistent (i.e., bias->0 as number of microsimulated individuals->infinity). Decision uncertainty depends on one tail of the INMB distribution (e.g., P(INMB<=0)) which depends on estimated variance (larger with iSTMs given first-order noise). While iSTMs overestimate EVPI, their direction of bias for the probability of being cost-effective is ambiguous. Bias is larger when uncertainties in incremental costs and effects are negatively correlated. While more samples at the population uncertainty level are interchangeable with more microsimulations for estimating EINMB, minimizing iSTM bias in estimating decision uncertainty and VOI depends on sufficient microsimulations. Analysts should account for this when allocating their computational budgets and, at minimum, characterize such bias in their reported results.