Multi-level Monte Carlo Dropout for Efficient Uncertainty Quantification

TL;DR

This paper introduces a multilevel Monte Carlo framework for uncertainty quantification using dropout, which reduces variance and improves efficiency in predictive modeling, especially for physics-informed neural networks.

Contribution

The paper proposes a novel multilevel Monte Carlo approach that reuses dropout masks across levels, providing unbiased estimates with lower variance and computational cost.

Findings

Confirmed variance reduction and efficiency gains through numerical experiments.

Demonstrated effectiveness on forward and inverse physics-informed neural networks.

Provided explicit bias, variance, and cost analysis for the proposed method.

Abstract

We develop a multilevel Monte Carlo (MLMC) framework for uncertainty quantification with Monte Carlo dropout. Treating dropout masks as a source of epistemic randomness, we define a fidelity hierarchy by the number of stochastic forward passes used to estimate predictive moments. We construct coupled coarse--fine estimators by reusing dropout masks across fidelities, yielding telescoping MLMC estimators for both predictive means and predictive variances that remain unbiased for the corresponding dropout-induced quantities while reducing sampling variance at fixed evaluation budget. We derive explicit bias, variance and effective cost expressions, together with sample-allocation rules across levels. Numerical experiments on forward and inverse PINNs--Uzawa benchmarks confirm the predicted variance rates and demonstrate efficiency gains over single-level MC-dropout at matched cost.