Monte Carlo Stochastic Depth for Uncertainty Estimation in Deep Learning

TL;DR

This paper introduces Monte Carlo Stochastic Depth (MCSD), a theoretically grounded and empirically validated method for uncertainty estimation in deep learning, demonstrating competitive accuracy and calibration improvements over existing stochastic regularizers.

Contribution

It provides the first theoretical connection of MCSD to Bayesian inference and benchmarks its performance against MCD and MCDB on modern detectors and datasets.

Findings

MCSD achieves highly competitive predictive accuracy (mAP).

MCSD slightly improves calibration (ECE) and uncertainty ranking (AUARC).

MCSD is computationally efficient and robust.

Abstract

The deployment of deep neural networks in safety-critical systems necessitates reliable and efficient uncertainty quantification (UQ). A practical and widespread strategy for UQ is repurposing stochastic regularizers as scalable approximate Bayesian inference methods, such as Monte Carlo Dropout (MCD) and MC-DropBlock (MCDB). However, this paradigm remains under-explored for Stochastic Depth (SD), a regularizer integral to the residual-based backbones of most modern architectures. While prior work demonstrated its empirical promise for segmentation, a formal theoretical connection to Bayesian variational inference and a benchmark on complex, multi-task problems like object detection are missing. In this paper, we first provide theoretical insights connecting Monte Carlo Stochastic Depth (MCSD) to principled approximate variational inference. We then present the first comprehensive empirical benchmark of MCSD against MCD and MCDB on state-of-the-art detectors (YOLO, RT-DETR) using the COCO and COCO-O datasets. Our results position MCSD as a robust and computationally efficient method that achieves highly competitive predictive accuracy (mAP), notably yielding slight improvements in calibration (ECE) and uncertainty ranking (AUARC) compared to MCD. We thus establish MCSD as a theoretically-grounded and empirically-validated tool for efficient Bayesian approximation in modern deep learning.