Notes on the Behavior of MC Dropout

TL;DR

This paper investigates the behavior of Monte-Carlo dropout in deep neural networks, highlighting its properties and considerations for effective uncertainty estimation.

Contribution

It offers a new perspective on MC Dropout's behavior, providing insights into its properties for better uncertainty estimation in neural networks.

Findings

MC Dropout's uncertainty estimates vary with architecture and training.

Certain properties of MC Dropout influence its effectiveness for uncertainty quantification.

The study provides guidelines for using MC Dropout more reliably.

Abstract

Among the various options to estimate uncertainty in deep neural networks, Monte-Carlo dropout is widely popular for its simplicity and effectiveness. However the quality of the uncertainty estimated through this method varies and choices in architecture design and in training procedures have to be carefully considered and tested to obtain satisfactory results. In this paper we present a study offering a different point of view on the behavior of Monte-Carlo dropout, which enables us to observe a few interesting properties of the technique to keep in mind when considering its use for uncertainty estimation.