Deep Neural Network Uncertainty Quantification for LArTPC Reconstruction

TL;DR

This paper evaluates uncertainty quantification methods for deep learning in LArTPC physics analysis, demonstrating improved accuracy and OOD detection, with ensembling methods providing well-calibrated predictions.

Contribution

It provides a comprehensive assessment of UQ methods on complex LArTPC datasets, highlighting ensembling as the most effective approach for calibration and accuracy.

Findings

UQ methods improve mistake rejection and OOD detection.

Ensembling achieves better calibration and higher accuracy.

UQ metrics effectively evaluate uncertainty quality.

Abstract

We evaluate uncertainty quantification (UQ) methods for deep learning applied to liquid argon time projection chamber (LArTPC) physics analysis tasks. As deep learning applications enter widespread usage among physics data analysis, neural networks with reliable estimates of prediction uncertainty and robust performance against overconfidence and out-of-distribution (OOD) samples are critical for their full deployment in analyzing experimental data. While numerous UQ methods have been tested on simple datasets, performance evaluations for more complex tasks and datasets are scarce. We assess the application of selected deep learning UQ methods on the task of particle classification using the PiLArNet [1] monte carlo 3D LArTPC point cloud dataset. We observe that UQ methods not only allow for better rejection of prediction mistakes and OOD detection, but also generally achieve higher overall accuracy across different task settings. We assess the precision of uncertainty quantification using different evaluation metrics, such as distributional separation of prediction entropy across correctly and incorrectly identified samples, receiver operating characteristic curves (ROCs), and expected calibration error from observed empirical accuracy. We conclude that ensembling methods can obtain well calibrated classification probabilities and generally perform better than other existing methods in deep learning UQ literature.