Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification

TL;DR

Torch-Uncertainty is a comprehensive PyTorch framework that simplifies the integration and benchmarking of uncertainty quantification methods in deep neural networks for various tasks.

Contribution

It provides a unified, easy-to-use platform for evaluating and applying multiple UQ techniques across different deep learning applications.

Findings

Benchmarking results across multiple tasks and methods.

Demonstration of the framework's ease of use and extensibility.

Improved understanding of UQ method performance in practice.

Abstract

Deep Neural Networks (DNNs) have demonstrated remarkable performance across various domains, including computer vision and natural language processing. However, they often struggle to accurately quantify the uncertainty of their predictions, limiting their broader adoption in critical real-world applications. Uncertainty Quantification (UQ) for Deep Learning seeks to address this challenge by providing methods to improve the reliability of uncertainty estimates. Although numerous techniques have been proposed, a unified tool offering a seamless workflow to evaluate and integrate these methods remains lacking. To bridge this gap, we introduce Torch-Uncertainty, a PyTorch and Lightning-based framework designed to streamline DNN training and evaluation with UQ techniques and metrics. In this paper, we outline the foundational principles of our library and present comprehensive experimental results that benchmark a diverse set of UQ methods across classification, segmentation, and regression tasks. Our library is available at https://github.com/ENSTA-U2IS-AI/Torch-Uncertainty