Teaching Uncertainty Quantification in Machine Learning through Use Cases

TL;DR

This paper proposes a short curriculum with practical use cases to teach uncertainty quantification in machine learning, aiming to enhance safety awareness and understanding among students.

Contribution

It introduces a novel curriculum complemented by practical use cases focused on uncertainty in machine learning, promoting safety and deeper understanding.

Findings

Use cases effectively trigger student discussion.

Curriculum covers key uncertainty concepts like Bayesian neural networks.

Aims to motivate community adoption for AI safety.

Abstract

Uncertainty in machine learning is not generally taught as general knowledge in Machine Learning course curricula. In this paper we propose a short curriculum for a course about uncertainty in machine learning, and complement the course with a selection of use cases, aimed to trigger discussion and let students play with the concepts of uncertainty in a programming setting. Our use cases cover the concept of output uncertainty, Bayesian neural networks and weight distributions, sources of uncertainty, and out of distribution detection. We expect that this curriculum and set of use cases motivates the community to adopt these important concepts into courses for safety in AI.