How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics

TL;DR

This paper proposes an axiomatic framework for evaluating risk metrics in robotics, aiming to establish a foundational theory for risk assessment that enhances safety and decision-making under uncertainty.

Contribution

It introduces axioms for rational risk metrics in robotics, characterizes the class of distortion risk metrics, and discusses their application and limitations in robotic decision-making.

Findings

Characterizes risk metrics satisfying proposed axioms.

Identifies pitfalls of common risk metrics in robotics.

Provides guidelines for sequential decision-making risk assessment.

Abstract

Endowing robots with the capability of assessing risk and making risk-aware decisions is widely considered a key step toward ensuring safety for robots operating under uncertainty. But, how should a robot quantify risk? A natural and common approach is to consider the framework whereby costs are assigned to stochastic outcomes - an assignment captured by a cost random variable. Quantifying risk then corresponds to evaluating a risk metric, i.e., a mapping from the cost random variable to a real number. Yet, the question of what constitutes a "good" risk metric has received little attention within the robotics community. The goal of this paper is to explore and partially address this question by advocating axioms that risk metrics in robotics applications should satisfy in order to be employed as rational assessments of risk. We discuss general representation theorems that precisely characterize the class of metrics that satisfy these axioms (referred to as distortion risk metrics), and provide instantiations that can be used in applications. We further discuss pitfalls of commonly used risk metrics in robotics, and discuss additional properties that one must consider in sequential decision making tasks. Our hope is that the ideas presented here will lead to a foundational framework for quantifying risk (and hence safety) in robotics applications.