# Metrics for the Evaluation of localisation Robustness

## TL;DR

This paper introduces new metrics to quantify the robustness of localisation systems in autonomous vehicles, addressing a gap in evaluating their reliability under environmental changes.

## Contribution

It proposes novel metrics for assessing localisation robustness, applicable with or without ground truth, and analyzes various localisation strategies using these metrics.

## Key findings

- Metrics effectively quantify localisation robustness.
- Robustness varies significantly across localisation strategies.
- Evaluation highlights the need for robustness-focused improvements.

## Abstract

Robustness and safety are crucial properties for the real-world application of autonomous vehicles. One of the most critical components of any autonomous system is localisation. During the last 20 years there has been significant progress in this area with the introduction of very efficient algorithms for mapping, localisation and SLAM. Many of these algorithms present impressive demonstrations for a particular domain, but fail to operate reliably with changes to the operating environment. The aspect of robustness has not received enough attention and localisation systems for self-driving vehicle applications are seldom evaluated for their robustness. In this paper we propose novel metrics to effectively quantify localisation robustness with or without an accurate ground truth. The experimental results present a comprehensive analysis of the application of these metrics against a number of well known localisation strategies.