Reliability comparison of vessel trajectory prediction models via Probability of Detection

TL;DR

This paper evaluates deep learning vessel trajectory prediction models by analyzing their reliability across different traffic scenarios using probability of detection, providing insights into their strengths and limitations for safer navigation.

Contribution

It introduces a probability of detection-based reliability assessment for VTP models, addressing traffic complexity and offering a more comprehensive evaluation than traditional error metrics.

Findings

Models show varying reliability depending on traffic complexity

Reliability decreases as prediction horizon lengthens

Some models maintain higher safety guarantees in complex scenarios

Abstract

This contribution addresses vessel trajectory prediction (VTP), focusing on the evaluation of different deep learning-based approaches. The objective is to assess model performance in diverse traffic complexities and compare the reliability of the approaches. While previous VTP models overlook the specific traffic situation complexity and lack reliability assessments, this research uses a probability of detection analysis to quantify model reliability in varying traffic scenarios, thus going beyond common error distribution analyses. All models are evaluated on test samples categorized according to their traffic situation during the prediction horizon, with performance metrics and reliability estimates obtained for each category. The results of this comprehensive evaluation provide a deeper understanding of the strengths and weaknesses of the different prediction approaches, along with their reliability in terms of the prediction horizon lengths for which safe forecasts can be guaranteed. These findings can inform the development of more reliable vessel trajectory prediction approaches, enhancing safety and efficiency in future inland waterways navigation.