OpenTraj: Assessing Prediction Complexity in Human Trajectories Datasets

TL;DR

This paper introduces a framework for evaluating the complexity of human trajectory datasets based on predictability, regularity, and context, aiding in more meaningful benchmarking of trajectory prediction algorithms.

Contribution

It proposes a set of indicators to assess dataset complexity in human trajectory prediction, providing a new tool for better benchmarking and comparison of methods.

Findings

Compared common datasets using the proposed indicators

Identified differences in dataset complexity affecting benchmarking

Released source code for dataset complexity assessment

Abstract

Human Trajectory Prediction (HTP) has gained much momentum in the last years and many solutions have been proposed to solve it. Proper benchmarking being a key issue for comparing methods, this paper addresses the question of evaluating how complex is a given dataset with respect to the prediction problem. For assessing a dataset complexity, we define a series of indicators around three concepts: Trajectory predictability; Trajectory regularity; Context complexity. We compare the most common datasets used in HTP in the light of these indicators and discuss what this may imply on benchmarking of HTP algorithms. Our source code is released on Github.