mvHOTA: A multi-view higher order tracking accuracy metric to measure spatial and temporal associations in multi-point detection

TL;DR

This paper introduces mvHOTA, a new metric for evaluating multi-point tracking that considers spatial and temporal associations across multiple views, improving upon existing metrics for complex multi-camera scenarios.

Contribution

The paper proposes mvHOTA, a multi-view higher order tracking accuracy metric that assesses spatial and temporal associations in multi-point tracking tasks, filling gaps in existing evaluation methods.

Findings

mvHOTA effectively evaluates multi-view multi-point tracking performance.

The metric provides insights into occlusion handling and spatial-temporal associations.

Application demonstrated on surgical endoscopic point detection dataset.

Abstract

Multi-point tracking is a challenging task that involves detecting points in the scene and tracking them across a sequence of frames. Computing detection-based measures like the F-measure on a frame-by-frame basis is not sufficient to assess the overall performance, as it does not interpret performance in the temporal domain. The main evaluation metric available comes from Multi-object tracking (MOT) methods to benchmark performance on datasets such as KITTI with the recently proposed higher order tracking accuracy (HOTA) metric, which is capable of providing a better description of the performance over metrics such as MOTA, DetA, and IDF1. While the HOTA metric takes into account temporal associations, it does not provide a tailored means to analyse the spatial associations of a dataset in a multi-camera setup. Moreover, there are differences in evaluating the detection task for points when compared to objects (point distances vs. bounding box overlap). Therefore in this work, we propose a multi-view higher order tracking metric (mvHOTA) to determine the accuracy of multi-point (multi-instance and multi-class) tracking methods, while taking into account temporal and spatial associations.mvHOTA can be interpreted as the geometric mean of detection, temporal, and spatial associations, thereby providing equal weighting to each of the factors. We demonstrate the use of this metric to evaluate the tracking performance on an endoscopic point detection dataset from a previously organised surgical data science challenge. Furthermore, we compare with other adjusted MOT metrics for this use-case, discuss the properties of mvHOTA, and show how the proposed multi-view Association and the Occlusion index (OI) facilitate analysis of methods with respect to handling of occlusions. The code is available at https://github.com/Cardio-AI/mvhota.