Limitation of Acyclic Oriented Graphs Matching as Cell Tracking Accuracy Measure when Evaluating Mitosis

TL;DR

This paper investigates the limitations of using acyclic oriented graphs matching (AOGM) as an evaluation metric for cell tracking, especially in the context of mitosis detection, revealing its shortcomings through experiments.

Contribution

The study highlights the inadequacy of AOGM for evaluating mitosis events in cell tracking, proposing a critical assessment of current evaluation metrics.

Findings

AOGM does not reliably evaluate mitosis events.

Experiments show discrepancies between AOGM and other metrics.

Limitations are demonstrated on both simulated and real data.

Abstract

Multi-object tracking (MOT) in computer vision and cell tracking in biomedical image analysis are two similar research fields, whose common aim is to achieve instance level object detection/segmentation and associate such objects across different video frames. However, one major difference between these two tasks is that cell tracking also aim to detect mitosis (cell division), which is typically not considered in MOT tasks. Therefore, the acyclic oriented graphs matching (AOGM) has been used as de facto standard evaluation metrics for cell tracking, rather than directly using the evaluation metrics in computer vision, such as multiple object tracking accuracy (MOTA), ID Switches (IDS), ID F1 Score (IDF1) etc. However, based on our experiments, we realized that AOGM did not always function as expected for mitosis events. In this paper, we exhibit the limitations of evaluating mitosis with AOGM using both simulated and real cell tracking data.