TIDE: A General Toolbox for Identifying Object Detection Errors

TL;DR

TIDE is a versatile toolbox that analyzes and quantifies different error types in object detection and segmentation models, providing insights into model weaknesses across datasets without needing detailed system knowledge.

Contribution

We introduce a novel, dataset-agnostic framework that segments detection errors into six types and measures their individual impact on overall performance.

Findings

Error contributions vary significantly across models and datasets.

The framework reveals specific error types that dominate in different scenarios.

TIDE enables targeted improvements by identifying key error sources.

Abstract

We introduce TIDE, a framework and associated toolbox for analyzing the sources of error in object detection and instance segmentation algorithms. Importantly, our framework is applicable across datasets and can be applied directly to output prediction files without required knowledge of the underlying prediction system. Thus, our framework can be used as a drop-in replacement for the standard mAP computation while providing a comprehensive analysis of each model's strengths and weaknesses. We segment errors into six types and, crucially, are the first to introduce a technique for measuring the contribution of each error in a way that isolates its effect on overall performance. We show that such a representation is critical for drawing accurate, comprehensive conclusions through in-depth analysis across 4 datasets and 7 recognition models. Available at https://dbolya.github.io/tide/