Multiscale IoU: A Metric for Evaluation of Salient Object Detection with Fine Structures

TL;DR

This paper introduces Multiscale IoU, a new evaluation metric that better captures fine structural details in salient object detection by combining IoU with fractal dimension across multiple resolutions.

Contribution

It proposes a novel metric, Multiscale IoU, that enhances evaluation sensitivity to fine object boundaries in detection tasks, addressing limitations of traditional IoU.

Findings

Multiscale IoU is more sensitive to fine boundary structures.

MIoU correlates well with boundary detail in synthetic and real datasets.

The new metric encourages development of more precise object detection algorithms.

Abstract

General-purpose object-detection algorithms often dismiss the fine structure of detected objects. This can be traced back to how their proposed regions are evaluated. Our goal is to renegotiate the trade-off between the generality of these algorithms and their coarse detections. In this work, we present a new metric that is a marriage of a popular evaluation metric, namely Intersection over Union (IoU), and a geometrical concept, called fractal dimension. We propose Multiscale IoU (MIoU) which allows comparison between the detected and ground-truth regions at multiple resolution levels. Through several reproducible examples, we show that MIoU is indeed sensitive to the fine boundary structures which are completely overlooked by IoU and f1-score. We further examine the overall reliability of MIoU by comparing its distribution with that of IoU on synthetic and real-world datasets of objects. We intend this work to re-initiate exploration of new evaluation methods for object-detection algorithms.