Visible-Thermal Tiny Object Detection: A Benchmark Dataset and Baselines

TL;DR

This paper introduces RGBT-Tiny, a large-scale, diverse benchmark dataset for visible-thermal small object detection, along with a new evaluation metric, to advance research in multi-modal small object detection and tracking.

Contribution

The paper presents the first large-scale RGBT small object detection dataset with high diversity and annotations, and proposes SAFit, a robust scale-adaptive evaluation measure.

Findings

Extensive evaluation of 23 algorithms on RGBT-Tiny.

SAFit provides more reliable performance assessment for small and large targets.

Benchmark facilitates future research in RGBT small object detection and tracking.

Abstract

Small object detection (SOD) has been a longstanding yet challenging task for decades, with numerous datasets and algorithms being developed. However, they mainly focus on either visible or thermal modality, while visible-thermal (RGBT) bimodality is rarely explored. Although some RGBT datasets have been developed recently, the insufficient quantity, limited category, misaligned images and large target size cannot provide an impartial benchmark to evaluate multi-category visible-thermal small object detection (RGBT SOD) algorithms. In this paper, we build the first large-scale benchmark with high diversity for RGBT SOD (namely RGBT-Tiny), including 115 paired sequences, 93K frames and 1.2M manual annotations. RGBT-Tiny contains abundant targets (7 categories) and high-diversity scenes (8 types that cover different illumination and density variations). Note that, over 81% of targets are smaller than 16x16, and we provide paired bounding box annotations with tracking ID to offer an extremely challenging benchmark with wide-range applications, such as RGBT fusion, detection and tracking. In addition, we propose a scale adaptive fitness (SAFit) measure that exhibits high robustness on both small and large targets. The proposed SAFit can provide reasonable performance evaluation and promote detection performance. Based on the proposed RGBT-Tiny dataset and SAFit measure, extensive evaluations have been conducted, including 23 recent state-of-the-art algorithms that cover four different types (i.e., visible generic detection, visible SOD, thermal SOD and RGBT object detection). Project is available at https://github.com/XinyiYing/RGBT-Tiny.