SeqCSIST: Sequential Closely-Spaced Infrared Small Target Unmixing

TL;DR

This paper introduces a new dataset, benchmark, and deep learning method for detecting and localizing closely-spaced infrared small targets in a sequential, multi-frame setting, significantly improving detection accuracy.

Contribution

It presents the first multi-frame approach to CSIST unmixing, along with an open-source dataset, evaluation toolkit, and a novel deformable feature alignment network.

Findings

Proposed method outperforms existing approaches with a 5.3% increase in mAP.

Introduced the first sequential dataset and toolkit for CSIST unmixing.

Demonstrated the effectiveness of temporal deformable feature alignment in target detection.

Abstract

Due to the limitation of the optical lens focal length and the resolution of the infrared detector, distant Closely-Spaced Infrared Small Target (CSIST) groups typically appear as mixing spots in the infrared image. In this paper, we propose a novel task, Sequential CSIST Unmixing, namely detecting all targets in the form of sub-pixel localization from a highly dense CSIST group. However, achieving such precise detection is an extremely difficult challenge. In addition, the lack of high-quality public datasets has also restricted the research progress. To this end, firstly, we contribute an open-source ecosystem, including SeqCSIST, a sequential benchmark dataset, and a toolkit that provides objective evaluation metrics for this special task, along with the implementation of 23 relevant methods. Furthermore, we propose the Deformable Refinement Network (DeRefNet), a model-driven deep learning framework that introduces a Temporal Deformable Feature Alignment (TDFA) module enabling adaptive inter-frame information aggregation. To the best of our knowledge, this work is the first endeavor to address the CSIST Unmixing task within a multi-frame paradigm. Experiments on the SeqCSIST dataset demonstrate that our method outperforms the state-of-the-art approaches with mean Average Precision (mAP) metric improved by 5.3\%. Our dataset and toolkit are available from https://github.com/GrokCV/SeqCSIST.