Rethinking Infrared Small Target Detection: A Foundation-Driven Efficient Paradigm

TL;DR

This paper introduces a novel foundation-driven paradigm for infrared small target detection that leverages large-scale visual foundation models to improve accuracy efficiently, with a new fusion module, optimization strategy, and comprehensive evaluation metric.

Contribution

The paper pioneers the integration of visual foundation models into SIRST detection, proposing a unified framework with novel modules and metrics for enhanced performance.

Findings

Achieves state-of-the-art results on multiple datasets.

Significantly improves detection accuracy without extra inference cost.

Provides a comprehensive evaluation system for fair comparison.

Abstract

While large-scale visual foundation models (VFMs) exhibit strong generalization across diverse visual domains, their potential for single-frame infrared small target (SIRST) detection remains largely unexplored. To fill this gap, we systematically introduce the frozen representations from VFMs into the SIRST task for the first time and propose a Foundation-Driven Efficient Paradigm (FDEP), which can seamlessly adapt to existing encoder-decoder-based methods and significantly improve accuracy without additional inference overhead. Specifically, a Semantic Alignment Modulation Fusion (SAMF) module is designed to achieve dynamic alignment and deep fusion of the global semantic priors from VFMs with task-specific features. Meanwhile, to avoid the inference time burden introduced by VFMs, we propose a Collaborative Optimization-based Implicit Self-Distillation (CO-ISD) strategy, which enables implicit semantic transfer between the main and lightweight branches through parameter sharing and synchronized backpropagation. In addition, to unify the fragmented evaluation system, we construct a Holistic SIRST Evaluation (HSE) metric that performs multi-threshold integral evaluation at both pixel-level confidence and target-level robustness, providing a stable and comprehensive basis for fair model comparison. Extensive experiments demonstrate that the SIRST detection networks equipped with our FDEP framework achieve state-of-the-art (SOTA) performance on multiple public datasets. Our code is available at https://github.com/YuChuang1205/FDEP-Framework