Spatio-Temporal Context Learning with Temporal Difference Convolution for Moving Infrared Small Target Detection

TL;DR

This paper introduces TDCNet, a novel neural network that combines temporal difference and 3D convolutions through re-parameterization and attention mechanisms to improve moving infrared small target detection in complex backgrounds.

Contribution

The paper proposes a new TDC re-parameterization module and a TDC-guided attention mechanism, enhancing spatio-temporal feature extraction for infrared small target detection.

Findings

Achieves state-of-the-art performance on IRSTD-UAV dataset.

Effectively suppresses pseudo-motion clutter in complex backgrounds.

Improves detection accuracy over existing methods.

Abstract

Moving infrared small target detection (IRSTD) plays a critical role in practical applications, such as surveillance of unmanned aerial vehicles (UAVs) and UAV-based search system. Moving IRSTD still remains highly challenging due to weak target features and complex background interference. Accurate spatio-temporal feature modeling is crucial for moving target detection, typically achieved through either temporal differences or spatio-temporal (3D) convolutions. Temporal difference can explicitly leverage motion cues but exhibits limited capability in extracting spatial features, whereas 3D convolution effectively represents spatio-temporal features yet lacks explicit awareness of motion dynamics along the temporal dimension. In this paper, we propose a novel moving IRSTD network (TDCNet), which effectively extracts and enhances spatio-temporal features for accurate target detection. Specifically, we introduce a novel temporal difference convolution (TDC) re-parameterization module that comprises three parallel TDC blocks designed to capture contextual dependencies across different temporal ranges. Each TDC block fuses temporal difference and 3D convolution into a unified spatio-temporal convolution representation. This re-parameterized module can effectively capture multi-scale motion contextual features while suppressing pseudo-motion clutter in complex backgrounds, significantly improving detection performance. Moreover, we propose a TDC-guided spatio-temporal attention mechanism that performs cross-attention between the spatio-temporal features from the TDC-based backbone and a parallel 3D backbone. This mechanism models their global semantic dependencies to refine the current frame's features. Extensive experiments on IRSTD-UAV and public infrared datasets demonstrate that our TDCNet achieves state-of-the-art detection performance in moving target detection.