SFFR: Spatial-Frequency Feature Reconstruction for Multispectral Aerial Object Detection

TL;DR

This paper introduces SFFR, a novel multispectral object detection method that leverages spatial and frequency domain feature reconstruction using KAN modules, improving robustness and accuracy in UAV imagery.

Contribution

The paper proposes a new frequency component exchange strategy and multi-scale Gaussian KAN modules for enhanced feature fusion in multispectral UAV object detection.

Findings

Outperforms existing methods on SeaDroneSee, DroneVehicle, and DVTOD datasets.

Effectively captures spatial and frequency features for better detection.

Demonstrates robustness to scale variations and multispectral data challenges.

Abstract

Recent multispectral object detection methods have primarily focused on spatial-domain feature fusion based on CNNs or Transformers, while the potential of frequency-domain feature remains underexplored. In this work, we propose a novel Spatial and Frequency Feature Reconstruction method (SFFR) method, which leverages the spatial-frequency feature representation mechanisms of the Kolmogorov-Arnold Network (KAN) to reconstruct complementary representations in both spatial and frequency domains prior to feature fusion. The core components of SFFR are the proposed Frequency Component Exchange KAN (FCEKAN) module and Multi-Scale Gaussian KAN (MSGKAN) module. The FCEKAN introduces an innovative selective frequency component exchange strategy that effectively enhances the complementarity and consistency of cross-modal features based on the frequency feature of RGB and IR images. The MSGKAN module demonstrates excellent nonlinear feature modeling capability in the spatial domain. By leveraging multi-scale Gaussian basis functions, it effectively captures the feature variations caused by scale changes at different UAV flight altitudes, significantly enhancing the model's adaptability and robustness to scale variations. It is experimentally validated that our proposed FCEKAN and MSGKAN modules are complementary and can effectively capture the frequency and spatial semantic features respectively for better feature fusion. Extensive experiments on the SeaDroneSee, DroneVehicle and DVTOD datasets demonstrate the superior performance and significant advantages of the proposed method in UAV multispectral object perception task. Code will be available at https://github.com/qchenyu1027/SFFR.