Dynamic Frequency Feature Fusion Network for Multi-Source Remote Sensing Data Classification

TL;DR

This paper introduces DFFNet, a novel neural network that adaptively fuses frequency domain features for improved classification of multi-source remote sensing data, including hyperspectral, SAR, and LiDAR images.

Contribution

It proposes a dynamic filter block for frequency domain feature learning and a spectral-spatial adaptive fusion block for effective cross-modal feature integration.

Findings

DFFNet outperforms existing methods on benchmark datasets.

The dynamic frequency filter improves feature representation.

The spectral-spatial fusion enhances cross-modal information integration.

Abstract

Multi-source data classification is a critical yet challenging task for remote sensing image interpretation. Existing methods lack adaptability to diverse land cover types when modeling frequency domain features. To this end, we propose a Dynamic Frequency Feature Fusion Network (DFFNet) for hyperspectral image (HSI) and Synthetic Aperture Radar (SAR) / Light Detection and Ranging (LiDAR) data joint classification. Specifically, we design a dynamic filter block to dynamically learn the filter kernels in the frequency domain by aggregating the input features. The frequency contextual knowledge is injected into frequency filter kernels. Additionally, we propose spectral-spatial adaptive fusion block for cross-modal feature fusion. It enhances the spectral and spatial attention weight interactions via channel shuffle operation, thereby providing comprehensive cross-modal feature fusion. Experiments on two benchmark datasets show that our DFFNet outperforms state-of-the-art methods in multi-source data classification. The codes will be made publicly available at https://github.com/oucailab/DFFNet.