Decoupled Complementary Spectral-Spatial Learning for Background Representation Enhancement in Hyperspectral Anomaly Detection

TL;DR

This paper introduces a decoupled spectral-spatial learning framework that enhances background representation in hyperspectral anomaly detection, enabling universal deployment without scene-specific retraining and improving detection performance.

Contribution

The paper proposes a novel two-stage decoupled learning framework with complementary spectral and spatial branches, improving background modeling in hyperspectral anomaly detection.

Findings

Significant performance improvements on HAD100 benchmark

Framework enables training-free, universal deployment

Modest computational overhead compared to baselines

Abstract

A recent class of hyperspectral anomaly detection methods can be trained once on background datasets and then deployed universally without per-scene retraining or parameter tuning, showing strong efficiency and robustness. Building upon this paradigm, we propose a decoupled complementary spectral--spatial learning framework for background representation enhancement. The framework follows a two-stage training strategy: (1) we first train a spectral enhancement network via reverse distillation to obtain robust background spectral representations; and (2) we then freeze the spectral branch as a teacher and train a spatial branch as a complementary student (the "rebellious student") to capture spatial patterns overlooked by the teacher. Complementary learning is achieved through decorrelation objectives that reduce representational redundancy between the two branches, together with reconstruction regularization to prevent the student from learning irrelevant noise. After training, the framework jointly enhances background representations from both spectral and spatial perspectives, and the resulting enhanced features can be plugged into parameter-free, training-free detectors (e.g., the Reed--Xiaoli (RX) detector) for test-time deployment without per-scene retraining or parameter tuning. Experiments on the HAD100 benchmark demonstrate substantial improvements over representative baselines with modest computational overhead, validating the effectiveness of the proposed complementary learning paradigm. Our code is publicly available at https://github.com/xjpp2016/FERS.