DuGI-MAE: Improving Infrared Mask Autoencoders via Dual-Domain Guidance

TL;DR

DuGI-MAE introduces a dual-domain guidance approach with a high-entropy token masking strategy and a large infrared dataset, significantly improving infrared image understanding tasks over existing models.

Contribution

The paper presents DuGI-MAE, a novel infrared foundation model that employs dual-domain guidance and entropy-based masking, advancing infrared image interpretation beyond prior models.

Findings

Outperforms existing models in infrared object detection

Enhances semantic segmentation accuracy in infrared images

Demonstrates strong generalization across multiple infrared tasks

Abstract

Infrared imaging plays a critical role in low-light and adverse weather conditions. However, due to the distinct characteristics of infrared images, existing foundation models such as Masked Autoencoder (MAE) trained on visible data perform suboptimal in infrared image interpretation tasks. To bridge this gap, an infrared foundation model known as InfMAE was developed and pre-trained on large-scale infrared datasets. Despite its effectiveness, InfMAE still faces several limitations, including the omission of informative tokens, insufficient modeling of global associations, and neglect of non-uniform noise. In this paper, we propose a Dual-domain Guided Infrared foundation model based on MAE (DuGI-MAE). First, we design a deterministic masking strategy based on token entropy, preserving only high-entropy tokens for reconstruction to enhance informativeness. Next, we introduce a Dual-Domain Guidance (DDG) module, which simultaneously captures global token relationships and adaptively filters non-uniform background noise commonly present in infrared imagery. To facilitate large-scale pretraining, we construct Inf-590K, a comprehensive infrared image dataset encompassing diverse scenes, various target types, and multiple spatial resolutions. Pretrained on Inf-590K, DuGI-MAE demonstrates strong generalization capabilities across various downstream tasks, including infrared object detection, semantic segmentation, and small target detection. Experimental results validate the superiority of the proposed method over both supervised and self-supervised comparison methods. Our code is available in the supplementary material.