IVGF: The Fusion-Guided Infrared and Visible General Framework
Fangcen Liu, Chenqiang Gao, Fang Chen, Pengcheng Li, Junjie Guo, Deyu, Meng

TL;DR
The paper introduces IVGF, a versatile fusion framework for infrared and visible images that enhances high-level vision tasks like segmentation and detection by leveraging foundation models and novel modules for feature and token enhancement, along with an attention-guided fusion strategy.
Contribution
It presents a general, extendable framework that improves dual-modality vision tasks through innovative feature, token, and fusion modules, outperforming existing methods.
Findings
Outperforms state-of-the-art dual-modality methods in segmentation and detection.
Effective modules validated through extensive ablation studies.
Enhanced anti-missing modality capability in dual-modality segmentation.
Abstract
Infrared and visible dual-modality tasks such as semantic segmentation and object detection can achieve robust performance even in extreme scenes by fusing complementary information. Most current methods design task-specific frameworks, which are limited in generalization across multiple tasks. In this paper, we propose a fusion-guided infrared and visible general framework, IVGF, which can be easily extended to many high-level vision tasks. Firstly, we adopt the SOTA infrared and visible foundation models to extract the general representations. Then, to enrich the semantics information of these general representations for high-level vision tasks, we design the feature enhancement module and token enhancement module for feature maps and tokens, respectively. Besides, the attention-guided fusion module is proposed for effectively fusing by exploring the complementary information of two…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInfrared Target Detection Methodologies
