Unified Restoration-Perception Learning: Maritime Infrared-Visible Image Fusion and Segmentation

TL;DR

This paper introduces a unified learning framework for maritime infrared-visible image fusion and segmentation, addressing degradation challenges in maritime scenes with a new dataset and multi-task model.

Contribution

It presents a novel end-to-end multi-task framework and a maritime-specific dataset to improve image restoration, fusion, and segmentation under challenging maritime conditions.

Findings

Achieves state-of-the-art segmentation performance on IVMSD.

Significantly improves robustness and perceptual quality in maritime environments.

Demonstrates effectiveness of multi-task learning in complex maritime scenarios.

Abstract

Marine scene understanding and segmentation plays a vital role in maritime monitoring and navigation safety. However, prevalent factors like fog and strong reflections in maritime environments cause severe image degradation, significantly compromising the stability of semantic perception. Existing restoration and enhancement methods typically target specific degradations or focus solely on visual quality, lacking end-to-end collaborative mechanisms that simultaneously improve structural recovery and semantic effectiveness. Moreover, publicly available infrared-visible datasets are predominantly collected from urban scenes, failing to capture the authentic characteristics of coupled degradations in marine environments. To address these challenges, the Infrared-Visible Maritime Ship Dataset (IVMSD) is proposed to cover various maritime scenarios under diverse weather and illumination conditions. Building upon this dataset, a Multi-task Complementary Learning Framework (MCLF) is proposed to collaboratively perform image restoration, multimodal fusion, and semantic segmentation within a unified architecture. The framework includes a Frequency-Spatial Enhancement Complementary (FSEC) module for degradation suppression and structural enhancement, a Semantic-Visual Consistency Attention (SVCA) module for semantic-consistent guidance, and a cross-modality guided attention mechanism for selective fusion. Experimental results on IVMSD demonstrate that the proposed method achieves state-of-the-art segmentation performance, significantly enhancing robustness and perceptual quality under complex maritime conditions.