Task-Driven Prompt Learning: A Joint Framework for Multi-modal Cloud Removal and Segmentation

TL;DR

This paper introduces TDP-CR, a task-driven multimodal framework that jointly performs cloud removal and land-cover segmentation in remote sensing imagery, improving data utility for Earth observation.

Contribution

The paper proposes a novel Prompt-Guided Fusion mechanism and a two-phase training strategy for effective joint cloud removal and segmentation, with superior performance and efficiency.

Findings

TDP-CR outperforms state-of-the-art methods by 0.18 dB in PSNR.

Achieves 1.4% higher mIoU than multi-task competitors.

Uses only 15% of the parameters compared to existing models.

Abstract

Optical remote sensing imagery is indispensable for Earth observation, yet persistent cloud occlusion limits its downstream utility. Most cloud removal (CR) methods are optimized for low-level fidelity and can over-smooth textures and boundaries that are critical for analysis-ready data (ARD), leading to a mismatch between visually plausible restoration and semantic utility. To bridge this gap, we propose TDP-CR, a task-driven multimodal framework that jointly performs cloud removal and land-cover segmentation. Central to our approach is a Prompt-Guided Fusion (PGF) mechanism, which utilizes a learnable degradation prompt to encode cloud thickness and spatial uncertainty. By combining global channel context with local prompt-conditioned spatial bias, PGF adaptively integrates Synthetic Aperture Radar (SAR) information only where optical data is corrupted. We further introduce a parameter-efficient two-phase training strategy that decouples reconstruction and semantic representation learning. Experiments on the LuojiaSET-OSFCR dataset demonstrate the superiority of our framework: TDP-CR surpasses heavy state-of-the-art baselines by 0.18 dB in PSNR while using only 15\% of the parameters, and achieves a 1.4\% improvement in mIoU consistently against multi-task competitors, effectively delivering analysis-ready data.