DiffuSAM: Diffusion Guided Zero-Shot Object Grounding for Remote Sensing Imagery

TL;DR

This paper introduces DiffuSAM, a hybrid approach combining diffusion models and segmentation techniques to improve object grounding accuracy in remote sensing images, achieving significant performance gains.

Contribution

The work presents a novel pipeline integrating diffusion-based cues with segmentation models for enhanced remote sensing object localization.

Findings

Achieved over 14% increase in Acc@0.5 compared to previous methods.

Demonstrated robustness and adaptability in complex scenes.

Leveraged complementary strengths of generative and segmentation models.

Abstract

Diffusion models have emerged as powerful tools for a wide range of vision tasks, including text-guided image generation and editing. In this work, we explore their potential for object grounding in remote sensing imagery. We propose a hybrid pipeline that integrates diffusion-based localization cues with state-of-the-art segmentation models such as RemoteSAM and SAM3 to obtain more accurate bounding boxes. By leveraging the complementary strengths of generative diffusion models and foundational segmentation models, our approach enables robust and adaptive object localization across complex scenes. Experiments demonstrate that our pipeline significantly improves localization performance, achieving over a 14% increase in Acc@0.5 compared to existing state-of-the-art methods.