WildSAT: Learning Satellite Image Representations from Wildlife Observations

TL;DR

WildSAT introduces a contrastive learning framework that leverages satellite images and wildlife observations to improve remote sensing representations and enable zero-shot location retrieval, advancing ecological monitoring tools.

Contribution

WildSAT is the first to combine satellite images with wildlife observations and textual habitat descriptions for contrastive learning, enhancing remote sensing tasks and zero-shot retrieval capabilities.

Findings

Outperforms ImageNet-pretrained and satellite-specific models on recognition tasks.

Enables zero-shot geographic location search using textual descriptions.

Surpasses recent cross-modal learning methods in remote sensing applications.

Abstract

Species distributions encode valuable ecological and environmental information, yet their potential for guiding representation learning in remote sensing remains underexplored. We introduce WildSAT, which pairs satellite images with millions of geo-tagged wildlife observations readily-available on citizen science platforms. WildSAT employs a contrastive learning approach that jointly leverages satellite images, species occurrence maps, and textual habitat descriptions to train or fine-tune models. This approach significantly improves performance on diverse satellite image recognition tasks, outperforming both ImageNet-pretrained models and satellite-specific baselines. Additionally, by aligning visual and textual information, WildSAT enables zero-shot retrieval, allowing users to search geographic locations based on textual descriptions. WildSAT surpasses recent cross-modal learning methods, including approaches that align satellite images with ground imagery or wildlife photos, demonstrating the advantages of our approach. Finally, we analyze the impact of key design choices and highlight the broad applicability of WildSAT to remote sensing and biodiversity monitoring.