Embedding-Only Uplink for Onboard Retrieval Under Shift in Remote Sensing

TL;DR

This paper investigates an onboard remote sensing system that uses compact embeddings for vector search to prioritize hazards, demonstrating its effectiveness across various shifts and tasks with minimal uplink telemetry.

Contribution

It introduces an embedding-only uplink pipeline for remote sensing that remains effective under different shifts and enables task-specific decision heads without extra uplink cost.

Findings

Embedding-only uplink remains useful under various shifts.

kNN retrieval excels in cloud classification tasks.

Centroid-based methods outperform in temporal change detection.

Abstract

Downlink bottlenecks motivate onboard systems that prioritize hazards without transmitting raw pixels. We study a strict setting where a ground station uplinks only compact embeddings plus metadata, and an onboard system performs vector search to triage new captures. We ask whether this embedding-only pipeline remains useful under explicit remote-sensing shift: cross-time (pre/post-event), cross-event/location (different disasters), cross-site cloud (15 geographic sites), and cross-city AOI holdout (buildings). Using OlmoEarth embeddings on a scaled public multi-task benchmark (27 Sentinel-2 L2A scenes, 15 cloud sites, 5 SpaceNet-2 AOIs; 10 seeds), we find that all effective methods rely on the same uplinked embeddings, but the optimal decision head is task-dependent: kNN retrieval is significantly superior for cloud classification (0.92 vs. centroid 0.91; p<0.01, Wilcoxon), while class centroids dominate temporal change detection (0.85 vs. retrieval 0.48; p<0.01). These results show that embedding-only uplink is the key enabler--once embeddings are onboard, the system can select the best head per task at no additional uplink cost, with all telemetry under 1 KB per query.