TL;DR
SMARTIES is a versatile foundation model that enables flexible multi-sensor remote sensing data processing by projecting diverse sensor data into a shared spectrum-aware space, improving generalization and scalability.
Contribution
The paper introduces SMARTIES, a unified transformer-based model that handles diverse remote sensing sensors without sensor-specific training, advancing multi-sensor data integration.
Findings
Outperforms sensor-specific models on various tasks
Enables arbitrary sensor combinations during training and inference
Achieves sensor-agnostic representations through cross-sensor token mixup
Abstract
From optical sensors to microwave radars, leveraging the complementary strengths of remote sensing (RS) sensors is crucial for achieving dense spatio-temporal monitoring of our planet. In contrast, recent deep learning models, whether task-specific or foundational, are often specific to single sensors or to fixed combinations: adapting such models to different sensory inputs requires both architectural changes and re-training, limiting scalability and generalization across multiple RS sensors. On the contrary, a single model able to modulate its feature representations to accept diverse sensors as input would pave the way to agile and flexible multi-sensor RS data processing. To address this, we introduce SMARTIES, a generic and versatile foundation model lifting sensor-specific/dependent efforts and enabling scalability and generalization to diverse RS sensors: SMARTIES projects data…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
