Distribution Shifts at Scale: Out-of-distribution Detection in Earth Observation

TL;DR

This paper introduces TARDIS, a scalable post-hoc out-of-distribution detection method for Earth Observation models, which generates surrogate labels from feature space to identify distribution shifts without requiring OOD data.

Contribution

TARDIS is a novel method that leverages internal feature representations to generate surrogate labels for OOD detection, enabling scalable deployment without OOD data access.

Findings

TARDIS achieves near-upper-bound surrogate labeling performance in most cases.

It matches top existing methods in OOD detection performance.

Deployment on real-world data provides actionable insights into model behavior.

Abstract

Training robust deep learning models is crucial in Earth Observation, where globally deployed models often face distribution shifts that degrade performance, especially in low-data regions. Out-of-distribution (OOD) detection addresses this by identifying inputs that deviate from in-distribution (ID) data. However, existing methods either assume access to OOD data or compromise primary task performance, limiting real-world use. We introduce TARDIS, a post-hoc OOD detection method designed for scalable geospatial deployment. Our core innovation lies in generating surrogate distribution labels by leveraging ID data within the feature space. TARDIS takes a pre-trained model, ID data, and data from an unknown distribution (WILD), separates WILD into surrogate ID and OOD labels based on internal activations, and trains a binary classifier to detect distribution shifts. We validate on EuroSAT and xBD across 17 setups covering covariate and semantic shifts, showing near-upper-bound surrogate labeling performance in 13 cases and matching the performance of top post-hoc activation- and scoring-based methods. Finally, deploying TARDIS on Fields of the World reveals actionable insights into pre-trained model behavior at scale. The code is available at \href{https://github.com/microsoft/geospatial-ood-detection}{https://github.com/microsoft/geospatial-ood-detection}