NeuCo-Bench: A Novel Benchmark Framework for Neural Embeddings in Earth Observation

TL;DR

NeuCo-Bench is a comprehensive benchmark framework for evaluating neural embeddings in Earth Observation, featuring an evaluation pipeline, challenge mode, and a new dataset to promote standardized assessment and reproducibility.

Contribution

It introduces NeuCo-Bench, a novel framework with a challenge mode and a new dataset for evaluating neural embeddings in Earth Observation tasks.

Findings

Public challenge results at CVPR 2025

Reproducibility with SSL4EO-S12-downstream dataset

Ablation studies with state-of-the-art models

Abstract

We introduce NeuCo-Bench, a novel benchmark framework for evaluating (lossy) neural compression and representation learning in the context of Earth Observation (EO). Our approach builds on fixed-size embeddings that act as compact, task-agnostic representations applicable to a broad range of downstream tasks. NeuCo-Bench comprises three components: (i) an evaluation pipeline built around embeddings, (ii) a challenge mode with a hidden-task leaderboard designed to mitigate pretraining bias, and (iii) a scoring system that balances accuracy and stability. To support reproducibility, we release SSL4EO-S12-downstream, a curated multispectral, multitemporal EO dataset. We present results from a public challenge at the 2025 CVPR EARTHVISION workshop and conduct ablations with state-of-the-art foundation models. NeuCo-Bench provides a step towards community-driven, standardized evaluation of neural embeddings for EO and beyond.