Proba-V-ref: Repurposing the Proba-V challenge for reference-aware super resolution

TL;DR

This paper introduces PROBA-V-REF, a modified dataset for satellite image super-resolution that includes the reference image, enabling more accurate evaluation of methods in reference-aware scenarios.

Contribution

The paper proposes a new dataset variant, PROBA-V-REF, which provides the reference image, and demonstrates how this changes the ranking of super-resolution methods.

Findings

Performance varies with different reference image choices.

PROBA-V-REF better reflects real-world reference-aware super-resolution.

Method rankings change when using the new dataset.

Abstract

The PROBA-V Super-Resolution challenge distributes real low-resolution image series and corresponding high-resolution targets to advance research on Multi-Image Super Resolution (MISR) for satellite images. However, in the PROBA-V dataset the low-resolution image corresponding to the high-resolution target is not identified. We argue that in doing so, the challenge ranks the proposed methods not only by their MISR performance, but mainly by the heuristics used to guess which image in the series is the most similar to the high-resolution target. We demonstrate this by improving the performance obtained by the two winners of the challenge only by using a different reference image, which we compute following a simple heuristic. Based on this, we propose PROBA-V-REF a variant of the PROBA-V dataset, in which the reference image in the low-resolution series is provided, and show that the ranking between the methods changes in this setting. This is relevant to many practical use cases of MISR where the goal is to super-resolve a specific image of the series, i.e. the reference is known. The proposed PROBA-V-REF should better reflect the performance of the different methods for this reference-aware MISR problem.