Exoplanet Imaging Data Challenge, phase II: Comparison of algorithms in terms of characterization capabilities

TL;DR

This paper reports on the second phase of the Exoplanet Imaging Data Challenge, comparing algorithms' abilities to accurately characterize exoplanet signals in multispectral high-contrast images from ground-based telescopes.

Contribution

It introduces a standardized dataset and evaluation framework for exoplanet signal characterization, and analyzes algorithm performance based on recent challenge submissions.

Findings

Algorithms vary in accuracy for astrometry and spectro-photometry

Participants provided uncertainties and posterior distributions for their estimates

Results highlight current capabilities and limitations in exoplanet characterization

Abstract

In this communication, we report on the results of the second phase of the Exoplanet Imaging Data Challenge started in 2019. This second phase focuses on the characterization of point sources (exoplanet signals) within multispectral high-contrast images from ground-based telescopes. We collected eight data sets from two high-contrast integral field spectrographs (namely Gemini-S/GPI and VLT/SPHERE-IFS) that we calibrated homogeneously, and in which we injected a handful of synthetic planetary signals (ground truth) to be characterized by the data challenge participants. The tasks of the participants consist of (1) extracting the precise astrometry of each injected planetary signals, and (2) extracting the precise spectro-photometry of each injected planetary signal. Additionally, the participants may provide the 1-sigma uncertainties on their estimation for further analyses. When available, the participants can also provide the posterior distribution used to estimate the position/spectrum and uncertainties. The data are permanently available on a Zenodo repository and the participants can submit their results through the EvalAI platform. The EvalAI submission platform opened on April 2022 and closed on the 31st of May 2024. In total, we received 4 valid submissions for the astrometry estimation and 4 valid submissions for the spectrophotometry (each submission, corresponding to one pipeline, has been submitted by a unique participant). In this communication, we present an analysis and interpretation of the results.