The ASKAP/EMU Source Finding Data Challenge

TL;DR

The paper reports on a comprehensive data challenge for radio source finding tools in the EMU survey, evaluating their accuracy, limitations, and potential improvements for ASKAPsoft.

Contribution

It provides a comparative analysis of multiple source finders on simulated data, highlighting strengths, weaknesses, and recommendations for enhancing ASKAPsoft.

Findings

Most finders achieve near 100% completeness at 10sigma.

Reliability remains high at 10sigma but varies at lower sensitivities.

Trade-offs between completeness and reliability are evident across tools.

Abstract

The Evolutionary Map of the Universe (EMU) is a proposed radio continuum survey of the Southern Hemisphere up to declination +30 deg., with the Australian Square Kilometre Array Pathfinder (ASKAP). EMU will use an automated source identification and measurement approach that is demonstrably optimal, to maximise the reliability, utility and robustness of the resulting radio source catalogues. As part of the process of achieving this aim, a "Data Challenge" has been conducted, providing international teams the opportunity to test a variety of source finders on a set of simulated images. The aim is to quantify the accuracy of existing automated source finding and measurement approaches, and to identify potential limitations. The Challenge attracted nine independent teams, who tested eleven different source finding tools. In addition, the Challenge initiators also tested the current ASKAPsoft source-finding tool to establish how it could benefit from incorporating successful features of the other tools. Here we present the results of the Data Challenge, identifying the successes and limitations for this broad variety of the current generation of radio source finding tools. As expected, most finders demonstrate completeness levels close to 100% at 10sigma dropping to levels around 10% by 5sigma. The reliability is typically close to 100% at 10sigma, with performance to lower sensitivities varying greatly between finders. All finders demonstrate the usual trade-off between completeness and reliability, whereby maintaining a high completeness at low signal-to-noise comes at the expense of reduced reliability, and vice-versa. We conclude with a series of recommendations for improving the performance of the ASKAPsoft source-finding tool.