The completeness and reliability of threshold and false-discovery-rate source extraction algorithms for compact continuum sources

TL;DR

This paper evaluates the completeness and reliability of various source extraction algorithms for astronomical images, emphasizing the importance of background estimation and false-discovery rate control in producing robust radio source catalogues.

Contribution

It provides a quantitative comparison of source extraction tools like SExtractor, Selavy, and sfind, highlighting the advantages of false-discovery rate methods for reliability.

Findings

sfind with FDR yields more reliable catalogues

Background estimation significantly impacts detection performance

False-discovery rate control improves catalogue robustness

Abstract

The process of determining the number and characteristics of sources in astronomical images is so fundamental to a large range of astronomical problems that it is perhaps surprising that no standard procedure has ever been defined that has well understood properties with a high degree of statistical rigour on completeness and reliability. There are now a large number of commonly used software tools for accomplishing this task, typically with different tools being used for images acquired using different technologies. Despite this, there have been relatively few quantitative analyses of the robustness or reliability of individual tools, or the details of the techniques they implement. We have an opportunity to redress this omission in the context of surveys planned with the Australian Square Kilometre Array Pathfinder (ASKAP). The Evolutionary Map of the Universe (EMU) survey with ASKAP, a continuum survey of the Southern Hemisphere up to declination +30 deg, aims to utilise an automated source identification and measurement approach that is demonstrably optimal, to maximise the reliability, utility and robustness of the resulting radio source catalogues. A key stage in source extraction methods is the background estimation (background level and noise level) and the choice of a threshold high enough to reject false sources yet not so high that the catalogues are significantly incomplete. In this analysis we present results from testing such algorithms as implemented in the SExtractor, Selavy (Duchamp), and sfind tools on simulated data. In particular the effects of background estimation, threshold and false-discovery rate settings are explored. For parameters that give similar completeness, the false-discovery rate method employed by sfind results in a more reliable catalogue compared to the peak threshold methods of SExtractor and Selavy.