A blind test of photometric redshifts on ground-based data

TL;DR

This study evaluates the performance of different photometric redshift codes on ground-based data, revealing inconsistencies in error estimates and emphasizing the importance of calibration and template choice for accurate redshift determination.

Contribution

It provides a comparative analysis of multiple photo-z codes on various datasets, highlighting the variability in accuracy and the need for improved error estimation methods.

Findings

Photo-z error estimates often do not correlate with actual accuracy.

Different codes perform better in different regimes.

Secure spectroscopic subsamples may bias performance assessments.

Abstract

Aims. We analyse the relative performance of different photo-z codes in blind applications to ground-based data. Methods. We tested the codes on imaging datasets with different depths and filter coverages and compared the results to large spectroscopic catalogues. The photo-z error behaviour was analysed to select cleaner subsamples with more secure photo-z estimates. We consider Hyperz, BPZ, and the code used in the CADIS, COMBO-17, and HIROCS surveys. Results. The photo-z error estimates of the three codes do not correlate tightly with the accuracy of the photo-z's. While very large errors sometimes indicate a true catastrophic photo-z failure, smaller errors are usually not meaningful. For any given dataset, we find significant differences in redshift accuracy and outlier rates between the different codes when compared to spectroscopic redshifts. However, different codes excel in different regimes. The agreement between different sets of photo-z's is better for the subsample with secure spectroscopic redshifts than for the whole catalogue. Conclusions. Running today's photo-z codes on well-calibrated ground-based data can lead to reasonable accuracy. The actual performance on a given dataset is largely dependent on the template choice and on realistic instrumental response curves. It would be desirable to improve the photo-z error estimation for future applications so as to get a better handle on rejecting objects with grossly inaccurate photo-z's. The secure spectroscopic subsamples commonly used for assessments of photo-z accuracy may be biased toward objects for which the photo-z's are easier to estimate than for a complete flux-limited sample, resulting in very optimistic estimates. (abridged)