Overconfidence in Photometric Redshift Estimation

TL;DR

This paper introduces a new method to evaluate the accuracy of photometric redshift probability density functions, revealing overconfidence issues in some codes and emphasizing the need for improved tail modeling for precision cosmology.

Contribution

It presents a novel test for photometric redshift PDFs, compares two codes, and highlights the importance of modeling uncertainties and tails for high-precision cosmological surveys.

Findings

BPZ is overconfident with too narrow PDFs

EAZY produces approximately correct confidence levels

Post-hoc smoothing can improve BPZ PDFs

Abstract

We describe a new test of photometric redshift performance given a spectroscopic redshift sample. This test complements the traditional comparison of redshift {\it differences} by testing whether the probability density functions $p(z)$ have the correct {\it width}. We test two photometric redshift codes, BPZ and EAZY, on each of two data sets and find that BPZ is consistently overconfident (the $p(z)$ are too narrow) while EAZY produces approximately the correct level of confidence. We show that this is because EAZY models the uncertainty in its spectral energy distribution templates, and that post-hoc smoothing of the BPZ $p(z)$ provides a reasonable substitute for detailed modeling of template uncertainties. Either remedy still leaves a small surplus of galaxies with spectroscopic redshift very far from the peaks. Thus, better modeling of low-probability tails will be needed for high-precision work such as dark energy constraints with the Large Synoptic Survey Telescope and other large surveys.