Rebutting fake news on full spectral fitting

TL;DR

This paper identifies that the biases reported in spectral fitting results are due to an initialization constraint in the code, and shows that correcting this yields more accurate, less biased results consistent with expectations.

Contribution

It demonstrates that the biases in spectral fitting are caused by an initialization condition, and provides corrected analysis showing reduced biases and improved convergence.

Findings

Biases are due to an A_V < 1 mag initialization constraint.

Correcting the initialization reduces biases significantly.

Bias and scatter decrease with increasing S/N, as expected.

Abstract

A recent paper by Ge et al. performs a series of experiments with two full spectral fitting codes, pPXF and starlight, finding that the two yield consistent results when the input spectrum is not heavily reddened. For E(B-V) > 0.2, however, they claim starlight leads to severe biases in the derived properties. Counterintuitively, and at odds with previous simulations, they find that this behaviour worsens significantly as the signal-to-noise ratio of the input spectrum increases. This communication shows that this is entirely due to an A_V < 1 mag condition imposed while initializing the Markov chains in the code. This choice is normally irrelevant in real-life galaxy work but can become critical in artificial experiments. Alleviating this usually harmless initialization constraint changes the Ge et al. results completely, as was explained to the authors before their publication. We replicate their spectral fitting experiments, finding much smaller biases. Furthermore both bias and scatter in the derived properties all converge as S/N increases, as one would expect. We also show how the very output of the code provides ways of diagnosing anomalies in the fits. The code behaviour has been documented in careful and extensive experiments in the literature, but the biased analysis of Ge et al. is just not representative of starlight at all.