Pitfalls of amateur regression: The Dutch New Herring controversies

TL;DR

This paper critiques a flawed analysis of a Dutch New Herring ranking, demonstrating that simple regression errors led to false accusations of manipulation, emphasizing the importance of proper model specification.

Contribution

It exposes errors in a previous regression analysis, highlights the complexity of factors influencing rankings, and advocates for more robust modeling approaches.

Findings

Errors in the original regression analysis were identified.

Simple models are insufficient for complex, multifactorial data.

Misinterpretation of artifacts as manipulation was demonstrated.

Abstract

Applying simple linear regression models, an economist analysed a published dataset from an influential annual ranking in 2016 and 2017 of consumer outlets for Dutch New Herring and concluded that the ranking was manipulated. His finding was promoted by his university in national and international media, and this led to public outrage and ensuing discontinuation of the survey. We reconstitute the dataset, correcting errors and exposing features already important in a descriptive analysis of the data. The economist has continued his investigations, and in a follow-up publication repeats the same accusations. We point out errors in his reasoning and show that alleged evidence for deliberate manipulation of the ranking could easily be an artefact of specification errors. Temporal and spatial factors are both important and complex, and their effects cannot be captured using simple models, given the small sample sizes and many factors determining perceived taste of a food product.