Unlucky Number 13? Manipulating Evidence Subject to Snooping

TL;DR

This paper examines how secret data snooping practices, termed MESSing, can significantly distort statistical inference, demonstrated through empirical analysis of lottery data suggesting 13 is an unlucky number.

Contribution

It introduces the concept of MESSing, categorizes various snooping practices, and quantifies their impact on statistical conclusions through empirical and theoretical examples.

Findings

MESSing can dramatically alter statistical inferences

Empirical example shows potential bias in lottery data

Quantitative analysis highlights risks of secret data manipulation

Abstract

Questionable research practices like HARKing or p-hacking have generated considerable recent interest throughout and beyond the scientific community. We subsume such practices involving secret data snooping that influences subsequent statistical inference under the term MESSing (manipulating evidence subject to snooping) and discuss, illustrate and quantify the possibly dramatic effects of several forms of MESSing using an empirical and a simple theoretical example. The empirical example uses numbers from the most popular German lottery, which seem to suggest that 13 is an unlucky number.