# There must be an error here! Experimental evidence on coding errors' biases

## TL;DR

This study provides experimental evidence that researchers are more likely to detect coding errors when these errors produce unexpected results, indicating potential biases in scientific findings due to error detection asymmetries.

## Contribution

It demonstrates experimentally that coding errors are more likely to be identified when they lead to unexpected outcomes, highlighting a bias in error detection that can affect research validity.

## Key findings

- Individuals are 20% more likely to detect errors with unexpected results.
- Coding errors can bias scientific research outcomes.
- Error detection is asymmetrical based on result expectations.

## Abstract

Quantitative research relies heavily on coding, and coding errors are relatively common even in published research. In this paper, we examine whether individuals are more or less likely to check their code depending on the results they obtain. We test this hypothesis in a randomized experiment embedded in the recruitment process for research positions at a large international economic organization. In a coding task designed to assess candidates' programming abilities, we randomize whether participants obtain an expected or unexpected result if they commit a simple coding error. We find that individuals are almost 20% more likely to detect coding errors when they lead to unexpected results. This asymmetry in error detection depending on the results they generate suggests that coding errors may lead to biased findings in scientific research.