Position: Embracing Negative Results in Machine Learning

TL;DR

This paper advocates for the publication of negative results in machine learning to improve research quality, reduce inefficiencies, and correct community incentives, emphasizing the importance of a balanced scientific record.

Contribution

It argues for normalizing negative result publications in machine learning and provides concrete measures to implement this paradigm shift.

Findings

Publishing negative results can reduce research inefficiencies.

Negative results help correct publication bias in ML.

Encouraging negative results improves scientific rigor.

Abstract

Publications proposing novel machine learning methods are often primarily rated by exhibited predictive performance on selected problems. In this position paper we argue that predictive performance alone is not a good indicator for the worth of a publication. Using it as such even fosters problems like inefficiencies of the machine learning research community as a whole and setting wrong incentives for researchers. We therefore put out a call for the publication of "negative" results, which can help alleviate some of these problems and improve the scientific output of the machine learning research community. To substantiate our position, we present the advantages of publishing negative results and provide concrete measures for the community to move towards a paradigm where their publication is normalized.