How to avoid machine learning pitfalls: a guide for academic researchers

TL;DR

This paper provides a comprehensive guide for academic researchers on avoiding common machine learning pitfalls through best practices across all stages of the ML process.

Contribution

It offers practical advice tailored to academic research to improve rigor and reliability in machine learning studies.

Findings

Highlights common ML mistakes in academic research

Provides strategies for rigorous model evaluation

Emphasizes importance of fair model comparison

Abstract

Mistakes in machine learning practice are commonplace, and can result in a loss of confidence in the findings and products of machine learning. This guide outlines common mistakes that occur when using machine learning, and what can be done to avoid them. Whilst it should be accessible to anyone with a basic understanding of machine learning techniques, it focuses on issues that are of particular concern within academic research, such as the need to do rigorous comparisons and reach valid conclusions. It covers five stages of the machine learning process: what to do before model building, how to reliably build models, how to robustly evaluate models, how to compare models fairly, and how to report results.