A Hierarchy of Limitations in Machine Learning

TL;DR

This paper provides a structured overview of the various limitations of machine learning models in social contexts, highlighting conceptual, procedural, and statistical shortcomings and their implications.

Contribution

It introduces a hierarchy of limitations in machine learning models applied to society, aiding modelers and users in identifying and addressing potential failure points.

Findings

Identifies inherent commitments in quantification.

Highlights how unmodeled dependencies affect validation.

Provides a hierarchy to understand model limitations.

Abstract

"All models are wrong, but some are useful", wrote George E. P. Box (1979). Machine learning has focused on the usefulness of probability models for prediction in social systems, but is only now coming to grips with the ways in which these models are wrong---and the consequences of those shortcomings. This paper attempts a comprehensive, structured overview of the specific conceptual, procedural, and statistical limitations of models in machine learning when applied to society. Machine learning modelers themselves can use the described hierarchy to identify possible failure points and think through how to address them, and consumers of machine learning models can know what to question when confronted with the decision about if, where, and how to apply machine learning. The limitations go from commitments inherent in quantification itself, through to showing how unmodeled dependencies can lead to cross-validation being overly optimistic as a way of assessing model performance.