The Pragmatic Frames of Spurious Correlations in Machine Learning: Interpreting How and Why They Matter

TL;DR

This paper explores how machine learning researchers interpret and judge the significance of correlations, especially spurious ones, through pragmatic frames based on practical, ethical, and normative considerations.

Contribution

It introduces the concept of pragmatic frames to understand how ML research assesses the relevance, generalizability, human-likeness, and harmfulness of correlations beyond formal definitions.

Findings

Identifies four pragmatic frames: relevance, generalizability, human-likeness, harmfulness.

Shows that correlation desirability depends on situated judgments, not just statistical properties.

Highlights the social and ethical dimensions in defining spuriousness in ML.

Abstract

Learning correlations from data forms the foundation of today's machine learning (ML) and artificial intelligence research. While contemporary methods enable the automatic discovery of complex patterns, they are prone to failure when unintended correlations are captured. This vulnerability has spurred a growing interest in interrogating spuriousness, which is often seen as a threat to model performance, fairness, and robustness. In this article, we trace departures from the conventional statistical definition of spuriousness-which denotes a non-causal relationship arising from coincidence or confounding-to examine how its meaning is negotiated in ML research. Rather than relying solely on formal definitions, researchers assess spuriousness through what we call pragmatic frames: Judgments based on what a correlation does in practice-how it affects model behavior, supports or impedes task performance, or aligns with broader normative goals. Drawing on a broad survey of ML literature, we identify four such frames: Relevance (Models should use correlations that are relevant to the task), generalizability (Models should use correlations that generalize to unseen data), human-likeness (Models should use correlations that a human would use to perform the same task), and harmfulness (Models should use correlations that are not socially or ethically harmful). These representations reveal that correlation desirability is not a fixed statistical property but a situated judgment informed by technical, epistemic, and ethical considerations. By examining how a foundational ML conundrum is problematized in research literature, we contribute to broader conversations on the contingent practices through which technical concepts like spuriousness are defined and operationalized.