Does the Model Say What the Data Says? A Simple Heuristic for Model Data Alignment
Henry Salgado, Meagan R. Kendall, Martine Ceberio

TL;DR
This paper introduces a simple, efficient, and model-agnostic framework to evaluate if machine learning models truly reflect the structure of the data they are trained on, enhancing interpretability.
Contribution
It presents a novel baseline method inspired by Rubin's framework that compares data-driven feature effects with model explanations for better alignment assessment.
Findings
The method effectively measures model-data alignment in binary classification.
It provides a transparent baseline for interpretability beyond existing explanation techniques.
The approach is computationally efficient and applicable across models.
Abstract
In this work, we propose a simple and computationally efficient framework for evaluating whether machine learning models align with the structure of the data they learn from; that is, whether the model says what the data says. Unlike existing interpretability methods that focus exclusively on explaining model behavior, our approach establishes a baseline derived directly from the data itself. Drawing inspiration from Rubin's Potential Outcomes Framework, we quantify how strongly each feature separates the two outcome groups in a binary classification task, moving beyond traditional descriptive statistics to estimate each feature's effect on the outcome. By comparing these data-derived feature rankings with model-based explanations, we provide practitioners with an interpretable and model-agnostic method for assessing model-data alignment.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
