How statistical model development can obscure inequities in STEM student outcomes

TL;DR

This paper investigates how different statistical model specification methods can bias findings about inequities in STEM student outcomes, highlighting the importance of method choice in accurately capturing intersectional disparities.

Contribution

It introduces a critical perspective to evaluate model specification methods and recommends using information criterion for more accurate and equitable analysis of STEM inequities.

Findings

Information criterion aligns best with intersectionality considerations.

Models using information criterion provide more accurate coefficients and uncertainties.

Other methods may obscure or bias inequity findings.

Abstract

Researchers often frame quantitative research as objective, but every step in data collection and analysis can bias findings in often unexamined ways. In this investigation, we examined how the process of selecting variables to include in regression models (model specification) can bias findings about inequities in science and math student outcomes. We identified the four most used methods for model specification in discipline-based education research about equity: a priori, statistical significance, variance explained, and information criterion. Using a quantitative critical perspective that blends statistical theory with critical theory, we reanalyzed the data from a prior publication using each of the four methods and compared the findings from each. We concluded that using information criterion produced models that best aligned with our quantitative critical perspective's emphasis on intersectionality and models with more accurate coefficients and uncertainties. Based on these findings, we recommend researchers use information criterion for specifying models about inequities in STEM student outcomes.