Practice Makes Better: Quantifying Grade Benefits of Study

TL;DR

This study quantifies how increased practice via an online platform improves final grades in introductory physics, showing a quadratic relationship between study volume and grade gains across diverse student demographics.

Contribution

It provides the first large-scale quantitative analysis of the impact of practice volume on final grades, accounting for demographic factors and revealing persistent benefits.

Findings

Final grade increases quadratically with log of practice questions encountered.

Students with higher practice volume gain approximately 0.77 grade points.

Demographic factors like parental education and minority status influence grade outcomes.

Abstract

Problem Roulette (PR), an online study service at the University of Michigan, offers points-free formative practice to students preparing for examinations in introductory STEM courses. Using four years of PR data involving millions of problem attempts by thousands of students, we quantify benefits of increased practice study volume in introductory physics. After conditioning mean final grade on standardized (ACT/SAT) math test score, we analyze deviations based on student study volume. We find a strong effect; mean course grade rises quadratically with the logarithm of the total number of PR questions encountered over the term ($N_{\rm Q,tot}$), with an overall gain of $0.77 \pm 0.12$ grade points between $1 < N_{\rm Q,tot} < 1000$. The gains are persistent across the range of math test score represented in our sample. While $N_{\rm Q,tot}$ surely correlates with other study habits, the benefits of increased study in general still hold. A model for final grade using test score and study volume largely accounts for demographic stratification, including by sex, parental education level, number of parents at home, nationality / underrepresented minority status, and regional income level, with two significant exceptions: students whose parents did not earn a college degree, who earn $-0.27 \pm 0.04$ grade points ($6.1\sigma$) below expectations and underrepresented minority students at $-0.14 \pm 0.04$ points ($3.6\sigma$). Residual scatter in final grade remains comparable to the maximal study gains, implying that the model is far from deterministic: individual variation trumps mean trends. Our findings can help motivate students to study more and help teachers to identify which types of students may especially need such encouragement.