Do Most Students Need In-Person Lectures? A Study of a Large Statistics Class

TL;DR

This study compares in-person and online lecture formats in a large undergraduate statistics class, finding minimal differences in student performance and attendance, with online students slightly outperforming in exams but not significantly.

Contribution

It provides empirical evidence on the effectiveness of online versus in-person lectures, using a large sample and robust causal analysis methods.

Findings

Online students rarely attended live lectures.

Online students performed slightly better on exams.

No significant differences across demographic groups.

Abstract

Over 1100 students over four semesters were given the option of taking an introductory undergraduate statistics class either by in-person attendance in lectures or by taking exactly the same class (same instructor, recorded lectures, homework, blind grading, website, etc.) without the in-person lectures. Roughly equal numbers of students chose each option. The online lectures were available to all. Attendance by online students was rare. The online students did slightly better on computer-graded exams. The causal effect of choosing only online lectures was estimated by adjusting for measured confounders, of which the incoming ACT math scores turned out to be most important, using four standard methods. The four nearly identical point estimates remained positive but were small and not statistically significant at the 95% confidence level. Sensitivity analysis indicated that unmeasured confounding was unlikely to be large but might plausibly reduce the point estimate to zero. No statistically significant differences were found in preliminary comparisons of effects on females/males, U.S./non-U.S. citizens, freshmen/non-freshman, and lower-scoring/higher-scoring math ACT groups.