Using and Designing Platforms for In Vivo Education Experiments

TL;DR

This paper discusses how online educational platforms can embed in vivo experiments to improve their educational tools through design principles and practical implementations, based on real-world examples from Khan Academy, edX, and ASSISTments.

Contribution

It introduces design principles for platform technology to support randomized experiments that lead to practical educational improvements.

Findings

Successful implementation of randomized experiments in three platforms.

Design principles enabling iterative improvement and collaboration.

Practical benefits demonstrated in ASSISTments platform.

Abstract

In contrast to typical laboratory experiments, the everyday use of online educational resources by large populations and the prevalence of software infrastructure for A/B testing leads us to consider how platforms can embed in vivo experiments that do not merely support research, but ensure practical improvements to their educational components. Examples are presented of randomized experimental comparisons conducted by subsets of the authors in three widely used online educational platforms Khan Academy, edX, and ASSISTments. We suggest design principles for platform technology to support randomized experiments that lead to practical improvements enabling Iterative Improvement and Collaborative Work and explain the benefit of their implementation by WPI co-authors in the ASSISTments platform.