Experimenting with Experimentation: Rethinking The Role of Experimentation in Educational Design

TL;DR

This paper broadens the concept of educational experimentation to include iterative design processes, exemplified through interventions teaching meta-skills to first-year students, and shares lessons learned from this approach.

Contribution

It introduces experiment-inspired design as a new perspective on educational experiments and demonstrates its application through real-world meta-skills interventions.

Findings

Experiment-inspired design fosters iterative educational improvements

Meta-skills interventions can be effectively tested through sequential experiments

Six lessons for applying experimentation to educational design

Abstract

What if we take a broader view of what it means to run an education experiment? In this paper, we explore opportunities that arise when we think beyond the commonly-held notion that the purpose of an experiment is to either accept or reject a pre-defined hypothesis and instead, reconsider experimentation as a means to explore the complex design space of creating and improving instructional content. This is an approach we call experiment-inspired design. Then, to operationalize these ideas in a real-world experimentation venue, we investigate the implications of running a sequence of interventions teaching first-year students "meta-skills": transferable skills applicable to multiple areas of their lives, such as planning, and managing stress. Finally, using two examples as case studies for meta-skills interventions (stress-reappraisal and mental contrasting with implementation intentions), we reflect on our experiences with experiment-inspired design and share six preliminary lessons on how to use experimentation for design.