# Testing the testing effect on prolific: when retrieval practice fails to boost learning

## TL;DR

This paper investigates whether the testing effect—where testing improves learning—can be reliably observed in online studies using Prolific, finding that it may not work as expected in such settings.

## Contribution

The study explores the replicability of the testing effect in online crowdsourced research, revealing potential limitations of such platforms for studying learning phenomena.

## Key findings

- No significant differences were found between retrieval-practice and restudy conditions in delayed posttests.
- Crowdsourced samples may struggle to sustain the cognitive engagement needed for learning effects like the testing effect.
- Methodological safeguards failed to produce the expected testing effect in online experiments.

## Abstract

The testing effect—whereby retrieving information through testing improves long-term retention more than restudying—is one of the most robust and well-documented phenomena in educational psychology. However, its replicability in online crowdsourced settings remains uncertain. In this paper, we report two preliminary experiments examining whether the testing effect can be detected in studies conducted on Prolific, a platform widely used for behavioral and educational research. Although not conceived as direct replications, both studies relied on validated learning materials and incorporated key methodological features known to enhance the testing effect, including delayed posttests, corrective feedback, and both factual and application-based outcome measures. In both experiments, participants were randomly assigned to a retrieval-practice (test) or restudy condition. No significant differences were found between groups at delayed test, despite multiple safeguards aimed at promoting participant engagement (e.g., prescreening filters, attention checks, fair compensation). These null findings suggest that investigating learning phenomena that require sustained cognitive engagement may be more challenging with crowdsource samples. We argue that the absence of the testing effect in our studies is unlikely to reflect a theoretical limitation of the effect itself, but rather highlights challenges inherent to online research environments. While platforms like Prolific offer valuable advantages in terms of speed and sample diversity, their constraints should be carefully considered when studying learning effects that depend on deep engagement with learning material.