A Crowdsourcing Approach To Collecting Tutorial Videos -- Toward Personalized Learning-at-Scale

TL;DR

This study demonstrates that crowdsourcing tutorial videos on logarithms is feasible, cost-effective, and produces diverse, mostly correct content that can enhance personalized learning outcomes at scale.

Contribution

It introduces a scalable crowdsourcing method for collecting math tutorial videos, showing their quality, diversity, and effectiveness for personalized learning.

Findings

Over 80% of crowdsourced videos are mathematically correct.

Approximately 100 videos can be collected weekly at $5 each.

Learning gains from top crowdsourced videos are comparable to Khan Academy videos.

Abstract

We investigated the feasibility of crowdsourcing full-fledged tutorial videos from ordinary people on the Web on how to solve math problems related to logarithms. This kind of approach (a form of learnersourcing) to efficiently collecting tutorial videos and other learning resources could be useful for realizing personalized learning-at-scale, whereby students receive specific learning resources -- drawn from a large and diverse set -- that are tailored to their individual and time-varying needs. Results of our study, in which we collected 399 videos from 66 unique "teachers" on Mechanical Turk, suggest that (1) approximately 100 videos -- over $80\%$ of which are mathematically fully correct -- can be crowdsourced per week for \$5/video; (2) the crowdsourced videos exhibit significant diversity in terms of language style, presentation media, and pedagogical approach; (3) the average learning gains (posttest minus pretest score) associated with watching the videos was stat.~sig.~higher than for a control video ($0.105$ versus $0.045$); and (4) the average learning gains ($0.1416$) from watching the best tested crowdsourced videos was comparable to the learning gains ($0.1506$) from watching a popular Khan Academy video on logarithms.