How to Evaluate Games in Education: A Literature Review

TL;DR

This literature review analyzes empirical studies on gamification in higher education, highlighting inconsistencies in study descriptions and offering recommendations for more structured future research to better evaluate game-based learning effects.

Contribution

The paper provides a systematic review of controlled experiments in educational gamification, emphasizing the need for clearer descriptions and standardized methodologies across disciplines.

Findings

Studies often poorly describe control group activities

Educational materials in control groups are often unclear

Computer science studies report results more clearly

Abstract

Adding game elements to higher education is an increasingly common practice. As a result, many recent empirical studies focus on studying the effectiveness of gamified or game-based educational experiences. The findings of these studies are very diverse, showing both positive and negative effects, and thus calling for comparative meta-studies. In this paper we review and analyze different studies, aiming to summarise and evaluate controlled experiments conducted within different scientific disciplines. We focus on the clarity of non-experimental conditions' descriptions and show that in most cases a. educational methods used in control groups' activities are poorly described, b. educational materials used in control groups' activities are often unclear, and c. the starting conditions are unclear. We also noticed that studies in the fields of computer science and engineering, in general, report results more clearly than in other fields. Based on the above finding, we conclude with a few recommendations for the execution of future empirical studies of games in education for the sake of allowing a more structured comparison.