The Richest Paradigm You're Not Using: Commercial Videogames at the Intersection of Human-Computer Interaction and Cognitive Science

TL;DR

This paper advocates for using commercial video games as rich, ecologically valid environments to study cognition, combining insights from human-computer interaction and cognitive science with practical research methods.

Contribution

It introduces an affordance-cognition mapping framework and practical guidelines for leveraging commercial games in cognitive science research.

Findings

Commercial games are suitable for studying perception, attention, and executive functions.

A minimal observational toolkit can effectively analyze cognitive processes in games.

The proposed framework guides systematic game selection and research design.

Abstract

Synthesizing from Corbett and Munneke (2025), who demonstrated that questions originating in human-computer interaction (HCI) and game design can be answered through the theoretical toolkit of cognitive science, this perspective argues that commercial videogames represent a largely underutilized research environment at the intersection of these two fields. Cognitive science has long relied on carefully controlled laboratory paradigms to study perception, attention, and executive functioning, raising persistent questions about ecological validity. HCI, by contrast, has spent decades developing methods for studying behavior in rich, complex, interactive environments, but has been less concerned with what that behavior reveals about underlying cognitive mechanisms. Commercial videogames sit precisely at this intersection. They are cognitively demanding by design, motivating by nature, and consistent enough across players to support systematic behavioral comparison. The affordance structure of a game does the work that experimental manipulations typically require of the researcher, instantiating cognitive demands that are genuine, sustained, and meaningful to the player. We argue that perception, attention, and executive functioning can be meaningfully studied within commercial games using a minimal observational toolkit of screen recording, eye tracking, and behavioral timing. We propose an affordance-cognition mapping framework as a systematic basis for game selection and research design and offer practical methodological recommendations for researchers wishing to work in this space.