Can Multisensory Cues in VR Help Train Pattern Recognition to Citizen Scientists?

TL;DR

This paper explores how multisensory cues in virtual reality can enhance training for citizen scientists, focusing on realism, immersion, and knowledge transfer, through the development of a VR platform called StreamBED.

Contribution

It introduces StreamBED, a VR training platform that investigates the impact of mulsemedia on immersion and learning in citizen science contexts.

Findings

Potential for mulsemedia to improve realism and immersion

Enhanced knowledge transfer through multisensory VR experiences

Future research directions for multisensory learning environments

Abstract

As the internet of things (IoT) has integrated physical and digital technologies, designing for multiple sensory media (mulsemedia) has become more attainable. Designing technology for multiple senses has the capacity to improve virtual realism, extend our ability to process information, and more easily transfer knowledge between physical and digital environments. HCI researchers are beginning to explore the viability of integrating multimedia into virtual experiences, however research has yet to consider whether mulsemedia truly enhances realism, immersion and knowledge transfer. My work developing StreamBED, a VR training platform to train citizen science water monitors plans to consider the role of mulsemedia in immersion and learning goals. Future findings about the role of mulsemedia in learning contexts will potentially allow learners to experience, connect to, learn from spaces that are impossible to experience firsthand.