Lets Make A Story Measuring MR Child Engagement

TL;DR

This pilot study evaluates child engagement in a mixed reality storytelling system compared to paper stories, highlighting challenges in measuring engagement and emphasizing the importance of user comments as a key metric.

Contribution

The paper introduces a novel MR storytelling system for children and provides insights into effective engagement metrics, especially emphasizing qualitative feedback over quantitative measures.

Findings

Quantitative metrics like time and facial expressions are unreliable due to confounding variables.

User comments in interviews are the most reliable indicator of engagement.

Challenges in measuring engagement in mixed reality settings are identified.

Abstract

We present the result of a pilot study measuring child engagement with the Lets Make A Story system, a novel mixed reality, MR, collaborative storytelling system designed for grandparents and grandchildren. We compare our MR experience against an equivalent paper story experience. The goal of our pilot was to test the system with actual child users and assess the goodness of using metrics of time, user generated story content and facial expression analysis as metrics of child engagement. We find that multiple confounding variables make these metrics problematic including attribution of engagement time, spontaneous non-story related conversation and having the childs full forward face continuously in view during the story. We present our platform and experiences and our finding that the strongest metric was user comments in the post-experiential interview.