Metabook: A Mobile-to-Headset Pipeline for 3D Story Book Creation in Augmented Reality

TL;DR

Metabook is an innovative system that automates the creation of 3D augmented reality storybooks from text, making it accessible for novices and enhancing children's engagement and learning outcomes.

Contribution

This paper introduces Metabook, the first end-to-end system for automated 3D AR book creation, supported by a design workshop and user studies with adults and children.

Findings

Metabook reduces effort and time for 3D book creation.

Children show increased interest and memory retention with AR 3D books.

No significant difference in comprehension between AR 3D and paper books.

Abstract

The AR 3D book has shown significant potential in enhancing students' learning outcomes. However, the creation process of 3D books requires a significant investment of time, effort, and specialized skills. Thus, in this paper, we first conduct a three-day workshop investigating how AI can support the automated creation of 3D books. Informed by the design insights derived from the workshop, we developed Metabook, a system that enables even novice users to create 3D books from text automatically. To our knowledge, Metabook is the first system to offer end-to-end 3D book generation. A follow-up study with adult users indicates that Metabook enables inexperienced users to create 3D books, achieving reduced efforts and shortened preparation time. We subsequently recruited 22 children to examine the effects of AR 3D books on children's learning compared with paper-based books. The findings indicate that 3D books significantly enhance children's interest, improve memory retention, and reduce cognitive load, though no significant improvement was observed in comprehension. We conclude by discussing strategies for more effectively leveraging 3D books to support children's learning and offer practical recommendations for educators.