Augmented Math: Authoring AR-Based Explorable Explanations by Augmenting Static Math Textbooks

TL;DR

Augmented Math is a machine learning-based system that enables non-technical users to convert static math textbooks into interactive AR explorable explanations by automatically augmenting formulas and figures.

Contribution

The paper presents a novel approach to automatically augment static math textbooks with interactive AR content without programming, based on content extraction and design strategies.

Findings

Users found augmented explanations more engaging for learning.

System successfully extracts and augments math content from textbooks.

User studies confirm improved learning engagement.

Abstract

We introduce Augmented Math, a machine learning-based approach to authoring AR explorable explanations by augmenting static math textbooks without programming. To augment a static document, our system first extracts mathematical formulas and figures from a given document using optical character recognition (OCR) and computer vision. By binding and manipulating these extracted contents, the user can see the interactive animation overlaid onto the document through mobile AR interfaces. This empowers non-technical users, such as teachers or students, to transform existing math textbooks and handouts into on-demand and personalized explorable explanations. To design our system, we first analyzed existing explorable math explanations to identify common design strategies. Based on the findings, we developed a set of augmentation techniques that can be automatically generated based on the extracted content, which are 1) dynamic values, 2) interactive figures, 3) relationship highlights, 4) concrete examples, and 5) step-by-step hints. To evaluate our system, we conduct two user studies: preliminary user testing and expert interviews. The study results confirm that our system allows more engaging experiences for learning math concepts.