Captivate! Contextual Language Guidance for Parent-Child Interaction

TL;DR

Captivate! is a real-time system that guides parents during play to enhance children's language development by providing contextually relevant phrases, especially aiding linguistically diverse families.

Contribution

This work introduces the first system offering contextual language guidance during parent-child play, integrating visual and spoken cues for real-time suggestions.

Findings

Improved responsive language use among parents using Captivate!

Effective real-time guidance tailored for diverse linguistic backgrounds

Insights into designing technology for vulnerable populations

Abstract

To acquire language, children need rich language input. However, many parents find it difficult to provide children with sufficient language input, which risks delaying their language development. To aid these parents, we design Captivate!, the first system that provides contextual language guidance to parents during play. Our system tracks both visual and spoken language cues to infer targets of joint attention, enabling the real-time suggestion of situation-relevant phrases for the parent. We design our system through a user-centered process with immigrant families--a highly vulnerable yet understudied population--as well as professional speech language therapists. Next, we evaluate Captivate! on parents with children aged 1-3 to observe improvements in responsive language use. We share insights into developing contextual guidance technology for linguistically diverse families.