Colin: A Multimodal Human-AI Co-Creation Storytelling System To Support Children's Multi-Level Narrative Skills

TL;DR

Colin is an interactive multimodal storytelling system designed to enhance children's multi-level narrative skills through voice and visual support, fostering understanding and active story co-creation.

Contribution

This paper introduces Colin, a novel multimodal storytelling tool that actively supports children's narrative development with interactive feedback and multi-sensory engagement.

Findings

Children using Colin showed improved understanding of cause-effect relationships.

Participants created higher quality stories with better connections.

Colin increased children's engagement in storytelling activities.

Abstract

Children develop narrative skills by understanding and actively building connections between elements, image text matching, and consequences. However, it is challenging for children to clearly grasp these multi level links only through explanations of text or the facilitator's speech. To address this, we developed Colin, an interactive storytelling tool that supports children's multi level narrative skills through both voice and visual modalities. In the generation stage, Colin supports the facilitator to define and review the generated text and image content freely. In the understanding stage, a question feedback model helps children understand multi level connections while co creating stories with Colin. In the building phase, Colin actively encourages children to create connections between elements through drawing and speaking. A user study with 20 participants evaluated Colin by measuring children's engagement, understanding of cause and effect relationships, and the quality of their new story creations. Our results demonstrate that Colin significantly enhances the development of children's narrative skills across multiple levels.