Beyond Text: Probing K-12 Educators' Perspectives and Ideas for Learning Opportunities Leveraging Multimodal Large Language Models

TL;DR

This paper explores K-12 educators' perspectives on using multimodal large language models to enhance learning experiences, highlighting opportunities, challenges, and practical considerations for future educational applications.

Contribution

It provides novel insights into educators' envisioned uses of MLLMs in K-12 education through workshops and case studies, focusing on practical implementation and user perspectives.

Findings

Educators see potential for multimodal models to enrich learning interactions.

Practical concerns include interpretability and integration challenges.

Case studies reveal diverse approaches to using MLLMs in classrooms.

Abstract

Multimodal Large Language Models (MLLMs) are beginning to empower new user experiences that can flexibly generate content from a range of inputs, including images, text, speech, and video. These capabilities have the potential to enrich learning by enabling users to capture and interact with information using a variety of modalities, but little is known about how educators envision how MLLMs might shape the future of learning experiences, what challenges diverse teachers encounter when interpreting how these models work, and what practical needs should be considered for successful implementation in educational contexts. We investigated educator perspectives through formative workshops with 12 K-12 educators, where participants brainstormed learning opportunities, discussed practical concerns for effective use, and prototyped their own MLLM-powered learning applications using Claude 3.5 and its Artifacts feature for previewing code-based output. We use case studies to illustrate two contrasting end-user approaches (teacher-and student-driven), and share insights about opportunities and concerns expressed by our participants, ending with implications for leveraging MLLMs for future learning experiences.