AI-Supported Mini-Labs: Combining Smartphone-Based Experiments and Multimodal AI

TL;DR

This paper introduces AI-supported Mini-Labs that leverage smartphone sensors and multimodal AI to facilitate physics experiments, data analysis, and interpretation, enhancing science education through personalized, inquiry-based learning.

Contribution

It presents a novel hybrid approach combining smartphone experiments with multimodal large language models to improve data analysis and scientific understanding in education.

Findings

Enhanced data collection with smartphone sensors

AI-assisted analysis improves accuracy and efficiency

Supports inquiry-based and personalized science learning

Abstract

This paper presents the concept of AI-supported Mini-Labs, combining smartphone-based experiments with multimodal large language models (MLLMs). Smartphones, with their integrated sensors and computational power, function as versatile mobile laboratories for physics education. While they enable the collection of rich experimental data, the analysis of complex everyday phenomena has often been limited in the classroom. Advances in MLLMs now allow learners to process multimodal data, text, images, audio, and video, and receive support in experiment design, data analysis, and scientific interpretation. Three case studies highlight the approach: determining a vehicle drag coefficient from accelerometer data, measuring the ionospheric reflection height from lightning-generated signals analyzed as audio spectrograms, and real-time spectroscopy of blood volume dynamics using smartphone video. The results show clear advantages over conventional methods, including time savings, high-quality visualizations, and individualized guidance. Beyond simplifying data analysis, AI-augmented pocket labs foster representational competence, critical thinking, and 21st-century skills. This hybrid approach offers a promising pathway for individualized and inquiry-based science education, though further studies are needed to assess long-term learning effects and potential risks.