Can Vision Language Models Be Adaptive in Mathematics Education? A Learner Model-based Rubric Study

TL;DR

This study evaluates the adaptivity of vision language models in mathematics education using a learner model-based rubric, revealing current limitations in personalized instruction capabilities.

Contribution

It introduces a systematic rubric for assessing VLM adaptivity in math tutoring, focusing on cognitive, motivational, and complexity aspects.

Findings

Measurable differences in model adaptivity were observed.

Current VLMs struggle with consistent, personalized responses.

Limited learner information reduces model adaptivity.

Abstract

Adaptive learning refers to educational technologies that track learners' learning progress and adapt the instructional process based on individual learners' learning performance. It is increasingly recognized as critical for developing an effective learning support tool. Vision language models (VLMs) have seen adoption in mathematics education, and students have been using them as learning aids for personalized instruction. However, it is unknown whether VLMs have the ability to adapt to different learner profiles when providing mathematical instructions. Current VLMs lack a systematic evaluation framework for this adaptivity to different learner profiles in mathematics tutoring tasks. To address this gap, we draw on the learner model from the adaptive learning framework (Shute and Towle, 2018) and propose a learner model-based rubric. Our rubric formalizes adaptivity assessment into three aspects: cognitive aspects, motivational aspects, and complexity. We also evaluate two additional dimensions of VLM responses: correctness (of answers and solutions) and quality (of the response itself). Our experimental results show measurable differences in adaptivity across models and also reveal that current VLMs struggle to consistently produce learner model-based instructional responses, especially when receiving limited learner information.