AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling

TL;DR

This paper emphasizes the importance of interpretability in AI systems for education, highlighting lessons from Open Learner Modelling (OLM) research to inform broader interpretable AI frameworks.

Contribution

It reviews three decades of OLM research to extract principles for creating interpretable AI, applicable beyond education in high-stakes domains.

Findings

OLM research demonstrates effective methods for making AI models transparent.

Interpretability enhances human understanding and trust in AI systems.

Lessons from education can inform interpretable AI in other fields.

Abstract

Interpretability of the underlying AI representations is a key raison d'\^{e}tre for Open Learner Modelling (OLM) -- a branch of Intelligent Tutoring Systems (ITS) research. OLMs provide tools for 'opening' up the AI models of learners' cognition and emotions for the purpose of supporting human learning and teaching. Over thirty years of research in ITS (also known as AI in Education) produced important work, which informs about how AI can be used in Education to best effects and, through the OLM research, what are the necessary considerations to make it interpretable and explainable for the benefit of learning. We argue that this work can provide a valuable starting point for a framework of interpretable AI, and as such is of relevance to the application of both knowledge-based and machine learning systems in other high-stakes contexts, beyond education.