Modelling Assessment Rubrics through Bayesian Networks: a Pragmatic Approach

TL;DR

This paper introduces a Bayesian network-based method for automatically deriving learner models from assessment rubrics, simplifying expert elicitation and enabling real-time assessment in intelligent tutoring systems.

Contribution

It presents a novel approach to model assessment rubrics with Bayesian networks using noisy gates, facilitating quick automation of learner assessment tasks.

Findings

Effective modeling of assessment rubrics using Bayesian networks.

Simplified parameter elicitation through logical gates with uncertainty.

Potential for real-time, automated assessment in intelligent tutoring systems.

Abstract

Automatic assessment of learner competencies is a fundamental task in intelligent tutoring systems. An assessment rubric typically and effectively describes relevant competencies and competence levels. This paper presents an approach to deriving a learner model directly from an assessment rubric defining some (partial) ordering of competence levels. The model is based on Bayesian networks and exploits logical gates with uncertainty (often referred to as noisy gates) to reduce the number of parameters of the model, so to simplify their elicitation by experts and allow real-time inference in intelligent tutoring systems. We illustrate how the approach can be applied to automatize the human assessment of an activity developed for testing computational thinking skills. The simple elicitation of the model starting from the assessment rubric opens up the possibility of quickly automating the assessment of several tasks, making them more easily exploitable in the context of adaptive assessment tools and intelligent tutoring systems.