Rubric-based Learner Modelling via Noisy Gates Bayesian Networks for Computational Thinking Skills Assessment

TL;DR

This paper introduces an advanced Bayesian network model with rubric-based and noisy gates for more accurate and flexible assessment of computational thinking skills, addressing previous limitations in skill ordering and supplementary skills inclusion.

Contribution

It proposes a novel Bayesian network with dummy nodes and layered gates to improve learner modeling for skill assessment, enhancing coherence and flexibility.

Findings

Improved model coherence and interpretability.

Effective incorporation of skill ordering and supplementary skills.

Validated approach on Computational Thinking assessment framework.

Abstract

In modern and personalised education, there is a growing interest in developing learners' competencies and accurately assessing them. In a previous work, we proposed a procedure for deriving a learner model for automatic skill assessment from a task-specific competence rubric, thus simplifying the implementation of automated assessment tools. The previous approach, however, suffered two main limitations: (i) the ordering between competencies defined by the assessment rubric was only indirectly modelled; (ii) supplementary skills, not under assessment but necessary for accomplishing the task, were not included in the model. In this work, we address issue (i) by introducing dummy observed nodes, strictly enforcing the skills ordering without changing the network's structure. In contrast, for point (ii), we design a network with two layers of gates, one performing disjunctive operations by noisy-OR gates and the other conjunctive operations through logical ANDs. Such changes improve the model outcomes' coherence and the modelling tool's flexibility without compromising the model's compact parametrisation, interpretability and simple experts' elicitation. We used this approach to develop a learner model for Computational Thinking (CT) skills assessment. The CT-cube skills assessment framework and the Cross Array Task (CAT) are used to exemplify it and demonstrate its feasibility.