Assessment HTN (A-HTN) for Automated Task Performance Assessment in 3D Serious Games

TL;DR

This paper introduces Assessment HTN (A-HTN), a novel framework that models tasks and automatically assesses user performance in 3D serious games, matching expert evaluations.

Contribution

A-HTN is the first to integrate task modeling with automatic performance assessment in 3D serious games, enabling efficient and accurate evaluation.

Findings

High correlation between system scores and instructor evaluations

Effective in both single-user and multi-user assessments

Demonstrated success in VR serious game scenarios

Abstract

In the recent years, various 3D mixed reality serious games have been developed for different applications such as physical training, rehabilitation, and education. Task performance in a serious game is a measurement of how efficiently and accurately users accomplish the game's objectives. Prior research includes a graph-based representation of tasks, e.g. Hierarchical Task Network (HTN), which only models a game's tasks but does not perform assessment. In this paper, we propose Assessment HTN (A-HTN), which both models the task efficiently and incorporates assessment logic for game objectives. Based on how the task performance is evaluated, A-HTN automatically performs: (a) Task-level Assessment by comparing object manipulations and (b) Action-level Assessment by comparing motion trajectories. The system can also categorize the task performance assessment into single user or multi-user based on who is being assessed. We showcase the effectiveness of the A-HTN using two 3D VR serious games: a hydrometer experiment and a multi-user chemistry experiment. The A-HTN experiments show a high correlation between instructor scores and the system generated scores indicating that the proposed A-HTN generalizes automatic assessment at par with Subject Matter Experts.