An Approach for Combining Multimodal Fusion and Neural Architecture Search Applied to Knowledge Tracing

TL;DR

This paper introduces a novel framework combining multimodal fusion with neural architecture search for knowledge tracing, improving model performance and incorporating time-aware evaluation.

Contribution

It presents a unified approach that integrates multimodal data fusion with neural architecture search, a novel metric for time-sensitive performance, and demonstrates superior results on real datasets.

Findings

Achieved higher weighted AUC scores compared to baselines

Discovered architectures outperform existing models statistically

Validated effectiveness with real datasets and significance testing

Abstract

Knowledge Tracing is the process of tracking mastery level of different skills of students for a given learning domain. It is one of the key components for building adaptive learning systems and has been investigated for decades. In parallel with the success of deep neural networks in other fields, we have seen researchers take similar approaches in the learning science community. However, most existing deep learning based knowledge tracing models either: (1) only use the correct/incorrect response (ignoring useful information from other modalities) or (2) design their network architectures through domain expertise via trial and error. In this paper, we propose a sequential model based optimization approach that combines multimodal fusion and neural architecture search within one framework. The commonly used neural architecture search technique could be considered as a special case of our proposed approach when there is only one modality involved. We further propose to use a new metric called time-weighted Area Under the Curve (weighted AUC) to measure how a sequence model performs with time. We evaluate our methods on two public real datasets showing the discovered model is able to achieve superior performance. Unlike most existing works, we conduct McNemar's test on the model predictions and the results are statistically significant.