Breaking Robustness Barriers in Cognitive Diagnosis: A One-Shot Neural Architecture Search Perspective

TL;DR

This paper introduces OSCD, a novel neural architecture search method for cognitive diagnosis that enhances robustness and performance in noisy data environments, reducing reliance on manual design and improving learner assessment accuracy.

Contribution

The paper presents a multi-objective one-shot neural architecture search framework tailored for cognitive diagnosis, addressing noise robustness and architectural exploration beyond manual design.

Findings

OSCD outperforms existing models in noisy scenarios.

Discovered architectures demonstrate improved accuracy and robustness.

The method efficiently explores diverse architectures via weight-sharing supernet.

Abstract

With the advancement of network technologies, intelligent tutoring systems (ITS) have emerged to deliver increasingly precise and tailored personalized learning services. Cognitive diagnosis (CD) has emerged as a core research task in ITS, aiming to infer learners' mastery of specific knowledge concepts by modeling the mapping between learning behavior data and knowledge states. However, existing research prioritizes model performance enhancement while neglecting the pervasive noise contamination in observed response data, significantly hindering practical deployment. Furthermore, current cognitive diagnosis models (CDMs) rely heavily on researchers' domain expertise for structural design, which fails to exhaustively explore architectural possibilities, thus leaving model architectures' full potential untapped. To address this issue, we propose OSCD, an evolutionary multi-objective One-Shot neural architecture search method for Cognitive Diagnosis, designed to efficiently and robustly improve the model's capability in assessing learner proficiency. Specifically, OSCD operates through two distinct stages: training and searching. During the training stage, we construct a search space encompassing diverse architectural combinations and train a weight-sharing supernet represented via the complete binary tree topology, enabling comprehensive exploration of potential architectures beyond manual design priors. In the searching stage, we formulate the optimal architecture search under heterogeneous noise scenarios as a multi-objective optimization problem (MOP), and develop an optimization framework integrating a Pareto-optimal solution search strategy with cross-scenario performance evaluation for resolution. Extensive experiments on real-world educational datasets validate the effectiveness and robustness of the optimal architectures discovered by our OSCD model for CD tasks.