Enhancing Item Response Theory for Cognitive Diagnosis

TL;DR

This paper introduces DIRT, a deep learning framework that enhances traditional Item Response Theory by incorporating question text semantics to improve cognitive diagnosis accuracy and interpretability.

Contribution

The paper proposes a novel Deep Item Response Theory (DIRT) framework that integrates semantic question representations with deep learning to improve cognitive diagnosis.

Findings

DIRT outperforms traditional IRT in real-world data analysis.

DIRT provides more accurate and interpretable student performance diagnostics.

Semantic question representations enhance the diagnosis process.

Abstract

Cognitive diagnosis is a fundamental and crucial task in many educational applications, e.g., computer adaptive test and cognitive assignments. Item Response Theory (IRT) is a classical cognitive diagnosis method which can provide interpretable parameters (i.e., student latent trait, question discrimination, and difficulty) for analyzing student performance. However, traditional IRT ignores the rich information in question texts, cannot diagnose knowledge concept proficiency, and it is inaccurate to diagnose the parameters for the questions which only appear several times. To this end, in this paper, we propose a general Deep Item Response Theory (DIRT) framework to enhance traditional IRT for cognitive diagnosis by exploiting semantic representation from question texts with deep learning. In DIRT, we first use a proficiency vector to represent students' proficiency in knowledge concepts and embed question texts and knowledge concepts to dense vectors by Word2Vec. Then, we design a deep diagnosis module to diagnose parameters in traditional IRT by deep learning techniques. Finally, with the diagnosed parameters, we input them into the logistic-like formula of IRT to predict student performance. Extensive experimental results on real-world data clearly demonstrate the effectiveness and interpretation power of DIRT framework.

Tables

Table 1. Table 1: The statistics of the dataset.

Statistics	Original	Pruned
# of history records	65,368,739	5,068,039
# of students	1,016,235	81,624
# of questions	1,735,635	13,635
# of knowledge concepts	1,412	621
# percent of text length $\le$ 30	/	94%
Avg. questions per student	/	62.09
Avg. concepts per question	/	1.49

Equations

$$P(\theta )=\frac{1}{1+e^{-Da(\theta -b)}},$$

$$\mathbf{P}(\mathbf{\theta })=\frac{e^{{\mathbf{a}}^T\mathbf{\theta }+d}}{1+e^{{\mathbf{a}}^T\mathbf{\theta }+d}},$$

$$k_i=K_i{\text{W}}_k,$$

$$\theta ={\text{DNN}}_{\theta }(\Theta ),\Theta =\sum _{k_i\in {\mathcal{K}}_q}{\alpha }_i{k}_i,$$

$$a=8\times \text{sigmoid}({\text{DNN}}_a(A)-0.5),A=\sum _{k_i\in {\mathcal{K}}_q}{k}_i,$$

$$i_t=\sigma (W_{\text{xi}}{x}_t+W_{\text{hi}}{h}_{t-1}+b_i),$$

$$f_t=\sigma (W_{\text{xf}}{x}_t+W_{\text{hf}}{h}_{t-1}+b_f),$$

$$o_t=\sigma (W_{\text{xo}}{x}_t+W_{\text{ho}}{h}_{t-1}+b_o),$$

$$c_t=f_t{c}_{t-1}+i_t\cdot tanh(W_{\text{xc}}{x}_t+W_{\text{hc}}{h}_{t-1}+b_c),$$

$$h_t=o_{t}tanh(c_t),$$

$$x_t=\sum _{k_i\in {\mathcal{K}}_q}\text{softmax}(\frac{{\xi }_j}{\sqrt{d_0}})k_i+w_t,{\xi }_j=w_t^T{k}_i,$$

$$b=8\times (\text{sigmoid}(avgragePooling(h_N))-0.5),$$

$$\mathcal{L}=r_{ij}\text{log}\widetilde{r_{ij}}+(1-r_{ij})\text{log}(1-\widetilde{r_{ij}}),$$

Full text

1 Introduction

Many education systems such as intelligent tutoring systems and massive open online course, provide a series of computer aided applications, e.g., computer adaptive test Gershon (2005) and cognitive assessment Tatsuoka (2009). Among these applications, cognitive diagnosis which aims at discovering the latent traits or characteristics of students is a fundamental and crucial task. For instance, diagnosing student knowledge concept proficiency Wu et al. (2015); ze Wu et al. (2017), knowledge tracing Chen et al. (2017).

The well-known Item Response Theory (IRT) Embretson and Reise (2000) which roots in psychological measurement theory is a classical cognitive diagnosis method. It can predict student performance on the questions by the logistic-like IRT formula and provide interpretable parameters (i.e., student latent trait, discrimination and difficulty of questions) for performance analyzing. Since the strong interpretability of these parameters, IRT has been widely applied in many applications Hambleton (1989), such as personalized adaptive test Van der Linden et al. (2000), cognitive assessment Tatsuoka (2009).

Though IRT has made great success in cognitive diagnosis, there are still three issues limiting its usefulness. First, traditional IRT can only exploit student response results, and ignores the rich information and the associations between question texts and knowledge concepts. As shown in Figure 1, the question texts and the knowledge concepts on the underline with the same color are closely related, which is helpful for modeling questions Huang et al. (2017). Thus it is critical to improve IRT to exploit semantic representation from question texts. Second, IRT cannot diagnose parameters accurately for the questions which only have several records. Since students often meet these rare questions in real scene, therefore, improving IRT to have strong robustness for rare questions is important. Third, IRT cannot diagnose student proficiency on knowledge concepts, it only provides a overall latent trait for students, while each question usually assesses different knowledge concepts. Although its extended model Multidimensional Item Response Theory (MIRT) Yao and Schwarz (2006) can diagnose student proficiency on knowledge concepts, it is sensitive to the concepts on which students have high proficiency Yao and Schwarz (2006). Thus, enhancing IRT to provide diagnosis results on each knowledge concept in a reliable way is still an open problem.

To this end, in this paper, we propose a general deep item response theory (DIRT) framework to enhance IRT for cognitive diagnosis by deep learning. It is composed of three modules: input, deep diagnosis and prediction module. Specifically, in input module, for diagnosing each knowledge concept proficiency for students, we use a proficiency vector to represent the student proficiency on each knowledge concept, and embed question texts and knowledge concepts to dense vectors by World2Vec Mikolov et al. (2013). In deep diagnosis module, since deep learning has great feature representation ability, and can automatically represent features in the same domain robustly Kim et al. (2013); Zhang et al. (2016), we use deep learning methods to diagnose student latent trait, discrimination and difficulty of questions by exploiting semantic representation from question texts, and enhance the robustness for rare questions. In prediction module, we input the parameters diagnosed by deep diagnosis module into the logistic-like IRT formula to predict student performance. Extensive experimental results on real-world data clearly demonstrate the effectiveness and interpretation power of DIRT framework.

2 Preliminary

In this section, we first give the definition of cognitive diagnosis in this paper. Then we briefly review the classical cognitive diagnosis method IRT and its extended method MIRT.

2.1 Problem Definition

Suppose there are $L$ students, $M$ questions and total $P$ knowledge concepts. Student question records are represented by $R=\{R_{ij}\arrowvert 1\leq i\leq L,1\leq j\leq M\}$, where $R_{ij}=\langle S_{i},Q_{j},r_{ij}\rangle$ denotes the student $S_{i}$ gets score $r_{ij}$ on question $Q_{j}$. $Q_{j}=\langle QT_{j},QK_{j}\rangle$ is composed of question texts $QT_{j}$ and corresponding knowledge concepts $QK_{j}$.

Definition 1**.**

(Cognitive Diagnosis). Given the question records $R$ of student $S$, our goal is to build a uniform model $\mathcal{M}$ to diagnose the proficiency vector $\bm{\alpha}$ on all knowledge concepts $QK$ for each student.

Since there is no ground truth for diagnosis results, and generally, the more accurate the model $\mathcal{M}$ predicts on student performance prediction task, the better the diagnosis results are Wu et al. (2015), so we use performance prediction task to validate the effectiveness of cognitive diagnosis results.

2.2 Item Response Theory

2.2.1 Definition

IRT is one of the most important psychological and educational theories and it roots in psychological measurement Embretson and Reise (2000). With the student latent trait $\theta$, discrimination $a$ and difficulty $b$ of the question, IRT can predict the probability that the student answers the question correctly by logistic-like IRT formula which is defined as follow:

[TABLE]

where $P(\theta)$ is the correct probability, $D$ is a constant which often set as $1.7$. The more details of latent trait $\theta$, discrimination $a$ and difficulty $b$ are as follows:

Latent Trait.

Latent trait $\theta$ of a student represents student’s overall ability and has the same value for all questions. There is no theoretical limitation on the range of latent trait.

Discrimination.

Discrimination $a$ is the ability of the questions to divide students into different levels. The theoretical range of $a$ is $[-4,4]$.

Difficulty.

Difficulty $b$ means how difficult the question is, and the theoretical range of $b$ is also $[-4,4]$.

2.2.2 Shortcomings of Item Response Theory

The latent trait $\theta$, discrimination $a$ and difficulty $b$ described above are usually estimated by probability algorithms such as Maximum Likelihood Estimation (MLE) or Expectation Maximization (EM) only with students’ responses $r$ in students’ question records $R$. Thus, IRT cannot capture the rich semantic information in question texts $QT$. In addition, IRT is not robust enough to diagnose parameters $a$ and $b$ accurately for questions which only have several records, and IRT only diagnose student latent trait on question level, while cannot provide more detailed diagnosis results on concepts.

2.3 Multidimensional Item Response Theory

MIRT is extended from IRT, and its purpose is to meet the demands of multidimensional data Yao and Schwarz (2006). With the student multidimensional latent traits $\bm{\theta}=(\theta_{1},\theta_{2},...,\theta_{m})^{T}$ on each knowledge concepts, the knowledge concept discriminations $\bm{a}=(a_{1},a_{2},...,a_{m})^{T}$ and the intercept term $d$ of the question, MIRT can also predict the probability of the student answers the question correctly by MIRT formula which is defined as follow:

[TABLE]

where $P(\theta)$ is the correct probability the same as IRT.

Since the process of estimating parameters for MIRT is same as IRT, therefore, MIRT has the same shortcomings as IRT. Although MIRT can provide knowledge concept proficiency for students, it is sensitive to the knowledge concepts on which students have high latent trait Yao and Schwarz (2006).

3 DIRT Framework

To overcome the shortcomings in IRT and MIRT, we propose a general DIRT framework which shown in Figure 2 by enhancing IRT with deep learning to obtain latent trait, discrimination and difficulty in traditional IRT. DIRT contains three modules, i.e., input module, deep diagnosis module and prediction module. As shown in Figure 2, the input module initializes a proficiency vector which can provide diagnosis results on each knowledge concept for each student, and embeds question texts and knowledge concepts to vectors. In deep diagnosis module, latent trait, discrimination and difficulty are obtained by exploiting question texts with deep learning techniques(e.g., DNN and LSTM) to enhance model robustness. The prediction module predicts the probability the student answers the question correctly by the logistic-like IRT formula with parameters diagnosed by deep diagnosis module. We give a specific implementation in the section bellow.

3.1 A Specific Implementation of DIRT

In this section, we give a specific implementation of DIRT framework, we first introduce the details of this implementation, and then we specify the learning strategy of it.

3.1.1 Input Module

Given a student $S$, in order to diagnose her proficiency on all $P$ knowledge concepts. We use a proficiency vector $\bm{\alpha}=(\alpha_{1},\alpha_{2},...,\alpha_{P})$ to represent the proficiency on each knowledge concept for the student, where $\alpha_{l}\in[0,1]$ represents the degree a student masters the knowledge concept $l$. The proficiency vector $\bm{\alpha}$ is initialized randomly by normal distribution.

For the question $Q=\langle QT,QK\rangle$, the question texts are described with a sequence of words $QT=\{w_{1},w_{2},...,w_{u}\}$, where $u$ is the length of $QT$, and $w_{i}\in\mathbb{R}^{d_{0}}$ is initialized with a $d_{0}$-dimensional pre-trained word embedding with Word2Vec Mikolov et al. (2013) method. The knowledge concepts are represented by onehot vectors $QK=\{K_{1},K_{2},...,K_{v}\}$, $K_{i}\in\{0,1\}^{P}$, where $v$ is the number of knowledge concepts associate to the question $Q$. Since onehot representation is very sparse for neural network training, we utilize a $d_{1}$-dimension dense layer to acquire the dense embedding Guo et al. (2017) for each knowledge concept $K_{i}$. We sign the dense embedding of $K_{i}$ as $k_{i}$:

[TABLE]

where $\textbf{W}_{k}\in\mathbb{R}^{P\times d_{1}}$ are the parameters of the $d_{1}$-dimension dense layer, and $k_{i}\in\mathbb{R}^{d_{1}}$.

3.1.2 Deep Diagnosis Module

Deep diagnosis module is mainly achieved by deep learning techniques (e.g., DNN, LSTM), that is because deep learning has the great texts auto representation ability, and can exploit texts information from semantic perspective and enhance the robustness of the framework. In this module, we diagnose latent trait, discrimination and difficulty which have strong interpretability in traditional IRT. The details are as follows.

Latent Trait.

Latent trait in IRT has strong interpretability for students’ performance on questions. We diagnose latent trait parameter in this part. The latent trait $\theta$ is closely related to the proficiency of knowledge concepts Yao and Schwarz (2006), students who have great proficiency on knowledge concepts will have high latent trait because concepts proficiency can reflect the comprehensive ability of students. Because of the great feature representation ability of deep learning techniques, we use a deep neural network (DNN) to diagnose latent trait $\theta$. Specifically, given the proficiency vector $\bm{\alpha}=(\alpha_{1},\alpha_{2},...,\alpha_{P})$ of the student $S$ and a question $Q$, we multiply the corresponding proficiency in $\bm{\alpha}$ with the concepts dense embedding of the question, by doing this, we will get a $d_{1}$-dimension vector $\Theta\in\mathbb{R}^{d_{1}}$, and then we input $\Theta$ into DNN to model the latent trait automatically, which is defined as follow:

[TABLE]

where $\mathcal{K}_{q}$ is the set of the dense embedding of $v$ knowledge concepts. Latent trait diagnosed here is different for each question, which has stronger interpretability than it in IRT.

Discrimination.

Discrimination $a$ can be applied to analyze students’ performance distribution on the question. As for the method to obtain discrimination parameter, inspired by the relationship between Multidimensional Item Discrimination (MDISC) and knowledge concepts Yao and Schwarz (2006), we learn question discrimination $a$ from knowledge concepts corresponded to the question. Here, we use another DNN to diagnose question discrimination $a$ with the same reasons as latent trait diagnosing. Specifically, we sum the dense embedding of knowledge concepts in $\mathcal{K}_{q}$ to get a $d_{1}$-dimensional vector $A\in\mathbb{R}^{d_{1}}$, and then, we input $A$ into the DNN to diagnosis question discrimination. Because the theoretical range of $a$ is $[-4,4]$, we use the $sigmoid$ function to minus $0.5$ and multiply $8$ to meet the range requirement. The definition of $a$ is as follow:

[TABLE]

where the structure of $\text{DNN}_{a}$ is same as $\text{DNN}_{\theta}$, but the parameters are not shared between them.

Difficulty.

Difficulty $b$ determines how difficult the question is. The first perspective to diagnosis question difficulty is exploiting question texts, because the difficulty is closely related to question texts Huang et al. (2017), we can model question difficulty is by exploiting question texts. In this term, LSTM can handle and represent long time sequence texts from semantic perspective which can provide strong robustness for rare questions, so we use LSTM to model difficulty $b$ from text perspective. As for the second perspective, the depth and width of knowledge concepts examined by the question also have a great impact on difficulty. The deeper and wider the knowledge concepts are examined, the more difficult the question is. Obviously, the depth and width of the examined concepts can be reflected by the relevance between question texts and the corresponding concepts. As shown in Figure 1, the words in question texts on the underline are related to the knowledge concepts which on the underline with the same color. To capture the associations between question texts and knowledge concepts, we use an attention mechanism to learn the relevance between question texts and concepts. Totally, we design an attention-based LSTM to integrate question texts and knowledge concepts for diagnosing question difficulty $b$, and the architecture of attention-based LSTM is shown in Figure 4. Specifically, the sequence input to this LSTM is $x=(x_{1},x_{2},...,x_{N})$, where $N$ is the max step of the attention-based LSTM. The hidden state $h_{t}$ at $t$-th step is defined as:

[TABLE]

where $i_{*},f_{*},c_{*},o_{*}$ are the input gate, forget gate, memory cell and output gate of LSTM, and $W_{*},b_{*}$ are parameters of the corresponding gate or cell respectively. The $t$-th input step of attention-based LSTM is defined as follow:

[TABLE]

where $\sqrt{d_{0}}$ is the scaling factor Vaswani et al. (2017). $\xi_{j}$ is the relevance between word $w_{t}$ and the knowledge concepts in $\mathcal{K}_{q}$, and $\xi_{j}$ is regarded as the depth and width of the concept examined by the questions, it is calculated by “att” part shown in Figure 4. After that, an average-pooling operation is utilized to obtain parameter $b$. Also, because the theoretical range of $b$ is $[-4,4]$, we also use the $sigmoid$ function to minus $0.5$ and multiply $8$ to meet the range requirement:

[TABLE]

where $avgragePooling$ is an operation that calculates the mean of all elements in the vector $h_{N}$.

3.1.3 Prediction Module

The last module is prediction module. As shown before, traditional IRT can use student latent trait, discrimination and difficulty of the question to predict student performance by the logistic-like formula of IRT. In DIRT framework, in order to preserve the ability of performance prediction and the interpretation of latent trait, discrimination and difficulty in traditional IRT. We input the parameters diagnosed by deep diagnosis module into the logistic-like formula of IRT to predict the student performance on the question. In this way, we can preserve the same interpretability of parameters diagnosed by deep diagnosis module as traditional IRT, and obtain the enhancing performance prediction results which are enhanced by deep learning techniques.

3.1.4 DIRT Learning

The whole parameters to be updated in DIRT mainly exit in two parts: input module and deep diagnosis module. In input module, the parameters need to be updated contain proficiency vector $\bm{\alpha}$, question embedding weights and knowledge concept dense embedding weights $\{\bf{W_{Q}},\bf{W_{K}}\}$. In deep diagnosis module, the parameters need to be updated contain the weights of three neural networks $\{\bf{W_{DNN_{a}}},\bf{W_{DNN_{\theta}}},\bf{W_{LSTM}}\}$ which are used to learn the latent trait, discrimination and difficulty respectively. The objective function of DIRT is the negative log likelihood function. Formally, for student $i$ and question $j$, let $r_{ij}$ be the actual score, $\widetilde{r_{ij}}$ be the score predicted by DIRT. Thus the loss for student $i$ on question $j$ is defined as:

[TABLE]

In this way, we can learn DIRT by directly minimizing the objective function using Adam optimization Kingma and Ba (2014).

4 Experiments

In this section, we conduct extensive experiments to demonstrate the effectiveness of DIRT framework. First, we compare DIRT with baselines on performance prediction task to validate the effectiveness of DIRT for cognitive diagnosis by exploiting question texts. Then, we compare the performance between DIRT and representative baselines to validate the strong robustness of DIRT for rare questions. In the end, we conduct a case study to visualize the strong interpretability of DIRT.

4.1 Dataset Description

The experimental dataset is composed of the mathematical data collected from a number of senior high school in China, in order to make the experimental results more reliable, we filter out the students with less than 15 questions and the questions that have no associate students. After pruning, some statistics of the dataset are shown in Table 1. We can observe that each student has done about 62.09 questions, and each question requires about 1.49 knowledge concepts.

4.2 Experimental Setup

Word Embedding.

The word embedding utilized in input module are trained on a large-scale mathematical question texts of the dataset. Each word of all question texts is embedded to a 50 dimension (e.g., $d_{0}$=50) vector by the public Word2Vec tool Mikolov et al. (2013).

DIRT Setting.

In DIRT, we set the maximum word sequence length $N$ of each question as 30 (padding zero when necessary) because $94\%$ of the questions are less than 30 in length shown in Table 1. The dimension $d_{1}$ of knowledge concepts dense embedding is set as 50 which is the same as $d_{0}$, the size of $h_{t}$ of attention-based LSTM is 50, respectively.

Training Details.

For training DIRT, we randomly initialize all parameters of DIRT framework with uniform distribution which range in $(\sqrt{-6/(n_{in}+n_{out})},\sqrt{6/(n_{in}+n_{out})})$ according to Montavon et al. (2012), where $n_{in}$ and $n_{out}$ are the input and output size of the corresponding ones, respectively. Besides, the dropout technique Srivastava et al. (2014) is also applied to prevent over fitting with probability 0.2 and the batch size is set as 32 for training.

Baseline Approaches.

To demonstrate the effectiveness of DIRT, we compare it with some baseline methods, the details of them are shown as follows:

• •

IRT: IRT Embretson and Reise (2000) is a continuous cognitive diagnosis method that modeling students and questions’ associated parameters by a logistic-like formula.

• •

MIRT: Different from IRT, MIRT Yao and Schwarz (2006) is a multidimensional cognitive diagnosis method that can model multiple knowledge proficiency of student and the parameters of question.

• •

PMF: Probabilistic matrix factorization (PMF) Mnih and Salakhutdinov (2008) is a factorization method that can project students and questions into a low-rank latent factor space.

• •

NMF: Non-negative matrix factorization (NMF) Lee and Seung (2001) is also factorization method, but it is non-negative which can work as a topic model.

• •

DINA: DINA De La Torre (2011) is a discrete cognitive diagnosis method which is contrary to IRT, it models student concept proficiency by a binary vector.

• •

DIRTNA: DIRTNA is a variant of DIRT by only using DNN to represent question texts without attention-based LSTM.

All of the methods are implemented by PyTorch on a Linux server with four 2.0GHz Intel Xeon E5-2620 CPUs and 100G memory.

Evaluation Metrics.

We evaluate the performance of DIRT from two perspectives, regression perspective Willmott and Matsuura (2005): RMSE (Root Mean Square Error), MAE (Mean Absolute Error), and classification perspective Ling et al. (2003): AUC (Area Under ROC) and ACC (Prediction Accuracy).

4.3 Experimental Results

Performance Prediction Task.

Here, we conduct extensive experiments on performance prediction task at different data sparsity by splitting dataset into training and testing dataset with different ratio: $60\%,70\%,80\%,90\%$. The results on all metrics are shown in Figure 5. We can observe that compares to baselines, especially IRT, MIRT and DIRTNA, DIRT performs the best, which illustrates attention-based LSTM is effective for exploiting question texts and DIRT can provide more accurate diagnosis result to enhance traditional IRT. We can also observe that DIRT and IRT perform better than MIRT, which is mainly because MIRT is sensitive to the concept on which student has high proficiency Yao and Schwarz (2006). Therefore, DIRT framework is more reliable than MIRT to the concept on which students have a high proficiency.

Module Robustness.

We conduct experiments to evaluate the strong robustness of DIRT for questions which only appear several times. We set three classical cognitive diagnosis models (CDMs), IRT, MIRT and DINA as representative baselines. Here, we evaluate the strong robustness of DIRT framework on the questions which only appear $1,2,3$ and $4$ times in training dataset. Figure 6 shows DIRT performs better than all other baselines, the results of baselines are bad, because the parameters diagnosed by them are inaccurate. This observation demonstrates that DIRT can diagnose more accurate parameters for rare questions with strong robustness. This is because deep learning techniques have great feature representation ability which is general to all question texts including the rare questions. Thus, DIRT framework has stronger robustness for rare questions.

Case Study.

Here, we give an example of cognitive diagnosis of student knowledge proficiency. As shown in Figure 7, the radar chart shows a student’s concepts proficiency diagnosed by IRT, MIRT and DIRT. Because IRT only diagnose student latent trait which has the same value on all questions, we regard the latent trait in IRT as proficiency on all concepts, so the diagnosis result of IRT is a regular polygon in Figure 7. Therefore, DIRT can provide more accurate diagnosis results on knowledge concepts than IRT. We can also observe that DIRT predicts all three questions correctly, but IRT gets a wrong result on the second question, it is obvious that IRT gets a abnormal value of difficulty $b$ compares. Also, MIRT gets a wrong result on the third question, which is because MIRT is sensitive to concepts on which student has high proficiency Yao and Schwarz (2006) such as $K7$. Totally, DIRT can enhance traditional IRT with deep learning for cognitive diagnosis by exploiting question texts.

5 Related Work

Cognitive Diagnosis.

Cognitive diagnosis is wildly used in educational psychology. Many existing cognitive diagnosis models (CDM) can work well for performance prediction task. These models can be briefly divided into two types: continuous ones and discrete ones. The typical representative of the continuous models are IRT Birnbaum (1968); Embretson and Reise (2000) and MIRT Yao and Schwarz (2006), and they characterize each student by one or a series of continues variables. For discrete ones, deterministic inputs, noisy “and” gate model (DINA) Haertel (1984); De La Torre (2011) represents each student by a binary indicator vector which indicates whether the student masters the knowledge concept or not. Recently, many researches try to improve the prediction performance, some effective models are proposed, such as fuzzy cognitive diagnosis framework (FuzzyCDM) Wu et al. (2015).

Deep Learning.

Deep learning is a state-of-the-art method for many applications, e.g., image recognition Krizhevsky et al. (2012) and natural language processing Wang et al. (2016). Because of the great feature representation abilityMikolov et al. (2013); Bottou (2014), deep learning has been widely used to improve traditional methods. Such as DeepFM Guo et al. (2017) adopts deep neural network to improve traditional low-order FM Rendle (2010) model, EERNN Su et al. (2018) adopts long short-term memory network to improve the performance of deep knowledge tracing (DKT) Piech et al. (2015). Also, because of the strong robustness of deep learning, it also has been used to enhance the robustness of model Kim et al. (2013); Zhang et al. (2016), such as MANN Liu et al. (2018) adopts deep learning to model heterogenous data.

6 Conclusions

In this paper, in order to overcome the shortcomings in traditional IRT, we proposed a general DIRT framework which contained three modules: input module, deep diagnosis module and prediction module, to enhance traditional IRT with deep learning for cognitive diagnosis. Specifically, in input module, we initialized student with a proficiency vector and embedded question texts and knowledge concepts to dense vectors. In deep diagnosis module, we used deep learning to diagnosis latent trait, discrimination and difficulty in IRT by exploiting question texts. In prediction module, we input parameters which are diagnosed by deep diagnosis module into IRT formula to predict student performance. Extensive experiments on a real-world dataset clearly validated the effectiveness and the interpretation power of DIRT. We hope our approach will lead to more relevant researches.