Identifying Difficult exercises in an eTextbook Using Item Response Theory and Logged Data Analysis

TL;DR

This paper uses item response theory and logged student data from an eTextbook to identify the most difficult exercises and assess their quality, aiding educators in improving course content.

Contribution

It introduces a method combining IRT and log data analysis to evaluate exercise difficulty and quality in eTextbook courses.

Findings

Algorithm analysis exercises are most difficult for students.

Six exercises were identified as poor quality and needing improvement.

IRT effectively measures exercise difficulty and quality.

Abstract

The growing dependence on eTextbooks and Massive Open Online Courses (MOOCs) has led to an increase in the amount of students' learning data. By carefully analyzing this data, educators can identify difficult exercises, and evaluate the quality of the exercises when teaching a particular topic. In this study, an analysis of log data from the semester usage of the OpenDSA eTextbook was offered to identify the most difficult data structure course exercises and to evaluate the quality of the course exercises. Our study is based on analyzing students' responses to the course exercises. We applied item response theory (IRT) analysis and a latent trait mode (LTM) to identify the most difficult exercises .To evaluate the quality of the course exercises we applied IRT theory. Our findings showed that the exercises that related to algorithm analysis topics represented the most difficult exercises, and there existing six exercises were classified as poor exercises which could be improved or need some attention.