DialogID: A Dialogic Instruction Dataset for Improving Teaching Effectiveness in Online Environments

TL;DR

This paper introduces DialogID, a large-scale, well-annotated dataset of online dialogic instructions with 30,431 examples across 8 categories, and proposes an adversarial training method using pre-trained language models to enhance instruction detection accuracy.

Contribution

The paper provides the first large-scale dataset for online dialogic instruction detection and introduces an adversarial training approach to improve model performance.

Findings

Our method outperforms baseline models in instruction detection accuracy.

The dataset enables better understanding of pedagogical instructions in online education.

Adversarial training improves generalization of dialogic instruction detection.

Abstract

Online dialogic instructions are a set of pedagogical instructions used in real-world online educational contexts to motivate students, help understand learning materials, and build effective study habits. In spite of the popularity and advantages of online learning, the education technology and educational data mining communities still suffer from the lack of large-scale, high-quality, and well-annotated teaching instruction datasets to study computational approaches to automatically detect online dialogic instructions and further improve the online teaching effectiveness. Therefore, in this paper, we present a dataset of online dialogic instruction detection, \textsc{DialogID}, which contains 30,431 effective dialogic instructions. These teaching instructions are well annotated into 8 categories. Furthermore, we utilize the prevalent pre-trained language models (PLMs) and propose a simple yet effective adversarial training learning paradigm to improve the quality and generalization of dialogic instruction detection. Extensive experiments demonstrate that our approach outperforms a wide range of baseline methods. The data and our code are available for research purposes from: https://github.com/ai4ed/DialogID.