What Makes a Star Teacher? A Hierarchical BERT Model for Evaluating Teacher's Performance in Online Education

TL;DR

This paper introduces a hierarchical BERT model to evaluate online teachers' performance by analyzing course subtitles, revealing key verbal and structural cues that correlate with teaching effectiveness.

Contribution

The study presents a novel hierarchical BERT-based approach for predicting teacher performance from online course content, incorporating hierarchical and semantic features.

Findings

Verbal cues like question strategy and emotional appeal correlate with performance

Course structure design significantly impacts teacher evaluation

The proposed model outperforms existing state-of-the-art methods

Abstract

Education has a significant impact on both society and personal life. With the development of technology, online education has been growing rapidly over the past decade. While there are several online education studies on student behavior analysis, the course concept mining, and course recommendations (Feng, Tang, and Liu 2019; Pan et al. 2017), there is little research on evaluating teachers' performance in online education. In this paper, we conduct a systematic study to understand and effectively predict teachers' performance using the subtitles of 1,085 online courses. Our model-free analysis shows that teachers' verbal cues (e.g., question strategy, emotional appealing, and hedging) and their course structure design are both significantly correlated with teachers' performance evaluation. Based on these insights, we then propose a hierarchical course BERT model to predict teachers' performance in online education. Our proposed model can capture the hierarchical structure within each course as well as the deep semantic features extracted from the course content. Experiment results show that our proposed method achieves significant gain over several state-of-the-art methods. Our study provides a significant social impact in helping teachers improve their teaching style and enhance their instructional material design for more effective online teaching in the future.