Fair Knowledge Tracing in Second Language Acquisition

TL;DR

This paper evaluates the fairness of predictive models in second-language acquisition, highlighting how deep learning generally offers better fairness and accuracy than traditional machine learning, with biases influenced by platform and regional factors.

Contribution

It provides a comparative analysis of fairness between deep learning and machine learning models in language learning, emphasizing the importance of equitable predictive modeling.

Findings

Deep learning outperforms machine learning in accuracy and fairness.

Mobile users are favored over non-mobile users.

Bias against developing countries is stronger in machine learning models.

Abstract

In second-language acquisition, predictive modeling aids educators in implementing diverse teaching strategies, attracting significant research attention. However, while model accuracy is widely explored, model fairness remains under-examined. Model fairness ensures equitable treatment of groups, preventing unintentional biases based on attributes such as gender, ethnicity, or economic background. A fair model should produce impartial outcomes that do not systematically disadvantage any group. This study evaluates the fairness of two predictive models using the Duolingo dataset's en\_es (English learners speaking Spanish), es\_en (Spanish learners speaking English), and fr\_en (French learners speaking English) tracks. We analyze: 1. Algorithmic fairness across platforms (iOS, Android, Web). 2. Algorithmic fairness between developed and developing countries. Key findings include: 1. Deep learning outperforms machine learning in second-language knowledge tracing due to improved accuracy and fairness. 2. Both models favor mobile users over non-mobile users. 3. Machine learning exhibits stronger bias against developing countries compared to deep learning. 4. Deep learning strikes a better balance of fairness and accuracy in the en\_es and es\_en tracks, while machine learning is more suitable for fr\_en. This study highlights the importance of addressing fairness in predictive models to ensure equitable educational strategies across platforms and regions.