Explain in Plain Language Questions with Indic Languages: Drawbacks, Affordances, and Opportunities

TL;DR

This paper explores the challenges and opportunities of using language-agnostic 'Explain in Plain Language' questions in computer science education, focusing on Indian languages and evaluating a code generation grading approach.

Contribution

It introduces and evaluates a code generation-based grading method for multilingual EiPL questions, highlighting its potential and practical limitations in diverse linguistic contexts.

Findings

High correctness rate (75%+) for code generated from translations in multiple languages

Students prefer responding in English due to familiarity and perceived grader capabilities

Language barriers and perceptions affect student engagement in native languages

Abstract

Background: Introductory computer science courses use ``Explain in Plain English'' (EiPE) activities to develop and assess students' code comprehension skills, but creating effective autograders for these questions is challenging and limited to English. This is a particular challenge in linguistically diverse countries like India where students may have limited proficiency in English. Methods: We evaluate the efficacy of a recently introduced approach called Code Generation Based Grading (CGBG) in enabling language agnostic ``Explain in Plain Language'' (EiPL) activities. Here students' EiPL responses generate code that is tested for functional equivalence to the original which was being described. Objectives: We initially evaluate the correctness of code generated from correct EiPL responses provided in 10 of India's most commonly spoken languages. To evaluate the effectiveness of the approach in practice, we assess student success and perceptions of EiPL questions in a NPTEL (National Programme on Technology Enhanced Learning) course. Results: We find promising results for the correctness of code generated from translations of correct EiPL responses, with most languages achieving a correctness rate of 75% or higher. However, in practice, many students preferred to respond in English due to greater familiarity with English as a technical language, difficulties writing in their native language, and perceptions of the grader being less capable of generating code from prompts in their mother tongue.