Difficulty Generating Factors for Context-free Language Construction Assignments

TL;DR

This study identifies factors influencing the difficulty of constructing context-free grammars and pushdown automata, revealing that presentation and structural complexity significantly impact student performance in learning these concepts.

Contribution

It introduces and empirically tests four difficulty-generating factors affecting students' understanding of context-free languages, informing adaptive learning system design.

Findings

Three of four factors significantly affect grammar construction difficulty

Perceived difficulty partly aligns with objective performance

Pushdown automata tasks show inconsistent effects

Abstract

Computer science students often struggle with abstract theoretical concepts, particularly in introductory courses on theoretical computer science. One such challenge is understanding context-free languages and their various representations. In this study we investigate factors that influence the difficulty of constructing context-free grammars and pushdown automata for context-free languages. We propose two potential difficulty generating factors targeting how a language is presented to students: representation in natural language and as a verbose set notation. Furthermore, we propose two factors targeting the structure of the given context-free language: nesting of constructs and insertion of multiplicities. We conducted a controlled experiment using within-subject randomization in an interactive learning system, testing the proposed difficulty factors for constructing context-free grammars and pushdown automata. Our results suggest that three of the four factors significantly influence students' objective performance in solving exercises for constructing context-free grammars, while students' perceived difficulties only partly align with the objective performance measures. The findings for pushdown automata tasks differed markedly from those for context-free grammar tasks. Our variations either had negligible effects or, in some cases, even reduced difficulty. Thus, no robust statistical conclusions can be made for pushdown automata tasks. The results lay foundations for learning systems that adaptively choose appropriate exercises for individual students.