Construction and Preliminary Validation of a Dynamic Programming Concept Inventory

TL;DR

This paper presents the development and initial validation of a specialized assessment tool, the Dynamic Programming Concept Inventory, designed to evaluate undergraduate understanding of complex DP concepts in computer science.

Contribution

It introduces a novel, validated concept inventory for dynamic programming, addressing a gap in assessment tools for advanced computer science topics.

Findings

Questions effectively discriminate between student understanding levels

The inventory is of appropriate difficulty for undergraduates

Preliminary validation shows reliable assessment capabilities

Abstract

Concept inventories are standardized assessments that evaluate student understanding of key concepts within academic disciplines. While prevalent across STEM fields, their development lags for advanced computer science topics like dynamic programming (DP) -- an algorithmic technique that poses significant conceptual challenges for undergraduates. To fill this gap, we developed and validated a Dynamic Programming Concept Inventory (DPCI). We detail the iterative process used to formulate multiple-choice questions targeting known student misconceptions about DP concepts identified through prior research studies. We discuss key decisions, tradeoffs, and challenges faced in crafting probing questions to subtly reveal these conceptual misunderstandings. We conducted a preliminary psychometric validation by administering the DPCI to 172 undergraduate CS students finding our questions to be of appropriate difficulty and effectively discriminating between differing levels of student understanding. Taken together, our validated DPCI will enable instructors to accurately assess student mastery of DP. Moreover, our approach for devising a concept inventory for an advanced theoretical computer science concept can guide future efforts to create assessments for other under-evaluated areas currently lacking coverage.