Assessing the Impact of Metacognitive Post-Reflection Exercises on Problem-Solving Skillfulness

TL;DR

This study investigates how metacognitive post-reflection exercises influence students' problem-solving skills, revealing that conceptual understanding plays a significant role in skill development and that interventions should consider students' prior knowledge.

Contribution

It introduces and evaluates specific metacognitive reflection activities and highlights the importance of students' conceptual knowledge in improving problem-solving skills.

Findings

Small but significant improvement in problem-solving scores with reflection exercises.

Conceptual understanding (measured by FCI) better explains problem-solving skill differences.

Remediation for students with low conceptual understanding is crucial.

Abstract

We studied the impact of metacognitive reflections on recently completed work as a way to improve the retention of newly-learned problem-solving techniques. Students video-recorded themselves talking through problems immediately after finishing them, completed ongoing problem-solving strategy maps or problem-sorting exercises, and filled out detailed exam wrappers. We assessed students' problem-solving skillfulness using a combination of validated instruments and final exam questions scored using a rubric that targets problem-solving best practices. We found a small but significant difference between the rubric score distributions for the control and treatment groups. However, a multiple ordered logistic regression using treatment and Force Concept Inventory (FCI) pre-test score as predictors showed that this difference is better explained by the latter. The surprising impact of conceptual preparation on problem-solving skill suggests two things: the importance of remediation for students with insufficient conceptual understanding and the need to consider problem-solving interventions in the context of students' conceptual knowledge base.