The Impact of Looking Further Ahead: A Comparison of Two Data-driven Unsolicited Hint Types on Performance in an Intelligent Data-driven Logic Tutor

TL;DR

This study compares the effects of two data-driven hint types, Next-Step and Waypoints, on student performance in a logic tutor, finding Next-Step hints generally more beneficial but highlighting potential benefits of Waypoints with additional scaffolding.

Contribution

The paper introduces a modified data-driven hint generator providing Waypoints and compares its effectiveness to Next-Step hints in a logic tutoring context.

Findings

Next-Step hints improved student time, efficiency, and accuracy.

Higher use of Waypoints correlated with better efficiency and time.

Waypoints may require additional scaffolding to be effective.

Abstract

Research has shown assistance can provide many benefits to novices lacking the mental models needed for problem solving in a new domain. However, varying approaches to assistance, such as subgoals and next-step hints, have been implemented with mixed results. Next-Step hints are common in data-driven tutors due to their straightforward generation from historical student data, as well as research showing positive impacts on student learning. However, there is a lack of research exploring the possibility of extending data-driven methods to provide higher-level assistance. Therefore, we modified our data-driven Next-Step hint generator to provide Waypoints, hints that are a few steps ahead, representing problem-solving subgoals. We hypothesized that Waypoints would benefit students with high prior knowledge, and that Next-Step hints would most benefit students with lower prior knowledge. In this study, we investigated the influence of data-driven hint type, Waypoints versus Next-Step hints, on student learning in a logic proof tutoring system, Deep Thought, in a discrete mathematics course. We found that Next-Step hints were more beneficial for the majority of students in terms of time, efficiency, and accuracy on the posttest. However, higher totals of successfully used Waypoints were correlated with improvements in efficiency and time in the posttest. These results suggest that Waypoint hints could be beneficial, but more scaffolding may be needed to help students follow them.