A Survey of Automated Programming Hint Generation -- The HINTS Framework

TL;DR

This paper reviews and unifies various automated programming hint generation techniques into the HINTS framework, highlighting the importance of component analysis for improving educational tools.

Contribution

It introduces the HINTS framework that models hint techniques as components, enabling better comparison, development, and evaluation of automated programming hints.

Findings

All hint techniques can be viewed as combinations of simpler components.

The HINTS framework provides a unified description of diverse hint generation methods.

Emphasizes the need for improved evaluation methods for hint systems.

Abstract

Automated tutoring systems offer the flexibility and scalability necessary to facilitate the provision of high quality and universally accessible programming education. In order to realise the full potential of these systems, recent work has proposed a diverse range of techniques for automatically generating hints to assist students with programming exercises. This paper integrates these apparently disparate approaches into a coherent whole. Specifically, it emphasises that all hint techniques can be understood as a series of simpler components with similar properties. Using this insight, it presents a simple framework for describing such techniques, the Hint Iteration by Narrow-down and Transformation Steps (HINTS) framework, and it surveys recent work in the context of this framework. It discusses important implications of the survey and framework, including the need to further develop evaluation methods and the importance of considering hint technique components when designing, communicating and evaluating hint systems. Ultimately, this paper is designed to facilitate future opportunities for the development, extension and comparison of automated programming hint techniques in order to maximise their educational potential.