# Investigating the Essential of Meaningful Automated Formative Feedback for Programming Assignments

## TL;DR

This study examined the effectiveness of different types of automated feedback in programming assignments, finding that basic failure information alone is insufficient and hints do not significantly improve student performance.

## Contribution

It provides empirical evidence on the limited impact of various automated feedback types and offers practical guidance for designing more effective feedback systems.

## Key findings

- Knowing only what failed is insufficient and may encourage gaming behavior.
- Hints do not significantly improve student performance or feedback usage.
- More effective feedback design strategies are suggested based on the results.

## Abstract

This study investigated the essential of meaningful automated feedback for programming assignments. Three different types of feedback were tested, including (a) What's wrong - what test cases were testing and which failed, (b) Gap - comparisons between expected and actual outputs, and (c) Hint - hints on how to fix problems if test cases failed. 46 students taking a CS2 participated in this study. They were divided into three groups, and the feedback configurations for each group were different: (1) Group One - What's wrong, (2) Group Two - What's wrong + Gap, (3) Group Three - What's wrong + Gap + Hint. This study found that simply knowing what failed did not help students sufficiently, and might stimulate system gaming behavior. Hints were not found to be impactful on student performance or their usage of automated feedback. Based on the findings, this study provides practical guidance on the design of automated feedback.