Software Verification and Graph Similarity for Automated Evaluation of Students' Assignments

TL;DR

This paper introduces a new automated grading framework that combines software verification, control flow graph similarity, and automated testing to improve grading accuracy and provide meaningful feedback for student programs, demonstrated on C language assignments.

Contribution

The paper presents a novel grading framework integrating software verification and graph similarity with testing, along with open-source tools applicable across multiple programming languages.

Findings

High correlation between automatic and manual grades.

Tools effectively provide comprehensible feedback.

Framework improves grading precision.

Abstract

In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful in providing a comprehensible feedback that can help students to improve the quality of their programs We also present our corresponding tools that are publicly available and open source. The tools are based on LLVM low-level intermediate code representation, so they could be applied to a number of programming languages. Experimental evaluation of the proposed grading framework is performed on a corpus of university students' programs written in programming language C. Results of the experiments show that automatically generated grades are highly correlated with manually determined grades suggesting that the presented tools can find real-world applications in studying and grading.