Deduplicating and Ranking Solution Programs for Suggesting Reference Solutions

TL;DR

This paper proposes a deduplication and ranking method for solution programs in online judges to reduce redundancy and help learners access diverse, representative solutions efficiently.

Contribution

It introduces a novel approach to remove near-duplicate solutions and rank unique programs by popularity, significantly reducing the number of solutions users need to review.

Findings

Number of programs reduced by 60.20% after deduplication.

Top-10 programs cover nearly 30% of solutions.

Users only need to refer to about 40% of programs on average.

Abstract

Referring to solution programs written by other users is helpful for learners in programming education. However, current online judge systems just list all solution programs submitted by users for references, and the programs are sorted based on the submission date and time, execution time, or user rating, ignoring to what extent the programs can be helpful to be referenced. In addition, users struggle to refer to a variety of solution approaches since there are too many duplicated and near-duplicated programs. To motivate learners to refer to various solutions to learn better solution approaches, in this paper, we propose an approach to deduplicate and rank common solution programs in each programming problem. Inspired by the nature that the many-duplicated program adopts a more common approach and can be a general reference, we remove the near-duplicated solution programs and rank the unique programs based on the duplicate count. The experiments on the solution programs submitted to a real-world online judge system demonstrate that the number of programs is reduced by 60.20%, whereas the baseline only reduces by 29.59% after the deduplication, meaning that users only need to refer to 39.80% of programs on average. Furthermore, our analysis shows that top-10 ranked programs cover 29.95% of programs on average, indicating that users can grasp 29.95% of solution approaches by referring to only 10 programs. The proposed approach shows the potential of reducing the learners' burden of referring to too many solutions and motivating them to learn a variety of solution approaches.