Graded Relevance Scoring of Written Essays with Dense Retrieval
Salam Albatarni, Sohaila Eltanbouly, Tamer Elsayed

TL;DR
This paper introduces a novel dense retrieval-based method for graded relevance scoring of essays, focusing on the relevance trait, achieving state-of-the-art results and reducing labeling costs.
Contribution
It proposes a new unsupervised dense retrieval approach using clustering and 1-NN classification for relevance scoring in essays, with strong performance on the ASAP++ dataset.
Findings
Achieved state-of-the-art results in task-specific relevance scoring.
Performed comparably to the best models in cross-task scenarios.
Reduced labeling effort by 90% with minimal performance loss.
Abstract
Automated Essay Scoring automates the grading process of essays, providing a great advantage for improving the writing proficiency of students. While holistic essay scoring research is prevalent, a noticeable gap exists in scoring essays for specific quality traits. In this work, we focus on the relevance trait, which measures the ability of the student to stay on-topic throughout the entire essay. We propose a novel approach for graded relevance scoring of written essays that employs dense retrieval encoders. Dense representations of essays at different relevance levels then form clusters in the embeddings space, such that their centroids are potentially separate enough to effectively represent their relevance levels. We hence use the simple 1-Nearest-Neighbor classification over those centroids to determine the relevance level of an unseen essay. As an effective unsupervised dense…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Semantic Web and Ontologies · Advanced Text Analysis Techniques
MethodsContrastive Learning · Focus
