Avalia\c{c}\~ao de efici\^encia na leitura: uma abordagem baseada em PLN

TL;DR

This paper presents an automated evaluation model for cloze tests in Brazilian Portuguese, combining orthographic, grammatical, and semantic analyses to accurately assess reading comprehension at scale.

Contribution

It introduces a novel integrated method for automated cloze test evaluation that correlates highly with human judgment, improving scalability and nuance detection.

Findings

High correlation (0.832) with human evaluation

Effective in detecting linguistic variations

Suitable for scalable educational assessment

Abstract

The cloze test, widely used due to its low cost and flexibility, makes it possible to assess reading comprehension by filling in gaps in texts, requiring the mobilization of diverse linguistic repertoires. However, traditional correction methods, based only on exact answers, limit the identification of nuances in student performance. This study proposes an automated evaluation model for the cloze test in Brazilian Portuguese, integrating orthographic (edit distance), grammatical (POS tagging) and semantic (similarity between embeddings) analyses. The integrated method demonstrated its effectiveness, achieving a high correlation with human evaluation (0.832). The results indicate that the automated approach is robust, sensitive to variations in linguistic repertoire and suitable for educational contexts that require scalability.