# Consistency Analysis of Assessment Boards in University Entrance Examinations in Spain

## TL;DR

This study examines inconsistencies in university entrance exam assessments in Spain, revealing significant differences in grading severity across subjects and universities.

## Contribution

The study applies the Many-Facet Rasch Model to analyze rating inconsistencies in EBAU exams, highlighting the need for standardized evaluation criteria.

## Key findings

- Large inconsistencies were found in the rating process across subjects and universities.
- Differences in severity levels were observed both within and between universities.
- The study suggests the need for better criteria to ensure comparable examination standards.

## Abstract

University entrance examinations (EBAU in Spanish) are a nationwide program for assessing student achievement levels in Spain and determining access to public undergraduate degrees. Considering the need to measure the progress of rater performance, this study analyzes rating data from the June 2018 exam sitting in the Valencian Community, Spain. A total of 54 assessment boards and 3000 students from five public universities were observed. The Many-Facet Rasch Model (MFRM) was used as an extension of the one-parameter Rasch measurement model. All facets involved in analyses (examination board severity, subject difficulty, and group performance) were located on a common underlying linear scale. The results showed large inconsistencies in the rating process, with differences in the severity levels of many subjects both within and between universities. This study may serve as a starting point for a reflective debate on the need to apply better criteria for ensuring the comparability of examination standards in Spain.