From NLG Evaluation to Modern Student Assessment in the Era of ChatGPT: The Great Misalignment Problem and Pedagogical Multi-Factor Assessment (P-MFA)

TL;DR

This paper highlights the misalignment between traditional assessment methods and modern AI-assisted student work, proposing a multi-factor pedagogical framework to improve evaluation validity.

Contribution

It introduces the P-MFA model, a process-based, multi-evidence assessment framework inspired by multi-factor authentication to address assessment challenges in the AI era.

Findings

Traditional assessments are losing validity due to AI tools like ChatGPT.

The P-MFA model offers a new multi-evidence approach to student evaluation.

The framework aims to better align assessment with learning processes.

Abstract

This paper explores the growing epistemic parallel between NLG evaluation and grading of students in a Finnish University. We argue that both domains are experiencing a Great Misalignment Problem. As students increasingly use tools like ChatGPT to produce sophisticated outputs, traditional assessment methods that focus on final products rather than learning processes have lost their validity. To address this, we introduce the Pedagogical Multi-Factor Assessment (P-MFA) model, a process-based, multi-evidence framework inspired by the logic of multi-factor authentication.