Where Assessment Validation and Responsible AI Meet

TL;DR

This paper proposes a unified assessment framework integrating classical validation principles with Responsible AI practices to ensure ethical, valid, and socially responsible AI use in high-stakes assessments.

Contribution

It introduces a novel framework that combines traditional assessment validation with RAI principles for ethical AI application in assessments.

Findings

Framework supports validity and ethical AI use in assessments

Aligns assessment validation with AI ethics and social responsibility

Addresses responsible AI implementation in high-stakes testing

Abstract

Validity, reliability, and fairness are core ethical principles embedded in classical argument-based assessment validation theory. These principles are also central to the Standards for Educational and Psychological Testing (2014) which recommended best practices for early applications of artificial intelligence (AI) in high-stakes assessments for automated scoring of written and spoken responses. Responsible AI (RAI) principles and practices set forth by the AI ethics community are critical to ensure the ethical use of AI across various industry domains. Advances in generative AI have led to new policies as well as guidance about the implementation of RAI principles for assessments using AI. Building on Chapelle's foundational validity argument work to address the application of assessment validation theory for technology-based assessment, we propose a unified assessment framework that considers classical test validation theory and assessment-specific and domain-agnostic RAI principles and practice. The framework addresses responsible AI use for assessment that supports validity arguments, alignment with AI ethics to maintain human values and oversight, and broader social responsibility associated with AI use.