Watermark in the Classroom: A Conformal Framework for Adaptive AI Usage Detection

TL;DR

This paper introduces a conformal watermarking framework for AI usage detection in education, effectively controlling false positive rates and accommodating diverse student groups and AI editing levels.

Contribution

It adapts watermarking detection methods with conformal techniques for classroom settings, addressing bias and false positives in AI detection.

Findings

Effective FPR control across diverse classroom scenarios

Robust detection across multiple AI editing levels

Quantitative framework for academic integrity enforcement

Abstract

As artificial intelligence tools become ubiquitous in education, maintaining academic integrity while accommodating pedagogically beneficial AI assistance presents unprecedented challenges. Current AI detection systems fail to control false positive rates (FPR) and suffer from bias against minority student groups, prompting institutional suspensions of these technologies. Watermarking techniques offer statistical rigor through precise $p$-values but remain untested in educational contexts where students may use varying levels of permitted AI edits. We present the first adaptation of watermarking-based detection methods for classroom settings, introducing conformal methods that effectively control FPR across diverse classroom settings. Using essays from native and non-native English speakers, we simulate seven levels of AI editing interventions--from grammar correction to content expansion--across multiple language models and watermarking schemes, and evaluate our proposal under these different setups. Our findings provide educators with quantitative frameworks to enforce academic integrity standards while embracing AI integration in the classroom.