Auditing Student-AI Collaboration: A Case Study of Online Graduate CS Students

TL;DR

This study examines how online graduate CS students perceive and prefer AI collaboration in academic tasks, identifying gaps between current AI capabilities and student expectations to inform better AI system design.

Contribution

It provides a detailed analysis of student preferences and concerns regarding AI in education through surveys, highlighting areas for improving AI trustworthiness and alignment with student needs.

Findings

Students see benefits in AI but worry about over-automation and reliability.

Preferences vary across different academic tasks and individual concerns.

Design suggestions for AI systems to better meet student expectations.

Abstract

As generative AI becomes embedded in higher education, it increasingly shapes how students complete academic tasks. While these systems offer efficiency and support, concerns persist regarding over-automation, diminished student agency, and the potential for unreliable or hallucinated outputs. This study conducts a mixed-methods audit of student-AI collaboration preferences by examining the alignment between current AI capabilities and students' desired levels of automation in academic work. Using two sequential and complementary surveys, we capture students' perceived benefits, risks, and preferred boundaries when using AI. The first survey employs an existing task-based framework to assess preferences for and actual usage of AI across 12 academic tasks, alongside primary concerns and reasons for use. The second survey, informed by the first, explores how AI systems could be designed to address these concerns through open-ended questions. This study aims to identify gaps between existing AI affordances and students' normative expectations of collaboration, informing the development of more effective and trustworthy AI systems for education.