Relying on LLMs: Student Practices and Instructor Norms are Changing in Computer Science Education

TL;DR

This study explores how students and instructors in computer science education are adapting to the use of large language models, revealing evolving norms, conflicts, and instructional strategies across various learning scenarios.

Contribution

It provides empirical insights into changing practices and norms around LLM use in CS education, and proposes design guidelines for LLM deployment in learning environments.

Findings

Varying conflict levels between student practices and instructor norms across scenarios.

Instructors increasingly recognize LLMs as legitimate tools for high-quality work.

Guidelines for LLM design to support learning and prevent misuse.

Abstract

Prior research has raised concerns about students' over-reliance on large language models (LLMs) in higher education. This paper examines how Computer Science students and instructors engage with LLMs across five scenarios: "Writing", "Quiz", "Programming", "Project-based learning", and "Information retrieval". Through user studies with 16 students and 6 instructors, we identify 7 key intents, including increasingly complex student practices. Findings reveal varying levels of conflict between student practices and instructor norms, ranging from clear conflict in "Writing-generation" and "(Programming) quiz-solving", through partial conflict in "Programming project-implementation" and "Project-based learning", to broad agreement in "Writing-revision & ideation", "(Programming) quiz-correction" and "Info-query & summary". We document instructors are shifting from prohibiting to recognizing students' use of LLMs for high-quality work, integrating usage records into assessment grading. Finally, we propose LLM design guidelines: deploying default guardrails with game-like and empathetic interaction to prevent students from "deserting" LLMs, especially for "Writing-generation", while utilizing comprehension checks in low-conflict intents to promote learning.