Exploring the Role of AI Assistants in Computer Science Education: Methods, Implications, and Instructor Perspectives

TL;DR

This study evaluates ChatGPT's problem-solving performance across diverse CS courses, explores modification strategies to prevent misuse, and gathers instructor insights on integrating AI assistants into education.

Contribution

It provides a comprehensive assessment of ChatGPT's capabilities in CS education and proposes methods to adapt course materials to leverage AI effectively while mitigating risks.

Findings

ChatGPT solves a wide range of CS problems with varying accuracy.

Modified problems can reduce potential misuse by students.

Instructors see value in AI but express concerns about fairness and mental models.

Abstract

The use of AI assistants, along with the challenges they present, has sparked significant debate within the community of computer science education. While these tools demonstrate the potential to support students' learning and instructors' teaching, they also raise concerns about enabling unethical uses by students. Previous research has suggested various strategies aimed at addressing these issues. However, they concentrate on the introductory programming courses and focus on one specific type of problem. The present research evaluated the performance of ChatGPT, a state-of-the-art AI assistant, at solving 187 problems spanning three distinct types that were collected from six undergraduate computer science. The selected courses covered different topics and targeted different program levels. We then explored methods to modify these problems to adapt them to ChatGPT's capabilities to reduce potential misuse by students. Finally, we conducted semi-structured interviews with 11 computer science instructors. The aim was to gather their opinions on our problem modification methods, understand their perspectives on the impact of AI assistants on computer science education, and learn their strategies for adapting their courses to leverage these AI capabilities for educational improvement. The results revealed issues ranging from academic fairness to long-term impact on students' mental models. From our results, we derived design implications and recommended tools to help instructors design and create future course material that could more effectively adapt to AI assistants' capabilities.