"Give me the code" -- Log Analysis of First-Year CS Students' Interactions With GPT
Pedro Alves, Bruno Pereira Cipriano

TL;DR
This study analyzes how first-year CS students interact with GPT for coding tasks, revealing their ability to use AI tools effectively and exercise judgment, despite minimal prompt training.
Contribution
It provides insights into students' prompt usage and critical thinking development when using GPT without prior prompt training in a CS education context.
Findings
Most students successfully used GPT-generated solutions.
Half of the students demonstrated critical evaluation skills.
Students incorporated AI solutions into their projects effectively.
Abstract
The impact of Large Language Models (LLMs) like GPT-3, GPT-4, and Bard in computer science (CS) education is expected to be profound. Students now have the power to generate code solutions for a wide array of programming assignments. For first-year students, this may be particularly problematic since the foundational skills are still in development and an over-reliance on generative AI tools can hinder their ability to grasp essential programming concepts. This paper analyzes the prompts used by 69 freshmen undergraduate students to solve a certain programming problem within a project assignment, without giving them prior prompt training. We also present the rules of the exercise that motivated the prompts, designed to foster critical thinking skills during the interaction. Despite using unsophisticated prompting techniques, our findings suggest that the majority of students…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTeaching and Learning Programming
Methods15 Ways to Contact How can i speak to someone at Delta Airlines · Attention Is All You Need · Label Smoothing · Discriminative Fine-Tuning · Linear Layer · Cosine Annealing · Byte Pair Encoding · {Dispute@FaQ-s}How to file a dispute with Expedia? · Adam · Residual Connection
