LLMs Integration in Software Engineering Team Projects: Roles, Impact, and a Pedagogical Design Space for AI Tools in Computing Education

TL;DR

This study explores how generative AI tools like ChatGPT and GitHub Copilot influence undergraduate software engineering teams' learning, teamwork, and self-efficacy, offering insights and a pedagogical design space for integrating AI in education.

Contribution

It provides qualitative insights into students' perceptions of GenAI's impact on teamwork and learning, and proposes a pedagogical design space for AI tools in computing education.

Findings

GenAI affects students' coding experience and self-efficacy.

Students perceive GenAI as both supportive and challenging in teamwork.

A preliminary design space for GenAI in programming education is proposed.

Abstract

This work takes a pedagogical lens to explore the implications of generative AI (GenAI) models and tools, such as ChatGPT and GitHub Copilot, in a semester-long 2nd-year undergraduate Software Engineering Team Project. Qualitative findings from survey (39 students) and interviews (eight students) provide insights into the students' views on the impact of GenAI use on their coding experience, learning, and self-efficacy. Our results address a particular gap in understanding the role and implications of GenAI on teamwork, team-efficacy, and team dynamics. The analysis of the learning aspects is distinguished by the application of learning and pedagogy informed lenses to discuss the data. We propose a preliminary design space for GenAI-based programming learning tools highlighting the importance of considering the roles that GenAI can play during the learning process, the varying support-ability patterns that can be applied to each role, and the importance of supporting transparency in GenAI for team members and students in addition to educators.