Amplifiers or Equalizers? A Longitudinal Study of LLM Evolution in Software Engineering Project-Based Learning

TL;DR

This longitudinal study examines how recent LLMs influence software engineering education in project-based learning, revealing they can both level the playing field and widen performance gaps among students.

Contribution

It provides empirical insights into the dual role of advanced LLMs as equalizers and amplifiers in SE education over two years.

Findings

Latest LLMs boost performance of weaker students

They create wider gaps among high-performing students

Pose new challenges for equitable SE education

Abstract

As LLMs reshape software development, integrating LLM-augmented practices into SE education has become imperative. While existing studies explore LLMs' educational use in introductory programming or isolated SE tasks, their impact in more open-ended Project-Based Learning (PBL) remains unexplored. This paper introduces a two-year longitudinal study comparing a 2024 (using early free LLMs, $n$=48) and 2025 (using the latest paid LLMs, $n$=46) cohort. Our findings suggest the latest powerful LLMs' dual role: they act as "equalizers," boosting average performance even for programming-weak students, providing opportunities for more authentic SE practices; yet also as "amplifiers," dramatically widening absolute performance gaps, creating new pedagogical challenges for addressing educational inequities.