Learning to Code with Context: A Study-Based Approach

TL;DR

This study explores how students use generative AI tools in software development projects, analyzing their effectiveness, challenges, and the potential of context-aware AI assistants to enhance software engineering education.

Contribution

It introduces a repository-aware, locally deployed LLM assistant with RAG capabilities for educational software projects and provides insights into AI integration in programming education.

Findings

AI tools are most effective in coding and debugging tasks.

Students face challenges related to model reliability and understanding AI suggestions.

Context-aware AI support enhances learning and project outcomes.

Abstract

The rapid emergence of generative AI tools is transforming the way software is developed. Consequently, software engineering education must adapt to ensure that students not only learn traditional development methods but also understand how to meaningfully and responsibly use these new technologies. In particular, project-based courses offer an effective environment to explore and evaluate the integration of AI assistance into real-world development practices. This paper presents our approach and a user study conducted within a university programming project in which students collaboratively developed computer games. The study investigates how participants used generative AI tools throughout different phases of the software development process, identifies the types of tasks where such tools were most effective, and analyzes the challenges students encountered. Building on these insights, we further examine a repository-aware, locally deployed large language model (LLM) assistant designed to provide project-contextualized support. The system employs Retrieval-Augmented Generation (RAG) to ground responses in relevant documentation and source code, enabling qualitative analysis of model behavior, parameter sensitivity, and common failure modes. The findings deepen our understanding of context-aware AI support in educational software projects and inform future integration of AI-based assistance into software engineering curricula.