From Domain Understanding to Design Readiness: a playbook for GenAI-supported learning in Software Engineering

TL;DR

This study explores using a customized ChatGPT tutor in a software engineering course to enhance domain understanding and modeling skills, showing high accuracy and relevance but room for improvement in supportiveness.

Contribution

It introduces seventeen concrete teaching practices for effective genAI-supported learning in software engineering education.

Findings

ChatGPT achieved 98.9% accuracy and 92.2% relevance in student interactions.

Students reported significant gains in self-efficacy for domain learning and DDD.

Supportiveness of ChatGPT responses was relatively low at 37.78%.

Abstract

Software engineering courses often require rapid upskilling in supporting knowledge areas such as domain understanding and modeling methods. We report an experience from a two-week milestone in a master's course where 29 students used a customized ChatGPT (GPT-3.5) tutor grounded in a curated course knowledge base to learn cryptocurrency-finance basics and Domain-Driven Design (DDD). We logged all interactions and evaluated a 34.5% random sample of prompt-answer pairs (60/~174) with a five-dimension rubric (accuracy, relevance, pedagogical value, cognitive load, supportiveness), and we collected pre/post self-efficacy. Responses were consistently accurate and relevant in this setting: accuracy averaged 98.9% with no factual errors and only 2/60 minor inaccuracies, and relevance averaged 92.2%. Pedagogical value was high (89.4%) with generally appropriate cognitive load (82.78%), but supportiveness was low (37.78%). Students reported large pre-post self-efficacy gains for genAI-assisted domain learning and DDD application. From these observations we distill seventeen concrete teaching practices spanning prompt/configuration and course/workflow design (e.g., setting expected granularity, constraining verbosity, curating guardrail examples, adding small credit with a simple quality rubric). Within this single-course context, results suggest that genAI-supported learning can complement instruction in domain understanding and modeling tasks, while leaving room to improve tone and follow-up structure.