Identify Design Problems Through Questioning: Exploring Role-playing Interactions with Large Language Models to Foster Design Questioning Skills

TL;DR

This paper investigates how role-playing with LLM-powered conversational agents can enhance design questioning skills among novices, highlighting benefits and challenges observed in real-world interactions.

Contribution

It introduces a prototype LLM-powered conversational agent for design education and explores its impact on fostering questioning skills in novice students.

Findings

Stimulated questioning behavior among students.

Reduced pressure to ask questions during interactions.

Led to some over-reliance on LLM responses.

Abstract

Identifying design problems is a crucial step for creating plausible solutions, but it is challenging for design novices due to their limited knowledge and experience. Questioning is a promising skill that enables students to independently identify design problems without being passive or relying on instructors. This study explores role-playing interactions with Large Language Model (LLM)-powered Conversational Agents (CAs) to foster the questioning skills of novice design students. We proposed an LLM-powered CA prototype and conducted a preliminary study with 16 novice design students engaged in a real-world design class to observe the interactions between students and the LLM-powered CAs. Our findings indicate that while the CAs stimulated questioning and reduced pressure to ask questions, it also inadvertently led to over-reliance on LLM responses. We proposed design considerations and future works for LLM-powered CA to foster questioning skills.