Conceptual Design Generation Using Large Language Models

TL;DR

This paper explores how large language models can generate design solutions for various problems, comparing their outputs to crowdsourcing, and evaluates their feasibility, usefulness, and novelty through expert and computational assessments.

Contribution

It demonstrates the application of LLMs to design problem solving and evaluates the impact of prompt engineering on solution quality, providing insights for practical use in design tasks.

Findings

LLM-generated solutions have higher feasibility and usefulness.

Crowdsourced solutions exhibit more novelty.

Few-shot prompting aligns LLM outputs closer to crowdsourced solutions.

Abstract

Concept generation is a creative step in the conceptual design phase, where designers often turn to brainstorming, mindmapping, or crowdsourcing design ideas to complement their own knowledge of the domain. Recent advances in natural language processing (NLP) and machine learning (ML) have led to the rise of Large Language Models (LLMs) capable of generating seemingly creative outputs from textual prompts. The success of these models has led to their integration and application across a variety of domains, including art, entertainment, and other creative work. In this paper, we leverage LLMs to generate solutions for a set of 12 design problems and compare them to a baseline of crowdsourced solutions. We evaluate the differences between generated and crowdsourced design solutions through multiple perspectives, including human expert evaluations and computational metrics. Expert evaluations indicate that the LLM-generated solutions have higher average feasibility and usefulness while the crowdsourced solutions have more novelty. We experiment with prompt engineering and find that leveraging few-shot learning can lead to the generation of solutions that are more similar to the crowdsourced solutions. These findings provide insight into the quality of design solutions generated with LLMs and begins to evaluate prompt engineering techniques that could be leveraged by practitioners to generate higher-quality design solutions synergistically with LLMs.