SketchConcept: Sketching-based Concept Recomposition for Product Design using Generative AI

TL;DR

SketchConcept is a novel AI-powered tool that combines sketches and textual descriptions to facilitate the exploration and recomposition of product design concepts, integrating multimodal generative AI for enhanced functionality.

Contribution

It introduces a function-to-visual mapping workflow that leverages large language models and image GenAI to decompose, generate, and edit design concepts based on functions.

Findings

Validated workflow through multiple use cases

Demonstrated system's ability to decompose and generate design concepts

User study confirmed system's efficacy and usability

Abstract

Conceptual product design requires designers to explore the design space of visual and functional concepts simultaneously. Sketching has long been adopted to empower concept exploration. However, current sketch-based design tools mostly emphasize visual design using emerging techniques. We present SketchConcept, a design support tool that decomposes design concepts into visual representations and functionality of concepts using sketches and textual descriptions. We propose a function-to-visual mapping workflow that maps the function descriptions generated by a Large Language Model to a component of the concept produced by image Generative Artificial Intelligence(GenAI). The function-to-visual mapping allows our system to leverage multimodal GenAI to decompose, generate, and edit the design concept to satisfy the overall function and behavior. We present multiple use cases enabled by SketchConcept to validate the workflow. Finally, we evaluated the efficacy and usability of our system with a two-session user study.