Exploring the Potential of Metacognitive Support Agents for Human-AI Co-Creation

TL;DR

This paper investigates how metacognitive support agents can enhance human-AI co-creation in design by addressing cognitive challenges, demonstrating improved design feasibility through prototyping with mechanical designers.

Contribution

It introduces novel metacognitive support strategies for AI design tools and evaluates their impact through an exploratory Wizard of Oz study.

Findings

Supported users created more feasible designs

Different support strategies had varying impacts

Highlights opportunities and tradeoffs in metacognitive support

Abstract

Despite the potential of generative AI (GenAI) design tools to enhance design processes, professionals often struggle to integrate AI into their workflows. Fundamental cognitive challenges include the need to specify all design criteria as distinct parameters upfront (intent formulation) and designers' reduced cognitive involvement in the design process due to cognitive offloading, which can lead to insufficient problem exploration, underspecification, and limited ability to evaluate outcomes. Motivated by these challenges, we envision novel metacognitive support agents that assist designers in working more reflectively with GenAI. To explore this vision, we conducted exploratory prototyping through a Wizard of Oz elicitation study with 20 mechanical designers probing multiple metacognitive support strategies. We found that agent-supported users created more feasible designs than non-supported users, with differing impacts between support strategies. Based on these findings, we discuss opportunities and tradeoffs of metacognitive support agents and considerations for future AI-based design tools.