Cooperative Design Optimization through Natural Language Interaction

TL;DR

This paper introduces a cooperative design optimization framework that combines system-led Bayesian methods with natural language interaction via Large Language Models, enhancing user agency and reducing cognitive load.

Contribution

It presents a novel integration of LLMs with Bayesian optimization to enable natural language-based user intervention in the design process.

Findings

Higher user agency compared to purely system-led methods

Promising optimization performance relative to manual design

Lower cognitive load than existing cooperative methods

Abstract

Designing successful interactions requires identifying optimal design parameters. To do so, designers often conduct iterative user testing and exploratory trial-and-error. This involves balancing multiple objectives in a high-dimensional space, making the process time-consuming and cognitively demanding. System-led optimization methods, such as those based on Bayesian optimization, can determine for designers which parameters to test next. However, they offer limited opportunities for designers to intervene in the optimization process, negatively impacting the designer's experience. We propose a design optimization framework that enables natural language interactions between designers and the optimization system, facilitating cooperative design optimization. This is achieved by integrating system-led optimization methods with Large Language Models (LLMs), allowing designers to intervene in the optimization process and better understand the system's reasoning. Experimental results show that our method provides higher user agency than a system-led method and shows promising optimization performance compared to manual design. It also matches the performance of an existing cooperative method with lower cognitive load.