Step-by-step Layered Design Generation

TL;DR

This paper introduces a new machine learning framework called SLEDGE for step-by-step layered design generation, modeling iterative design modifications as layered changes guided by instructions, supported by a new dataset and benchmark.

Contribution

The paper proposes a novel problem setting and a corresponding model, SLEDGE, for layered, instruction-guided design generation, along with a new evaluation suite and benchmark.

Findings

SLEDGE outperforms existing approaches in layered design tasks.

The new dataset enables comprehensive evaluation of step-by-step design generation.

Experimental results validate the effectiveness of layered, instruction-guided design modeling.

Abstract

Design generation, in its essence, is a step-by-step process where designers progressively refine and enhance their work through careful modifications. Despite this fundamental characteristic, existing approaches mainly treat design synthesis as a single-step generation problem, significantly underestimating the inherent complexity of the creative process. To bridge this gap, we propose a novel problem setting called Step-by-Step Layered Design Generation, which tasks a machine learning model with generating a design that adheres to a sequence of instructions from a designer. Leveraging recent advancements in multi-modal LLMs, we propose SLEDGE: Step-by-step LayEred Design GEnerator to model each update to a design as an atomic, layered change over its previous state, while being grounded in the instruction. To complement our new problem setting, we introduce a new evaluation suite, including a dataset and a benchmark. Our exhaustive experimental analysis and comparison with state-of-the-art approaches tailored to our new setup demonstrate the efficacy of our approach. We hope our work will attract attention to this pragmatic and under-explored research area.