PRISM: Learning Design Knowledge from Data for Stylistic Design Improvement

TL;DR

PRISM leverages real-world design data to learn and apply style-specific knowledge, enabling more effective stylistic improvements in graphic design guided by natural language instructions.

Contribution

It introduces PRISM, a novel method that constructs a design knowledge base from data to improve style alignment in graphic design tasks.

Findings

PRISM achieves the highest average rank of 1.49 in style alignment.

User studies show designers prefer PRISM-enhanced designs.

PRISM outperforms baseline methods in style-aware design improvement.

Abstract

Graphic design often involves exploring different stylistic directions, which can be time-consuming for non-experts. We address this problem of stylistically improving designs based on natural language instructions. While VLMs have shown initial success in graphic design, their pretrained knowledge on styles is often too general and misaligned with specific domain data. For example, VLMs may associate minimalism with abstract designs, whereas designers emphasize shape and color choices. Our key insight is to leverage design data -- a collection of real-world designs that implicitly capture designer's principles -- to learn design knowledge and guide stylistic improvement. We propose PRISM (PRior-Informed Stylistic Modification) that constructs and applies a design knowledge base through three stages: (1) clustering high-variance designs to capture diversity within a style, (2) summarizing each cluster into actionable design knowledge, and (3) retrieving relevant knowledge during inference to enable style-aware improvement. Experiments on the Crello dataset show that PRISM achieves the highest average rank of 1.49 (closer to 1 is better) over baselines in style alignment. User studies further validate these results, showing that PRISM is consistently preferred by designers.