ColorGPT: Leveraging Large Language Models for Multimodal Color Recommendation

TL;DR

ColorGPT leverages pretrained large language models and prompt engineering to improve color recommendation and palette generation for design, outperforming existing methods in accuracy, diversity, and similarity.

Contribution

This paper introduces ColorGPT, a novel LLM-based pipeline that enhances color recommendation and palette generation through systematic testing and prompt optimization.

Findings

Outperforms existing methods in color suggestion accuracy

Improves diversity and similarity in palette generation

Demonstrates effectiveness of LLMs in color design tasks

Abstract

Colors play a crucial role in the design of vector graphic documents by enhancing visual appeal, facilitating communication, improving usability, and ensuring accessibility. In this context, color recommendation involves suggesting appropriate colors to complete or refine a design when one or more colors are missing or require alteration. Traditional methods often struggled with these challenges due to the complex nature of color design and the limited data availability. In this study, we explored the use of pretrained Large Language Models (LLMs) and their commonsense reasoning capabilities for color recommendation, raising the question: Can pretrained LLMs serve as superior designers for color recommendation tasks? To investigate this, we developed a robust, rigorously validated pipeline, ColorGPT, that was built by systematically testing multiple color representations and applying effective prompt engineering techniques. Our approach primarily targeted color palette completion by recommending colors based on a set of given colors and accompanying context. Moreover, our method can be extended to full palette generation, producing an entire color palette corresponding to a provided textual description. Experimental results demonstrated that our LLM-based pipeline outperformed existing methods in terms of color suggestion accuracy and the distribution of colors in the color palette completion task. For the full palette generation task, our approach also yielded improvements in color diversity and similarity compared to current techniques.