CoGen: Creation of Reusable UI Components in Figma via Textual Commands

TL;DR

CoGen leverages machine learning, including T5 transformers, to automatically generate reusable UI components in Figma from natural language prompts and structured JSON, streamlining UI design workflows.

Contribution

This work introduces CoGen, a novel system that combines Figma API data, Seq2Seq models, and fine-tuned T5 transformers to automate the creation of atomic UI components from natural language commands.

Findings

T5 model achieves 98% accuracy in prompt generation

JSON creation success rate reaches 100% for simple components

System improves efficiency of UI component design process

Abstract

The evolution of User Interface design has emphasized the need for efficient, reusable, and editable components to ensure an efficient design process. This research introduces CoGen, a system that uses machine learning techniques to generate reusable UI components directly in Figma, one of the most popular UI design tools. Addressing gaps in current systems, CoGen focuses on creating atomic components such as buttons, labels, and input fields using structured JSON and natural language prompts. The project integrates Figma API data extraction, Seq2Seq models, and fine-tuned T5 transformers for component generation. The key results demonstrate the efficiency of the T5 model in prompt generation, with an accuracy of 98% and a BLEU score of 0.2668, which ensures the mapping of JSON to descriptive prompts. For JSON creation, CoGen achieves a success rate of up to 100% in generating simple JSON outputs for specified component types.