Prototype2Code: End-to-end Front-end Code Generation from UI Design Prototypes

TL;DR

Prototype2Code is an end-to-end front-end code generation system that improves robustness, readability, and responsiveness of generated UI code by integrating design linting, hierarchical optimization, and flexbox support, validated through quantitative metrics and user studies.

Contribution

It introduces Prototype2Code, a novel approach that enhances code quality and robustness in UI-to-code generation by incorporating design linting and layout optimization.

Findings

Prototype2Code achieves higher visual similarity to prototypes than competitors.

Generated code is more readable and maintainable according to user evaluations.

Prototype2Code outperforms existing methods in robustness and responsiveness.

Abstract

UI-to-code technology has streamlined the front-end development process, reducing repetitive tasks for engineers. prior research mainly use design prototypes as inputs, with the effectiveness of the generated code heavily dependent on these prototypes' quality, leading to compromised robustness. Moreover, these approaches also exhibit shortcomings in code quality, including issues such as disorganized UI structures and the inability to support responsive layouts. To address these challenges, we introduce Prototype2Code, which achieves end-to-end front-end code generation with business demands. For Prototype2Code, we incorporate design linting into the workflow, addressing the detection of fragmented elements and perceptual groups, enhancing the robustness of the generated outcomes. By optimizing the hierarchical structure and intelligently recognizing UI element types, Prototype2Code generates code that is more readable and structurally clearer. To meet responsive design requirements, Prototype2Code primarily supports flexbox layout model, ensuring code compatibility across various device sizes. To validate the efficacy, we compare Prototype2Code with the commercial code generation platform CodeFun and Screenshot-to-code based on GPT-4 with vision. Employing structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), and mean squared error (MSE) for visual similarity assessment, Prototype2Code's rendered UI effects align most closely with the design prototypes, exhibiting the minimal errors. We also conduct a user study with five experienced front-end engineers, inviting them to review and revise code generated by the three methods. As a result, Prototype2Code surpasses other methods in readability, usability, and maintainability, better meeting the business needs of industrial development.