pix2code: Generating Code from a Graphical User Interface Screenshot

TL;DR

This paper presents a deep learning approach that automatically generates code from GUI screenshots with over 77% accuracy across iOS, Android, and web platforms, streamlining the development process.

Contribution

It introduces an end-to-end deep learning model capable of converting GUI images into code, a novel application in automated UI code generation.

Findings

Achieved over 77% accuracy in code generation

Effective across three platforms: iOS, Android, web

Demonstrated feasibility of deep learning for UI-to-code conversion

Abstract

Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications. In this paper, we show that deep learning methods can be leveraged to train a model end-to-end to automatically generate code from a single input image with over 77% of accuracy for three different platforms (i.e. iOS, Android and web-based technologies).