Gallery D.C.: Auto-created GUI Component Gallery for Design Search and Knowledge Discovery

TL;DR

Gallery D.C. is a web tool that uses deep learning and reverse engineering to index and search GUI components from real applications, aiding designers in inspiration, detailed search, and knowledge discovery.

Contribution

It introduces a novel GUI component gallery that combines computer vision and reverse engineering for advanced design search and knowledge extraction.

Findings

Effective indexing of GUI components from millions of applications.

Enhanced design search with multi-faceted filtering capabilities.

Positive informal feedback from professional designers.

Abstract

GUI design is an integral part of software development. The process of designing a mobile application typically starts with the ideation and inspiration search from existing designs. However, existing information-retrieval based, and database-query based methods cannot efficiently gain inspirations in three requirements: design practicality, design granularity and design knowledge discovery. In this paper we propose a web application, called \tool that aims to facilitate the process of user interface design through real world GUI component search. Gallery D.C. indexes GUI component designs using reverse engineering and deep learning based computer vision techniques on millions of real world applications. To perform an advanced design search and knowledge discovery, our approach extracts information about size, color, component type, and text information to help designers explore multi-faceted design space and distill higher-order of design knowledge. Gallery D.C. is well received via an informal evaluation with 7 professional designers. Web Link: http://mui-collection.herokuapp.com/. Demo Video Link: https://youtu.be/zVmsz_wY5OQ.