A Pairwise Dataset for GUI Conversion and Retrieval between Android Phones and Tablets

TL;DR

This paper introduces the Papt dataset, a large collection of paired Android phone and tablet GUIs, to facilitate deep learning-based automated GUI conversion and retrieval, addressing a key resource gap.

Contribution

The paper presents the first publicly available pairwise GUI dataset for Android phones and tablets, including data collection methods and analysis, to support deep learning in GUI development.

Findings

The dataset contains 10,035 phone-tablet GUI pairs from 5,593 app pairs.

Preliminary experiments reveal challenges in applying deep learning to GUI automation.

The dataset can aid in developing models for GUI conversion and retrieval tasks.

Abstract

With the popularity of smartphones and tablets, users have become accustomed to using different devices for different tasks, such as using their phones to play games and tablets to watch movies. To conquer the market, one app is often available on both smartphones and tablets. However, although one app has similar graphic user interfaces (GUIs) and functionalities on phone and tablet, current app developers typically start from scratch when developing a tablet-compatible version of their app, which drives up development costs and wastes existing design resources. Researchers are attempting to employ deep learning in automated GUIs development to enhance developers' productivity. Deep learning models rely heavily on high-quality datasets. There are currently several publicly accessible GUI page datasets for phones, but none for pairwise GUIs between phones and tablets. This poses a significant barrier to the employment of deep learning in automated GUI development. In this paper, we collect and make public the Papt dataset, which is a pairwise dataset for GUI conversion and retrieval between Android phones and tablets. The dataset contains 10,035 phone-tablet GUI page pairs from 5,593 phone-tablet app pairs. We illustrate the approaches of collecting pairwise data and statistical analysis of this dataset. We also illustrate the advantages of our dataset compared to other current datasets. Through preliminary experiments on this dataset, we analyse the present challenges of utilising deep learning in automated GUI development and find that our dataset can assist the application of some deep learning models to tasks involving automatic GUI development.