Plot2API: Recommending Graphic API from Plot via Semantic Parsing Guided Neural Network

TL;DR

This paper introduces a novel deep learning approach called SPGNN for recommending graphic APIs from plots, addressing challenges like varied appearances and data imbalance, and demonstrates its effectiveness on new datasets.

Contribution

The paper proposes SPGNN, a multi-task neural network that combines semantic parsing with image classification for improved Plot2API recommendations, and provides new datasets for evaluation.

Findings

SPGNN outperforms recent deep learning baselines.

Semantic parsing enhances feature learning by focusing on relevant visual information.

Data augmentation with random erasing alleviates API category imbalance.

Abstract

Plot-based Graphic API recommendation (Plot2API) is an unstudied but meaningful issue, which has several important applications in the context of software engineering and data visualization, such as the plotting guidance of the beginner, graphic API correlation analysis, and code conversion for plotting. Plot2API is a very challenging task, since each plot is often associated with multiple APIs and the appearances of the graphics drawn by the same API can be extremely varied due to the different settings of the parameters. Additionally, the samples of different APIs also suffer from extremely imbalanced. Considering the lack of technologies in Plot2API, we present a novel deep multi-task learning approach named Semantic Parsing Guided Neural Network (SPGNN) which translates the Plot2API issue as a multi-label image classification and an image semantic parsing tasks for the solution. In SPGNN, the recently advanced Convolutional Neural Network (CNN) named EfficientNet is employed as the backbone network for API recommendation. Meanwhile, a semantic parsing module is complemented to exploit the semantic relevant visual information in feature learning and eliminate the appearance-relevant visual information which may confuse the visual-information-based API recommendation. Moreover, the recent data augmentation technique named random erasing is also applied for alleviating the imbalance of API categories. We collect plots with the graphic APIs used to drawn them from Stack Overflow, and release three new Plot2API datasets corresponding to the graphic APIs of R and Python programming languages for evaluating the effectiveness of Plot2API techniques. Extensive experimental results not only demonstrate the superiority of our method over the recent deep learning baselines but also show the practicability of our method in the recommendation of graphic APIs.