Moderately Mighty: To What Extent Can Internal Software Metrics Predict App Popularity at Launch?

TL;DR

This study investigates whether internal software metrics from source code can predict the initial popularity of mobile apps, finding they are more useful for classification than regression tasks.

Contribution

It demonstrates that internal code metrics alone are insufficient for precise predictions but can effectively classify apps as popular or unpopular.

Findings

Regression models perform poorly due to data skewness.

Multilayer Perceptron achieves an F1-score of 0.72 in classification.

Internal metrics show meaningful correlations with app popularity.

Abstract

Predicting a mobile app's popularity before its first release can provide developers with a strategic advantage in a competitive marketplace, yet it remains a challenging problem. This study explores the extent to which internal software metrics, measurable from source code before deployment, can predict an app's popularity (i.e., ratings and downloads per year) at inception. For our analysis, we constructed a rigorously filtered dataset of 446 open-source Java-based Android apps that are available on both F-Droid and Google Play Store. Using app source code from F-Droid, we extracted a wide array of internal metrics, including system-, class-, and method-level code metrics, code smells, and app metadata. Popularity-related information, including reviews and download counts, was collected from the Play Store. We evaluate regression and classification models across three feature sets: a minimal Size-only baseline, a domain-informed Handpicked set, and a Voting set derived via feature selection algorithms. Our results show that, for both app ratings and number of downloads, regression models perform poorly due to skewed rating distributions and a highly scattered range of download counts in our dataset. However, when reframed as a binary classification (Popular vs. Unpopular), performance improves significantly-the best model, a Multilayer Perceptron, achieves an F1-score of 0.72. We conclude that, although internal code metrics alone are insufficient for accurately predicting an app's future popularity, they do exhibit meaningful correlations with it. Thus, our findings challenge prior studies that have entirely dismissed internal metrics as valid indicators of software quality. Instead, our results align with research suggesting that internal code metrics can be valuable when evaluated within the appropriate context-specifically, we found them useful for classification tasks.