A Systematic Mapping Study Addressing the Reliability of Mobile Applications: The Need to Move Beyond Testing Reliability

TL;DR

This systematic mapping study reviews the current state of research on mobile app reliability, highlighting significant gaps in understanding context-awareness, self-healing, and real-world application, and emphasizing the need for standardized evaluation methods.

Contribution

The paper provides a comprehensive classification of existing research on mobile app reliability and identifies key gaps and opportunities for future investigation.

Findings

Limited research on context-aware reliability and self-healing.

Few studies evaluate reliability in real-world industrial settings.

Lack of standardized quality metrics in reliability research.

Abstract

Intense competition in the mobile apps market means it is important to maintain high levels of app reliability to avoid losing users. Yet despite its importance, app reliability is underexplored in the research literature. To address this need, we identify, analyse, and classify the state-of-the-art in the field of mobile apps' reliability through a systematic mapping study. From the results of such a study, researchers in the field can identify pressing research gaps, and developers can gain knowledge about existing solutions, to potentially leverage them in practice. We found 87 relevant papers which were then analysed and classified based on their research focus, research type, contribution, research method, study settings, data, quality attributes and metrics used. Results indicate that there is a lack of research on understanding reliability with regard to context-awareness, self-healing, ageing and rejuvenation, and runtime event handling. These aspects have rarely been studied, or if studied, there is limited evaluation. We also identified several other research gaps including the need to conduct more research in real-world industrial projects. Furthermore, little attention has been paid towards quality standards while conducting research. Outcomes here show numerous opportunities for greater research depth and breadth on mobile app reliability.